# Reproduce the editorial retrieval experiment

This experiment uses fictional project records. It measures source retrieval and citation checks. It does not measure generated answers or agent sessions.

## Requirements

Use Python 3.11 with the dependencies required by your Gigabrain archive. Keep the purchased archive available locally. Prepare the local model in a separate vault through its documented setup. The recorded run used the Full 1.1.0 archive. The metadata file identifies its hash and runtime.

Download the tools bundle from https://gigabrain.download/assets/resources/editorial-experiment-tools.zip. Extract it into a separate working directory. The bundle contains evaluation scripts and fictional case definitions. It does not contain the paid product.

## Run

1. Open a terminal in the extracted tools directory.
2. Replace the archive path with your local archive.
3. Replace the model-seed path with your prepared test vault.
4. Choose an output directory that does not exist.
5. Run the command below.

```sh
python3.11 dev/public_editorial_experiment.py \
  --archive /path/to/gigabrain-full-1.1.0.zip \
  --model-seed /path/to/prepared-test-vault \
  --output /path/to/new-results
```

The runner extracts a separate product copy into temporary storage. It copies the local model files and executes the four retrieval modes. It preserves source files, result details, a summary and provenance metadata in the output directory.

## Interpret the result

Compare identical questions across modes. Inspect every required source and any irrelevant results. The fixture has two corpora with seven queries each. Six queries in each corpus have labelled supporting sources. One has no supporting source.

The split called `held-out` was inspected during earlier development. Treat it as regression data. It is not a blind validation set for this rerun.

Citation verification checks the recorded source. It does not prove that the source answers the question. In the published rerun, every mode returned passages for the unsupported query. Keep that failure visible.

Recorded source files: https://gigabrain.download/assets/evidence/editorial-experiment/sources.json
Recorded results: https://gigabrain.download/assets/evidence/editorial-experiment/results.json
Runtime and archive: https://gigabrain.download/assets/evidence/editorial-experiment/metadata.json