# Definitions (v1 exploratory)

All values are computed by `analysis/v1_analysis.py` from the sheet « Données brutes » of `satcove-consensus-benchmark-2026-10-22.xlsx` (read-only, SHA-256 `14c33d4b5730b70f0e1c0f7c17caa2bfe9084ce1f761b45d40db027d12080ced`) plus `analysis/recovered_cells.csv`. That CSV only says which cells of the sheet came from the recovery pass. It was derived from result logs already on disk (`build_recovered_cells.py`); no model, API or service was called. The workbook alone does not say which cells were recovered, so this file is the one input that does not come from the workbook.

**Jury.** The models that gave a valid letter for a question. Up to six models: Claude, GPT, Gemini, Mistral, Perplexity, Grok. Perplexity only answered 533 questions. The jury size is therefore 4, 5 or 6 and differs by question and by view. A question needs a jury of at least 4 to be analysed.

**Valid answer.** A reply that starts with the option letter (strict parser). Prose is never interpreted as an answer. A missing answer is not a wrong answer.

**Majority answer.** The letter given by most jurors. If two letters share the top count, it is a tie and there is no majority answer. Majority accuracy is computed on questions that have a majority; the figure with ties counted as failures is always given next to it.

**Agreement.** Share of the jury that gave the majority letter (top count / jury size).

**Unanimity.** All jurors gave the same letter (jury of at least 4). A unanimity of 5 is not the same bar as a unanimity of 6.

**False consensus.** A unanimous question where the unanimous letter is not the known answer. It is a measured event on this question set, not a general rate for AI.

**Primary protocol.** The first-pass protocol: one letter-only reply per model and question, same prompt for all, one fresh conversation per call. Differences between models: Claude had a system prompt, Perplexity (sonar) has web search, `max_tokens` differs by provider.

**Recovery pass (rattrapage).** A second pass, run only for cells where the primary reply had no valid letter (prose, truncation, API failure). It uses a different protocol per model: Claude forced tool call restricted to the letters (475 cells); Perplexity longer `max_tokens` (86); Gemini enum-constrained output (85); Mistral and Grok plain retry (7 and 1); GPT none. 654 cells in total.

**Primary-only view.** The letters of « Données brutes » with the 654 recovered cells removed. The jury shrinks, some questions drop below 4 jurors (2,460 questions remain). Warning: missing answers are not random, they cluster on hard questions, so this view is easier than the full set.

**Recovered answers.** The 654 cells above, analysed on their own.

**Final score.** Primary answers plus recovered answers (the workbook's published numbers). 2,500 questions, jury of 5 for 1,965, 6 for 533, 4 for 2.

**Equal footprint.** Models compared only on the same questions.

**Constant jury.** Same models for every question analysed (5 without Perplexity; 6 on the 533 Perplexity questions).

**Status labels used in `v1_publishable_claims.csv`.** PUBLISHABLE: reproduced, definition clear. NEEDS_CAVEAT: reproduced, must be published with the stated limit. DO_NOT_PUBLISH: not reproducible with a clear definition, confounded, or misleading.