Open dataset · CC BY 4.0
How AI models read the news
For each Satcove News story, several AI models read the same newsroom reports separately. We record what each model covered, what it left out and which claims the reports did not support. This page aggregates those scorecards and publishes the raw data.
What this is not
It measures how a model read the sources, not whether it is accurate in general or whether a claim is true. With 2 stories so far, differences between models are not statistically meaningful. The dataset grows with each published story.
Per model, so far
2 scored stories, 2026-10-15 to 2026-10-15.
| Model | Stories | Mean coverage | Critical omissions | Unsupported claims | Contradictions |
|---|---|---|---|---|---|
| Claude | 2 | 86% | 2 | 1 | 0 |
| GPT | 2 | 59% | 2 | 0 | 0 |
| Gemini | 2 | 82% | 0 | 0 | 0 |
| Mistral | 2 | 78% | 2 | 1 | 0 |
| Perplexity | 2 | 78% | 0 | 0 | 0 |
| Grok | 2 | 82% | 0 | 0 | 0 |
Stories in the dataset
- Pope Leo diagnosed with lung cancer after surgery: what six AIs agree on, and what they added or left out
2026-10-15 · world · panel agreement 93% · Al Jazeera, Irish Examiner, NPR
- OpenAI engineer charged over hidden listening device: what six AIs agree on, and what they left out
2026-10-15 · technology · panel agreement 91% · NBC News, Al Jazeera, CBS News
Method
- A story is published only when at least three recognised newsrooms report it. Each model receives the same extracted reports.
- Models answer independently and do not see each other's output. Models that fail or time out are recorded as skipped, not hidden.
- Coverage is the share of the panel's points that a model included, within the number of claims it was asked for.
- A claim counts as unsupported when its supporting quote is not found in the reports. Quotes are matched verbatim by a script.
- The stories are automated analyses. Treat them as a reading aid and check the linked reports before relying on a fact.
Download and cite
data.jsonLicence: CC BY 4.0. Please credit Satcove and link to this page.
Satcove. Satcove News dataset, version 1.0. https://satcove.com/news/data