Six AIs on Four Middle East Risks by End-2027: Gaza, Lebanon, Iran and Saudi-Israel, Three Runs Each

In April, a Satcove post asked six AIs whether there would be peace in the Middle East by 2027. It reported a "42% agreement" score without saying what the score measured. This article does the test again, properly: same question, six models, a fact dossier, six dated sub-questions, a probability for each, and three runs per model so that run-to-run noise can be seen.

Scope. "Peace" is a loaded word. This article does not cover the West Bank, Syria, Yemen, Iraq or wider Arab-Israeli relations; it tests four tracks only: Gaza, Lebanon, US-Iran and Saudi-Israel. It does not predict the Middle East, takes no political position and makes no operational recommendation. It separates verifiable facts, unverified allegations and model interpretations, and it does not treat contested claims as equivalent to established ones. Its subject is how six AIs reason when the facts are thin and the stakes are high. The earlier post is here: Will There Be Peace in the Middle East by 2027? 6 AIs Weigh In.

The protocol

On 6 November 2026 we compiled a dossier of dated facts from secondary sources only: Wikipedia pages, one Al Jazeera headline and a web-search summary. No primary document was opened. Every fact carries a label: secondary, unverified or contradictory. The dossier says plainly that the US-Iran truce status after 3 November and the Lebanon status after 21 October are unknown to us.

We sent the same dossier and the same questions to six models, three times each:

• Run 1 through the API, with the identifiers below.
• Runs 2 and 3 through each vendor's public chat app, in a fresh conversation each time, pasting the identical prompt.

The chat apps may run a different model version, system prompt, temperature or tool configuration than the API. We did not control them, so run 1 and runs 2-3 are not strictly the same experiment. We report all three runs and use the median so that one odd run does not decide a figure.

Rules given to the models: use only the dossier for 2025-2026 facts; mark any other fact "from model memory, unverified"; separate facts, estimates and interpretations; give a probability, a short reason and two measurable indicators for each item; avoid reusing the same split across items.

The six items, all resolved on 31 December 2027:

• S1. The Gaza ceasefire is still in force, with no Israeli ground offensive lasting 30 consecutive days.
• S2. An Israel-Lebanon agreement is formally signed and implementation has started.
• S3. Israel and Saudi Arabia have formally normalized relations.
• S4. The US and Iran have a written agreement and the Strait of Hormuz is open to commercial traffic.
• S5. There is no direct Israel-Iran military strike during 2027.
• S6. S1, S2, S4 and S5 are all true: our working definition of "comprehensive regional peace".

Limits that apply throughout: three runs are better than one but still a small sample; only Perplexity has live web access; the probabilities are each model's opinion on one date, not forecasts.

What the six answered

Median of three runs, with the range of the three runs in brackets. Probability in %.

The three individual runs for every model are kept in the raw files; the full set is available on request. Two things stand out. On the verdict, the six agree. On almost everything underneath it, they do not, and sometimes a model does not agree with itself.

First observation: the verdict converges

Every model's median calls comprehensive regional peace by end-2027 unlikely. The six medians run from 1.5% to 9%, and no single run of any model exceeds 20%. In the first (API) run, four models gave Low confidence in their own verdict, one gave Medium, and Grok's answer was conditional ("Medium if primary documents later confirm..."). We did not tabulate confidence labels for runs 2 and 3. Having read them, we note only that Gemini's Q10 label changed between runs and that Perplexity gave two-level labels in the three chats we read (for instance "Medium in direction, low in precise probabilities"), where one level was requested.

Asked what changed since the April post, in the first run no model became more optimistic:

• ChatGPT: "down modestly".
• Gemini: down, because the conflict has widened to direct Israel-Iran strikes and Saudi territory.
• Claude and Grok: unchanged to slightly down.
• Mistral and Perplexity: unchanged, though Mistral's own text calls the outlook "slightly more pessimistic".

The chat runs, read afterwards, are not identical on this. Gemini again said down. Claude said unchanged to slightly down, and Grok unchanged. Mistral said down or marginally down. Perplexity said unchanged to marginally lower in one run and "slightly down, not up" in the other. ChatGPT is the exception: in its chat runs it said slightly up. So "no model more optimistic" holds for the first run only.

This agreement deserves some suspicion. All six read the same dossier, and the dossier is itself pessimistic: repeated ceasefires, a collapse on 8 July, Hezbollah rejecting the June framework, no Saudi progress. Six models agreeing on a conclusion that the shared input points to is weaker evidence than six models agreeing from independent sources.

Second observation: the items split them, and so do the runs

The widest gap between model medians is S5, the absence of a direct Israel-Iran strike in 2027: Claude and Gemini 20%, ChatGPT 57%, a 37-point spread. S4 (US-Iran deal) spans 26 points, S2 (Lebanon) 21, S3 (Saudi normalization) 15 and S1 (Gaza) 10.

The first single-run version of this table showed a 50-point S5 gap, 20% (Claude) against 70% (Mistral). Repeating the runs shrinks it: Mistral's two later runs gave 28% and 25% on S5, close to the panel median. The 70% was the extreme of a noisy distribution, not Mistral's stable view. The same happened elsewhere: Mistral's first run gave 60% on S1 and 50% on S4, against medians of 35% and 12%. Gemini's S1 fell from 55% to 35% in both later runs, while its S2 jumped between 35%, 15% and 40%. ChatGPT's S5 ranged from 37% to 61%.

Model-to-model differences that survive the three runs:

• Claude stays low on Lebanon, Saudi normalization and Iran-related items (S2 14, S3 3, S4 11, S5 20).
• ChatGPT stays high on S4 and S5 (37 and 57).
• Perplexity and ChatGPT lead on S1 (Gaza ceasefire), at 45 and 43.

The models do not share one template for what is most likely, by median:

• Gaza ceasefire is the most likely item: Claude (38), Gemini (35), Mistral (35), Perplexity (45).
• No Iran strike is the most likely item: ChatGPT (57) and Grok (41).
• The Lebanon deal is not the top item for any model. In the first run ChatGPT put it first (47%); in its two later runs it gave 32% and 29%.

That is unlike our tech selloff test, where six AIs landed on near-identical probability splits. Here the instruction to avoid reusing a template, and a dated, resolvable question, produced real differences. Whether those differences carry information is another matter, and the next sections look at it.

Iran: the swing fact nobody verified

The S5 and S4 spread comes from one unverified headline. The dossier records an Al Jazeera headline of 6 November about a two-week US-Iran ceasefire conditional on shipping resuming through Hormuz. We did not read the article. The dossier also records the 17 June "Islamabad Memorandum", the 8 July resumption of fighting and, on 3 November, an Iranian statement that the supreme leader gave the government more latitude for talks. The July collapse and the November ceasefire are not reconciled.

How the models used this, from the first (API) run, whose text we read in full:

• Claude (S5 20%, S4 18%) weighs the frequency of 2026 strikes and says the November truce status is unknown. It also reasons in steps: about 35% for a written deal times about 55% for Hormuz open gives roughly 19%, shaded to 18%.
• Mistral (S5 70%, S4 50%) read the November ceasefire as de-escalation. Its S5 text said "no direct Israel-Iran strikes since June 2026". The dossier does not say that: it lists missiles at Israel on 7 June and resumed fighting on 8 July, and Mistral itself cited the July collapse in the same answer. Its 70% rested on a premise the dossier does not support, and its later runs did not repeat it (28% and 25%). We flag it rather than discard it.
• Gemini (S5 48%) called this a "coin-flip" tied to Iranian leadership transitions. In both chat runs it gave 20%.
• Grok (S5 41%) said it derived its figures by subtracting 15-20 points from a "neutral 45%" start to reflect gaps in the dossier. That is a stated method, and an arbitrary one. Its later runs stayed at 40% and 48%.

Gaza, Lebanon, Saudi Arabia

On Gaza (S1), the dossier describes the ceasefire as nominal since 10 October 2025, with Israel holding about half the enclave and Hamas apparently rejecting the Mladenov demilitarization plan (a single, unverified summary). Medians run from 35% (Gemini, Grok, Mistral) to 45% (Perplexity), the narrowest spread of the six items.

On Lebanon (S2), ChatGPT's median 32% is far above Claude's 14%. The difference is how each reads "signed": the dossier reports a framework announced on 26 June (27 June in another page) which Hezbollah rejected, with talks in Rome and a military-technical meeting in Tampa on 20-21 October. In the first run Claude said it did not know whether the framework counts as a signed text.

On Saudi normalization (S3), the dossier has no 2026 progress. The crown prince ties it to a Palestinian state, and Saudi Arabia has itself been under Houthi pressure. The medians run from 3% (Claude) to 18% (Grok), a sixfold gap on a question where nobody has new evidence. Mistral's first run (30%) was the outlier.

Third observation: the joint number hides an assumption

S6 is the logical conjunction of S1, S2, S4 and S5. If the four were independent, the joint probability would be the product of the four. We computed that product from each model's own median figures:

All six give a joint figure above the independent product, by a factor of about 2.4 (ChatGPT) to 19 (Mistral). Medians of separate items are not from one coherent run, so this is approximate. The four events are plainly dependent, so the product is not the right benchmark; it is only a floor that shows how much correlation each model assumed. A better design would ask each model to state its dependence structure and give a low/high range for S6 instead of one number. We did not, and note it as a gap. The position itself is reasonable: if a US-Iran deal holds, the Lebanon track gets easier, and the tracks are linked. ChatGPT and Claude said so explicitly in the first run. But it means each model assumed positive correlation without being asked to.

Grok is internally inconsistent: in its first run its S6 text referred to "independent barriers", yet its 9% is about ten times the independent product of its own four numbers (0.9%). Both cannot hold. In its other two runs the same gap persists (S6 of 4% and 9% against products of 0.6% and 2.0%); we have since read those runs and their S6 reasoning is qualitative rather than arithmetic, so the inconsistency is not resolved. By contrast Claude's S6 (about 1 to 1.5%) comes from explicit correlated arithmetic, and Mistral's from a stated product that it then raised for correlation. Perplexity's two chat runs also reason qualitatively ("conjunction of the four, hence 3%" in one). This is a consistency check on stated numbers, not a verdict on who is right.

What each model said it was least sure of

From the first (API) run:

• ChatGPT: S4 and S5, because coverage after 3 November is thin, and S6 which inherits them.
• Perplexity: S1, S2, S3 and S5, whose dossier inputs are secondary or unverified, and S6 which compounds them.
• Claude: S2, S4 and S1, because definitions are ambiguous and the latest facts are missing.
• Grok: S4 and S6, which rest on "a single unread Al Jazeera headline".
• Mistral: S2 and S6.
• Gemini: S4, because the dossier says the truce status is "UNKNOWN".

In the chat runs, Perplexity named S2 as least reliable and then S4. The runs add evidence on this: the items where a model moved most between runs were S2, S4 and S5 for Mistral, S5 for ChatGPT and S1, S2 and S5 for Gemini. These are also the items the models themselves flagged as shaky.

Missing facts, as named by the models in the first run: the text and status of the reported November US-Iran ceasefire and verified Hormuz traffic (four of six); the Israel-Lebanon negotiating text and field records (four of six); primary data on Gaza violations and force posture (four of six). Gemini also asked for the identity of Iran's next supreme leader and the Israeli election outcome.

Perplexity, the only model with live web access, wrote in its first run that the April post's 42% "was a model-consensus statistic, not a probability" and attached footnote markers. The saved output has no source list behind those markers, and we have not verified the claim. ChatGPT and Claude, working from the dossier, say only that the score had no stated method. We have since read both Perplexity Mideast chat runs: they show an empty Sources panel, answer from the dossier alone and say they cannot verify the November facts. Its quantum chat runs behaved differently (web sources shown), so web use is inconsistent across topics.

What changed since April

Facts from the dossier, not from the AIs:

• The Iran war began on 28 February 2026, with the supreme leader reported killed and Hormuz closed.
• US-Iran ceasefires were announced in April and June, extended and disputed. Fighting resumed on 8 July. A two-week ceasefire was reported on 6 November.
• Israel-Lebanon moved from first direct contacts on 14 April to a framework in late June, which Hezbollah rejected, and technical talks in October.
• Gaza: nominal ceasefire since October 2025; no confirmed political settlement.
• Saudi Arabia: no normalization progress found, and reported Houthi pressure since July.

Compared with April, the number of diplomatic channels has grown and the number of reversals has too. Each model reached the same reading: more activity, no more structure.

What these numbers do not show

Agreement is not validation. The six models read one dossier. Where they converge (a low joint figure, a pessimistic reading since April), the convergence measures how strongly the dossier frames the question, not whether the framing is right. "All six agree" is a statement about the panel, never evidence about the world.

S6 is not a measure of peace. It is the conjunction of four conditions, each uncertain, so it sits near zero by construction. It measures "everything holds at once", not "peace". The four items also mix three different things: a ceasefire (S1), a de-escalation with Iran (S4, S5) and a political settlement (S2, S3). We did not build a separate de-escalation score. Readers should look at S1 to S5 individually and treat S6 only as a stress test of the correlation assumption.

Why a model moves is an open question. Claude's figures were among the steadier across runs and it showed stepwise arithmetic in its first answer; Mistral's first-run 70% rested on a premise absent from the dossier and did not repeat. Reading the chat runs fits this: ChatGPT (at most 6 points between runs) and Claude (at most 8) were the steadiest, Grok and Mistral gaps reached 13 and tens of points, and Gemini had large gaps (S2 15 against 40). These are observations, not a mechanism. A model's stability could reflect the quality of its reasoning, or only its sampling settings and interface mode. With three runs and uncontrolled chat settings we cannot tell the two apart, and we do not claim to.

Predictive value is untested. We show that noise exists. We do not show that the signal does. Settling it would need either many more runs, a comparison with forecasts by human experts, or a backtest: freeze a dossier at an earlier date, ask questions that have since resolved, and score the models against outcomes. We have done none of these.

Weighting. Models are not weighted by protocol compliance or by stability. Every model counts equally in the medians, including those that used premises outside the dossier.

The practical rule

Use these figures as a map of what each item depends on, not as odds. Four habits help:

1. Prefer resolvable questions. "Peace by 2027" cannot be scored. "Is the Gaza ceasefire still in force on 31 December 2027, with no 30-day ground offensive" can.
2. Read the spread, not the average. A median of 29% on S5 conceals model medians of 20% and 57%, and single runs of 70%. The spread shows which fact is carrying the answer.
3. Never trust one run. One AI answer is one draw from a distribution. Here a model's range across its own three runs was often as wide as the gap between models.
4. Check the swing fact. Here it is the status of the November US-Iran ceasefire. Until someone reads a primary text, every Iran figure above is conditional.

Indicators to watch

Named by the models and kept for the update log:

• Gaza: a verified Hamas answer on demilitarization; Israeli mobilization or evacuation orders pointing to a multi-week offensive.
• Lebanon: a published text signed by both governments; verified Israeli pullbacks matched by Lebanese army deployment at named locations; Hezbollah's position on the disarmament clause.
• Iran: the authenticated text of the November arrangement; lifting of the US port blockade; sustained commercial transits through Hormuz.
• Israel-Iran: any confirmed direct missile, drone or air strike; a formal deconfliction arrangement.
• Saudi Arabia: a public statement softening the Palestinian-state condition; a credible Gaza pathway to statehood.

Method and limits

• Six models answered the same dossier and the same questions on 6 November 2026, three times each, with identical rules: run 1 through the API, runs 2 and 3 through the vendors' chat apps in a fresh conversation. Model identifiers for run 1 are listed above; the API returned all six with status 200. For runs 2 and 3 we recorded the mode shown by each app (table above); the exact underlying model version and any hidden system prompt are not known to us.
• The three runs are not independent draws from one fixed setup: run 1 and runs 2-3 may differ in model, settings and tools. We use the median and the range, and we do not compute confidence intervals on three points.
• Chat apps have conversation memory features on some accounts. We used a fresh conversation each time but cannot rule out account-level personalization.
• ChatGPT, Claude, Gemini, Grok and Mistral worked from the dossier only; Perplexity can search the web. In its first run it said it did not use facts outside the dossier; its two Mideast chat runs showed no web sources and said the same.
• The dossier is thin for questions of war, blockade, ceasefire and negotiation: Wikipedia pages, one Al Jazeera headline and a search summary. The most decisive recent facts are the ones missing. It comes from secondary sources and no primary document (UN, governments, ministries) was opened. Items after late October are thinly covered. A reported unread headline carries the November US-Iran ceasefire.
• The probabilities are opinions produced by language models on a single date. They are not forecasts, not statistics and not advice. They were not calibrated against outcomes.
• The saved raw answers are kept with their timestamps. Where a model used a fact absent from the dossier (for instance Mistral on strikes "since June"), we say so instead of correcting it silently. The probabilities for runs 2 and 3 were first extracted from the on-screen answers by pattern. Afterwards we re-opened all 12 Mideast chats (6 models, 2 runs) and read each answer on screen, from the top, in screenshots rather than copied text, comparing the probabilities with the stored values: they matched in every chat read. It is a human-style read, not a text diff.
• Medians, ranges and products are our arithmetic on the models' stated figures. They add no information the models did not give.
• No claim here comes from model memory except where an answer is quoted as such. Mistral states a training cutoff of 2023, which limits what it can know beyond the dossier.
• During the chat runs the tester's shared clipboard was overwritten once, and one unrelated message was sent by mistake into a Claude conversation. That conversation is not part of the data; each retained run was checked to hold the correct prompt before sending, and its answer to contain the six expected items.
• English draft only. Not published, not submitted to search engines.

Update log

This article is built to be updated. When new verified facts arrive, we will rerun the same six questions on an updated dossier and append the figures here, keeping every earlier row.