You ask the same question to ChatGPT and Claude. They give you two different answers. Your gut reaction: neither one is trustworthy now. That reaction is backwards, and understanding why changes how you should actually read a disagreement.
The instinct is "if they disagree, nobody knows"
It feels logical: if two systems trained on similar data can't agree, the question must be unanswerable, or both answers must be guesses. On r/ClaudeAI, someone asked exactly this after Claude and ChatGPT gave "completely different answers to the same dilemma" — the top response wasn't "here's the right one", it was closer to "that's the point, now you know it's not settled."
That's the correct read. A disagreement between independently trained models isn't proof that the question is unanswerable. It's proof that the question has more than one defensible angle — and it tells you which angle each model is anchored to, if you look at the reasoning instead of just the conclusion.
When agreement should worry you more than disagreement
Flip the framing. If six models trained by different companies, on different data, with different safety tuning, all independently land on the same conclusion, that convergence means something. It's not proof of truth — models can share a training-data blind spot and be confidently wrong together — but it's a real signal, stronger than any one model's stated confidence.
Disagreement is the opposite kind of signal, and it's just as informative: it tells you the models are drawing on different premises. One might be weighting a recent regulation change, another an older but more thorough dataset. Neither is "wrong" in isolation. The premise, not the model, is what you need to see.
What to actually do when models disagree
- Don't average the answers. "Somewhere in between" is rarely the right call — it's usually a fabricated middle ground that neither model actually argued for.
- Find the premise behind each answer. Ask (or have a tool ask) each model to state the assumption its answer depends on. Usually one assumption is closer to your actual situation than the other.
- Check what would flip the answer. If a model says "assuming the contract is under French law", and yours isn't, that answer just became irrelevant — not wrong, irrelevant.
- Weight by stakes, not by confidence. A confidently-stated wrong answer sounds identical to a confidently-stated right one. On a high-stakes question (health, money, legal, a big purchase), a disagreement is exactly the moment to slow down, not the moment to pick whichever answer sounded more authoritative.
The real cost of ignoring a disagreement
The risk isn't that you can't find an answer. It's that you pick the first confident-sounding one, on a question where the two other equally confident-sounding models would have told you something materially different — and you never found out, because you only asked once.
That's the actual argument for asking several models before a decision that matters: not that six opinions are automatically better than one, but that a single confident answer hides whether five other analyses would have agreed with it or flagged something you missed.