Three AI agents agree: why they can all be wrong

Three confirmations can come from one mistake

Three AI agents read a procedure and give the same answer. Agreement looks like verification, but a shared outdated page can repeat one mistake three times. How much does majority voting improve when errors are dependent? We construct a complete probability model rather than assigning invented performance numbers to real agents.

We assume binary truth and an odd number m of voters. E_i is one when agent i is wrong, zero otherwise. Each has marginal error p=0.2, a chosen value, not a measurement. With many distinct wrong answers, counting errors alone cannot describe answer voting; the binary restriction is essential. Probabilities are dimensionless.

Start with independent errors

With independence, majority fails when at least (m+1)/2 voters fail. For m=3 there are three two-error arrangements, each p²(1−p), plus all-three p³. Thus 3p²(1−p)+p³=0.104, below 0.2. The gain comes from errors falling on different cases, not agreement in the abstract.

C(m, k) counts which k voters fail; the product gives each arrangement probability. Summation covers precisely majority-error outcomes. Code checks all binary sequences for m=3 and 5 against the formula: a finite exact check, not simulated conversations. For p above 0.5 there is no improvement guarantee; the script checks that m=3, p=0.7 worsens individual risk.

Add a shared cause without changing individual error

With probability q use a shared branch: every voter copies one Bernoulli(p) error indicator. Otherwise use independent errors. q is a branch probability, not measured text similarity. Each agent still has error p in both branches; only the joint distribution, how errors occur together, changes.

In the shared branch majority fails exactly when the shared indicator fails, probability p; otherwise risk is B_m(p). Weighting gives the formula. With three agents and q=0.5 it is 0.152. At q=1 it remains 0.2 for any m: copies add no evidence. For p<0.5, growing m retains floor q·p in this model. The plot shows that limit, not a universal law of agent systems.

Correlation and unanimity answer different questions

For distinct agents E[E_iE_j]=q·p+(1−q)p². Subtract p² for covariance q·p(1−p), then divide by variance p(1−p): correlation is q for 0< p<1. This follows from the complete construction. Individual probabilities and pairwise correlations alone generally do not specify majority events; higher-order dependence matters. Answer agreement is also not error correlation without ground truth.

Accepting only unanimity requires counting rejected requests. Shared-branch outcomes always agree; independent unanimity has probability p^m+(1−p)^m. Coverage is q+(1−q)[p^m+(1−p)^m]. Accepted-case error divides q·p+(1−q)p^m by coverage. Code computes both, avoiding confusion between selective accuracy and a complete service.

What a real system must verify

The experiment enumerates probabilities, not language models, and measures no agent cost, latency or product quality. Practice needs representative tasks, independent truth and shared-source records, not just an agent count. Different names or prompts do not prove independence. Changing sources may reduce a common cause but requires measurement; a final verifier may share the original error too.

Voting helps when it combines evidence with sufficiently different errors, not merely many answers. Here individual 20% errors yield majority risks 10.4%, 15.2% and 20% under independence, half shared branch and full dependence. Identical individual ability gives different collective outcomes. Counting confirmations without their provenance can multiply confidence in one mistake.

NIST — Binomial distribution.

import math,itertools,json,sys
from pathlib import Path
p=.2;rows=[]
def independent_majority(m,p):return math.fsum(math.comb(m,k)*p**k*(1-p)**(m-k) for k in range(m//2+1,m+1))
for m in [1,3,5,9]:
 for q in [0,.25,.5,.75,1]:
  majority=q*p+(1-q)*independent_majority(m,p)
  all_wrong=q*p+(1-q)*p**m;unanimous=q+(1-q)*(p**m+(1-p)**m)
  rows.append(dict(agents=m,sharedBranchProbability=q,pairwiseErrorCorrelation=q,errorEach=p,majorityError=majority,unanimousCoverage=unanimous,errorGivenUnanimity=all_wrong/unanimous))
for m in [3,5]:
 exhaustive=math.fsum(p**sum(bits)*(1-p)**(m-sum(bits)) for bits in itertools.product([0,1],repeat=m) if sum(bits)>m//2)
 assert abs(exhaustive-independent_majority(m,p))<1e-14
assert abs(independent_majority(3,.2)-.104)<1e-14
assert all(abs(r['majorityError']-.2)<1e-14 for r in rows if r['sharedBranchProbability']==1)
assert independent_majority(3,.7)>.7
r=dict(python=sys.version.split()[0],seed=None,scope='Exact constructed mixture model of dependent Bernoulli error indicators, not an evaluation of real AI agents',model='With probability q all m errors equal one Bernoulli(p); otherwise errors independently Bernoulli(p). Binary ground truth and odd majority; no adversarial or language-model execution.',rows=rows)
Path(__file__).with_name('results.json').write_text(json.dumps(r,indent=2),encoding='utf8');print('All exact mixture and enumeration assertions passed; rows',len(rows))

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 25 October 2026.