Agents and best-of-n: why more attempts can select the worse answer

Abstract: search quality depends on what it selects

An agent can generate many solutions and choose the one rated highest by a verifier. The probability that at least one solution is correct increases with attempts; the probability that the selected one is correct can decrease. We construct a discrete example where the generator produces correct answers 60% of the time and the verifier has 99% binary classification accuracy. Best-of-n selection reaches 94.20% with five candidates but falls to 36.60% with one hundred. There is no contradiction: taking the maximum explores the verifier’s error tail.

This is an educational mathematical analysis, not a benchmark of an LLM or an EL-AI agent. It requires elementary probability, independent events, and geometric sums. We distinguish three often-confused objects: availability of a good solution, recognition of that solution, and the policy deciding when to stop. Python calculates the numbers directly from the stated model; they do not come from conversations, software tests, or real clients.

1. A verifier usually right, but vulnerable in the tail

Each independent candidate belongs to one of three categories: C is correct, with probability 0.60 and score 1; O is an ordinary error, probability 0.39 and score 0; E is a rare overrated error, probability 0.01 and score 2. E need not be a deliberate attack: it might be a superficially convincing solution exploiting a gap in the criterion. The verifier always ranks E above C above O. Ties within a category do not affect correctness.

At acceptance threshold score≥1, the verifier correctly classifies all C and O: accuracy 0.60+0.39=0.99. Pairwise comparison of a correct and an incorrect candidate also looks strong: the correct one wins 97.5% of the time, since only 0.01/0.40=2.5% of errors are E. Neither metric describes selecting the maximum among one hundred candidates. That policy changes the distribution of answers reaching the user.

2. Deriving selected-answer correctness

The selected answer is correct exactly when no E appears and at least one C appears. No E in n independent draws has probability 0.99^n. Within this event, subtract the all-O case, probability 0.39^n. If even one E appears, the maximum score is wrong. If only O appears, the answer is also wrong. These three disjoint events cover all outcomes; the script checks that their probabilities sum to one.

The last line describes an ideal selector that always recognizes a correct answer when one exists. It is a candidate-availability bound, not actual verifier performance. At n=5 availability is 98.976%, while actual selection is 94.196763%. At n=100 availability is practically one, but E appears in 63.396766% of groups and is selected. Increasing solution variety does not automatically fix a criterion that prefers an error to the truth.

3. Finding the optimal budget analytically

Let P_n be selected-answer correctness. The increment P_(n+1)−P_n is −0.01·0.99^n+0.61·0.39^n: another candidate helps only if the second term exceeds the first. Benefit comes from rescuing all-ordinary-error groups; harm comes from introducing E into a previously usable group. The ratio (0.39/0.99)^n strictly decreases, so there is one sign change. The integer maximum is n=5, also checked by enumerating n from 1 to 1000.

Five is not a product recommendation: it depends on invented probabilities. Keeping P(C)=0.60 and setting P(E)=ε gives (1−ε)^n−(0.40−ε)^n. For ε=0.001 the best budget is 7 with probability 99.1411%; for ε=0.05 it is 3 with 81.45%. Reducing high-score errors changes both attainable quality and useful budget. A positive generation or verification cost may also favor stopping before maximum accuracy.

4. Changing policy: first accepted answer and abstention

Now consider another policy: inspect candidates in order, stop at the first score≥1, and abstain if none passes within n attempts. Both C and E pass; O is rejected. Answer probability is 1−0.39^n. Joint probability of answering correctly is 0.60·Σ_(j=0)^(n−1)0.39^j, because j ordinary errors can precede the first C. Dividing by answer probability gives 0.60/0.61=98.3607%, independent of n.

This does not turn the verifier into an oracle: about 1.64% of accepted answers remain wrong. It avoids systematically preferring E over C after finding an acceptable answer. Report two measures: correctness conditional on answering, and coverage, the fraction of cases answered. Frequent abstention can look perfect if only the first is published. Expected attempts are (1−0.39^n)/0.61, including budget exhaustion, with limit about 1.63934 attempts, but only under our assumptions.

5. What breaks the model, and how to evaluate an agent

Independence is substantive. If all one hundred candidates copy the same outcome, correctness remains 0.60 and the powers above do not apply. Even different samples may share the same model or test blind spot. In an adaptive agent, each attempt depends on previous errors and verifier messages; probabilities can change at every step. Measure curves for the entire policy on problems separated between development and test, rather than extrapolating per-candidate estimates across a budget.

A second verifier helps only if it adds relevant evidence. Repeating the same judgment with a shared error does not justify multiplying false-positive probabilities. For a coding agent, tests independent of those used during solution search can check a specification; they do not automatically prove every program property. Preserve candidates, scores, decisions, abstentions, costs, and correctness criteria. This separates generation failure from recognition failure and policy failure.

6. Relation to the literature and conclusion

Gao, Schulman, and Hilton study reward-model overoptimization with best-of-n and PPO. In the consulted 2022 arXiv v1, the “gold” reference is itself a model, not human truth measured at every step. Methods and results motivate separate evaluation of an optimized proxy. Our example does not reproduce their curves or models: it isolates a mechanism using three categories with known probabilities. It is neither a state-of-the-art survey nor proof that every best-of-n search deteriorates.

The technical conclusion is more precise than “fewer attempts are better”: evaluate budget jointly with verifier error distribution and selection policy. A correct answer being present does not guarantee the agent returns it. The archive calculates all probabilities, checks the event partition, and repeats optimization for three rare-error frequencies. These numbers imply no randomness, trained model, or EL-AI operational result.

Gao L., Schulman J., Hilton J. (2022), Scaling Laws for Reward Model Overoptimization, arXiv:2210.10760v1; methodology §2, results §3.

for n in [1, 2, 5, 10, 50, 100]:
    correct = .99**n - .39**n
    oracle = 1 - .4**n
    print(n, correct, oracle)
print("precision_first_pass", .6/.61)

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 24 September 2026.