Choose Consensus Outputs with Minimum Bayes Risk Decoding

A generative model can assign high probability to an output that is not the most useful answer for your application. This is especially visible when several different outputs are plausible: a translation can have multiple valid phrasings, a summary can emphasize different details, and a structured generator can produce several semantically similar candidates.

Greedy decoding chooses locally likely tokens. Beam search searches for a high-probability sequence. Sampling gives you diverse candidates. None of those methods, by itself, asks a different question that is often closer to the application goal: which candidate agrees best with the distribution of plausible outputs?

Minimum Bayes risk (MBR) decoding provides a decision rule for that question. In a common sample-based form, you generate several candidates, score how useful each candidate would be if another sampled output were the reference, average those scores, and return the candidate with the highest expected utility.

This article builds that idea from a small example, explains what MBR is actually optimizing, and shows the practical trade-offs around candidate diversity, utility metrics, computation, and evaluation.

Start with the decision, not the probability

Suppose a model generates four candidate summaries for the same support incident:

A: Payment requests timed out after the database failover.
B: Database failover caused payment request timeouts.
C: Payment requests failed during a database failover.
D: The service experienced an incident.

Assume D has the highest model probability. It is fluent and broadly compatible with many incidents, so the model may consider it safe. But the other three candidates agree on two important details: payment requests were affected, and the problem was associated with a database failover.

If the application values semantic agreement with plausible summaries, choosing only by model probability can miss that consensus.

MBR separates two roles:

• the model distribution represents which outputs are plausible;
• the utility function represents what makes one output useful relative to another plausible output.

That separation is the central mental model. Probability supplies beliefs about possible outputs. Utility supplies the decision criterion.

The sample-based MBR rule

Let x be the input and let the model define a distribution p(y | x) over outputs. In an ideal decision-theoretic form, we would choose the hypothesis h with the highest expected utility:

h* = argmax_h E[y ~ p(y | x)] u(h, y)

Here, u(h, y) is a utility function. Larger values mean that choosing h is better when y is treated as a plausible reference.

The full output space is enormous for text generation, so practical MBR commonly approximates the expectation with samples. Draw n outputs from the model:

y1, y2, ..., yn ~ p(y | x)

Then score each candidate against the sampled references:

score(hi) = (1 / n) * sum_j u(hi, yj)

and choose the candidate with the largest average score.

If the candidate set and reference-sample set are the same n outputs, a straightforward implementation evaluates roughly n * n candidate-reference pairs. That quadratic number of utility evaluations can become the main cost when the utility metric itself is expensive.

Work through the smallest useful example

Return to the four incident summaries. For teaching purposes, suppose we already have a semantic similarity utility whose values range from 0 to 1. This is a simplified example, not a recommendation for a particular production metric.

Assume the pairwise scores are:

        ref A   ref B   ref C   ref D
cand A   1.00    0.92    0.84    0.30
cand B   0.92    1.00    0.86    0.28
cand C   0.84    0.86    1.00    0.32
cand D   0.30    0.28    0.32    1.00

Average each row:

A: 0.765
B: 0.765
C: 0.755
D: 0.475

Candidates A and B are tied. Although D may have had the highest individual model probability, it agrees poorly with the rest of the sampled distribution under this utility function.

This example demonstrates what MBR can do that ordinary reranking by model probability cannot: it can prefer a candidate near the center of a cluster of plausible outputs.

The result is not automatically “more correct.” It is correct only relative to the model samples and the chosen utility. If both are poor, the MBR decision can also be poor.

Why MBR is different from majority voting

For tasks with a small set of exact answers, majority voting can be enough. Generate several outputs, normalize them, and return the answer that appears most often.

That approach breaks down when valid outputs have many surface forms. These three strings express nearly the same event:

The database failover caused payment timeouts.
Payment requests timed out after the database failover.
Payment timeouts followed the database failover.

Exact-string voting treats them as three different answers. MBR can treat them as mutually supportive if the utility function captures the similarity that matters for the task.

This also explains why MBR is not simply “fancy voting.” Voting uses discrete agreement. MBR can use graded utility, so partial semantic agreement can contribute to the decision.

The utility function defines what consensus means

The utility function is not a minor implementation detail. It defines the geometry of the decision.

For translation, a utility may measure translation quality or similarity to a reference. For summarization, it may reward preservation of important content. For structured generation, it could compare normalized fields rather than raw strings. For a narrow classification-like generation task, exact match may be appropriate.

A useful metric should reflect the differences you actually care about. Consider two candidate incident summaries:

A: Payment requests timed out after the database failover.
B: Payment requests succeeded after the database failover.

They share many words but disagree on the critical outcome. A lexical similarity metric that rewards word overlap too strongly could consider them close even though the operational meaning is opposite.

Before adopting a utility metric, test it on deliberately chosen pairs:

• paraphrases that should score similarly;
• outputs with a critical factual contradiction;
• outputs that omit an important detail;
• verbose and concise versions with the same meaning;
• malformed outputs if formatting matters.

If the metric ranks these pairs incorrectly, MBR will optimize the wrong notion of consensus.

Candidate generation changes the result

Sample-based MBR only sees the outputs you give it. Candidate generation therefore affects the decision even when the utility function stays fixed.

If sampling is too narrow, every candidate may be a near-duplicate. MBR then has little meaningful choice. If sampling is too unconstrained, the pool may contain many low-quality or irrelevant outputs, increasing cost and potentially distorting the estimated expectation.

Temperature, truncation methods such as top-p sampling, prompt design, and the number of samples can all change the candidate distribution. These controls are model- and implementation-dependent; MBR does not guarantee that one sampling configuration is universally appropriate.

A practical evaluation should therefore treat candidate generation and MBR selection as separate stages:

input
  |
  v
generate candidate pool
  |
  v
measure pairwise utility
  |
  v
average utility per candidate
  |
  v
select highest expected utility

Log enough information to inspect both stages. If the final answer is poor, first ask whether a good candidate existed in the pool. If it did not, changing the MBR scorer cannot recover it. If a good candidate existed but lost, inspect the utility function and aggregation.

A simple implementation pattern

The core algorithm does not require a specialized API. The following pseudocode keeps generation and scoring explicit:

samples = generate_samples(input, n)

best_candidate = null
best_score = -infinity

for candidate in samples:
    total = 0

    for reference in samples:
        total += utility(candidate, reference)

    expected_utility = total / len(samples)

    if expected_utility > best_score:
        best_score = expected_utility
        best_candidate = candidate

return best_candidate

For a production system, you may use different sets for candidates and reference samples. That can be useful when you want a small set of outputs eligible for selection but a larger set to estimate the expectation. The general rule remains the same: evaluate each selectable candidate against a representation of plausible outputs.

Do not assume the self-comparison term u(candidate, candidate) is harmless. If every self-score is the same constant and every candidate is also a reference, it contributes equally to all candidates. But utility functions and candidate/reference sets do not always have those properties. Define the scoring convention explicitly rather than silently dropping terms.

Understand the cost before increasing the sample count

More samples can represent the model distribution more completely, but they cost more generation time and more utility evaluations.

With one shared pool of n candidates and references, naive pairwise scoring uses n^2 utility calls. Increasing the pool from 20 to 100 does not multiply pairwise work by five; it multiplies it by 25:

20 candidates  ->    400 pairs
100 candidates -> 10,000 pairs

The actual latency depends on the utility implementation. A cheap string metric and a neural evaluator have very different cost profiles. Batched metric evaluation can improve hardware utilization, but batching does not remove the underlying number of comparisons.

This creates a practical quality-cost trade-off. A larger pool may improve the approximation or provide better candidates, but the marginal benefit can flatten while cost continues to grow.

Measure quality against pool size on your own task instead of choosing n from convention.

Watch for metric bias

MBR deliberately chooses outputs that score well under its utility metric. Evaluating the resulting system with the same metric can therefore give an overly favorable picture: the selection procedure has directly optimized that measurement.

This is a form of metric bias. It is especially important when the utility is a learned evaluator with blind spots that the candidate selection process can exploit.

Keep selection and final evaluation conceptually separate. If MBR uses metric U for selection, validate with evidence that is not merely another reading of U: human judgments when justified, task-specific correctness checks, independent metrics with different failure modes, or downstream outcomes.

The same principle applies beyond MBR. Once a metric participates in optimization, it is weaker evidence for independently proving that optimization improved the real objective.

Common failure modes

The samples share the same mistake

Consensus does not imply truth. If the model repeatedly generates the same false claim, MBR may confidently select a representative of that cluster. The method aggregates the model’s beliefs; it does not add external knowledge.

For factual tasks, retrieval, tools, constrained data sources, or explicit verification may address a problem that consensus decoding cannot.

The metric rewards superficial similarity

A utility based mainly on token overlap may prefer a central-looking answer that preserves common wording while missing a decisive semantic difference. Test the metric with adversarially simple contrasts before trusting aggregate scores.

Diversity is mistaken for quality

Increasing sampling temperature can broaden the pool, but a broader pool is not automatically a better approximation for your decision problem. If generation becomes noisy, MBR spends computation comparing poor candidates.

The candidate pool is too small

With only a few samples, the estimated expectation can be unstable. A single unusual candidate can have disproportionate influence. Evaluate selection stability as you increase the sample count rather than assuming a small pool is representative.

Latency is ignored

Generating multiple full outputs and scoring many pairs can be inappropriate for an interactive endpoint with a tight latency budget. An offline translation or batch summarization pipeline may tolerate that cost more easily than token-by-token chat interaction.

When MBR is a good fit

MBR is worth considering when several conditions line up: multiple outputs are legitimately plausible, you can generate a useful candidate pool, you have a utility function that reflects task quality, and the extra inference work fits the latency and cost budget.

It is particularly natural when agreement between semantically equivalent outputs is informative and exact-string voting would fragment that agreement.

A simpler method is often better when the task has a deterministic verifier. If you generate code that can be compiled and tested, or a structured answer that can be checked against hard constraints, direct verification may provide a stronger selection signal than pairwise consensus. Likewise, if one model call already meets the quality target, adding MBR only adds complexity and cost.

For factual questions, do not use consensus as a substitute for evidence. Several samples repeating the same unsupported statement are still unsupported.

Conclusion

Minimum Bayes risk decoding turns generation into an explicit decision problem. Instead of returning the output with the highest model probability, sample-based MBR asks which candidate has the highest average utility across plausible outputs from the model.

That shift is useful because probability and application quality are not the same objective. It is also demanding: candidate generation determines what MBR can choose, the utility function determines what agreement means, and naive pairwise scoring can become expensive as the sample pool grows.

Use MBR when consensus among plausible outputs is genuinely informative and you can validate the utility independently. When correctness can be checked directly, or when latency matters more than the possible quality gain, a simpler selection rule is usually easier to justify.