Route Uncertain Classifier Predictions with Selective Classification

A classifier does not have to answer every request. In systems where a bad prediction is costly, forcing a label on every input can be a poor product decision even when the model has strong average accuracy.

Selective classification adds a reject option. The system returns a model prediction only when an acceptance rule considers the case suitable; otherwise it abstains and sends the case to a fallback such as human review, a second model, or a request for more information.

This changes the design question from “How accurate is the classifier?” to a more useful operational question: “How much traffic can the classifier handle at an acceptable error rate?” This article develops that mental model, shows how to measure the trade-off, and explains the traps that make an abstention policy look stronger than it is.

Start with a classifier that can decline

Consider a document classifier with three labels:

invoice
receipt
contract

For one input it returns:

invoice   0.92
receipt   0.05
contract  0.03

For another:

invoice   0.39
receipt   0.34
contract  0.27

A normal top-score decision emits invoice for both. A selective classifier can treat them differently:

if max_score >= 0.80:
    return top_label
else:
    return ABSTAIN

The first document is accepted and the second is rejected by the automated path.

The value does not come from the threshold itself. It comes from concentrating automated decisions on cases where the chosen score is informative about error. If low-score cases contain more mistakes than high-score cases, removing low-score cases can reduce the error rate among accepted predictions.

That final qualification matters. A high model score is not automatically a reliable estimate of correctness, and a threshold that works on one population can fail after the input distribution changes.

Coverage and selective risk describe the trade-off

An abstaining classifier needs metrics that account for both accepted and rejected cases.

Coverage is the fraction of examples on which the system makes a prediction:

coverage = accepted examples / all examples

If a validation set has 10,000 examples and the system accepts 7,500, coverage is 75%.

Selective risk measures error on the accepted subset. With ordinary classification error as the loss:

selective risk = incorrect accepted predictions / accepted predictions

If 150 of those 7,500 accepted predictions are wrong:

selective risk = 150 / 7,500 = 0.02

The accepted traffic therefore has a 2% observed error rate on that evaluation set.

Risk and coverage must be read together. A system can obtain tiny selective risk by accepting only a handful of trivial cases. That may be useless if the product needs automation for most traffic. At the other extreme, 100% coverage removes abstention and returns the base classifier’s ordinary error rate.

A useful evaluation sweeps the acceptance threshold and records both values:

These numbers are a teaching example, not a universal pattern. Real curves can be irregular, especially with small evaluation sets or shifted data.

The table makes the product decision explicit. Moving from a 0.80 to 0.90 threshold reduces observed selective risk by 0.8 percentage points, but it also routes another 24% of all traffic to the fallback.

The acceptance score is part of the system

The simplest acceptance score is the largest predicted class probability. It is easy to implement, but it should not be confused with a guarantee that an accepted prediction is correct.

For a classifier producing class scores (p_1, \ldots, p_K), a common rule is:

[ g(x) = \max_k p_k(x) ]

Accept when:

[ g(x) \ge \tau ]

where (\tau) is a threshold chosen using held-out data.

Other scores can be useful. The margin between the top two class scores can identify cases where two labels compete closely:

[ g_{\text{margin}}(x) = p_{(1)}(x) - p_{(2)}(x) ]

Here (p_{(1)}) is the largest score and (p_{(2)}) is the second largest. A large margin indicates that the top class is separated from its nearest competitor according to the model.

Entropy is another option when the outputs form a probability distribution:

[ H(p) = -\sum_k p_k \log p_k ]

Higher entropy means probability mass is spread more broadly across classes. A policy can reject examples above an entropy threshold.

None of these scores is universally superior. Their usefulness depends on whether they rank likely errors above likely correct predictions on data resembling the deployment population.

Pick a threshold from an operational constraint

Choosing 0.80 because it looks conservative is not a sound selection method. Start from a requirement that reflects the system around the model.

Suppose a review team can process at most 2,000 documents per day and incoming volume is 10,000 documents per day. The automated path therefore needs roughly 80% coverage unless another fallback absorbs some traffic.

On a held-out set, sweep candidate thresholds and find settings near that coverage. Then inspect selective risk, class-specific behavior, and uncertainty around the estimate. If no threshold reaches acceptable risk at the required coverage, threshold tuning cannot fix the underlying classifier. The options are to improve the model or data, change the fallback capacity, narrow the task, or accept a different service target.

The reverse setup is also common. A product may require accepted predictions to stay below a specified error budget. In that case, choose the highest coverage supported by validation evidence under that constraint.

Do not tune the threshold on the final test set. Threshold selection is a model-selection decision. Use validation data for the choice and preserve separate test data for the final estimate.

Calibration and selective classification are related but different

Probability calibration asks whether predicted probabilities correspond to observed frequencies. Selective classification asks whether an acceptance score can separate easier cases from cases more likely to be wrong.

A calibrated classifier can still have weak error ranking. Imagine many examples receive scores near 0.70 and the aggregate frequency is well calibrated, but errors and correct predictions are thoroughly mixed inside that group. A threshold has little ability to isolate the errors.

The reverse can also occur. A model’s numeric probabilities may be poorly calibrated while its score ordering still places most errors below most correct predictions. Such scores can support useful abstention even though 0.9 should not be interpreted literally as a 90% chance of correctness.

Calibration can still help when a policy attaches semantic meaning to a probability threshold or when stakeholders need probabilities for downstream decisions. It does not replace direct evaluation of coverage and selective risk.

Evaluate the fallback, not just the model

Abstention moves work somewhere else. A production design is incomplete until that destination is specified.

For human review, measure queue volume, turnaround time, reviewer accuracy, and the cost of disagreement. If rejected cases are systematically difficult, reviewer performance on that subset may be lower than reviewer performance on random samples.

For a second model, evaluate the complete cascade. The fallback may be more expensive and still share the first model’s blind spots. If both models were trained on similar data or use similar representations, sending low-confidence cases downstream does not guarantee independent correction.

For a request-for-information flow, measure whether users can actually provide the missing evidence. Rejecting a document because a field is unreadable can be useful if the system can ask for a clearer scan. Abstaining without an actionable recovery path merely converts prediction errors into dead ends.

A practical metric is total expected cost:

[ C = C_{\text{error}} E + C_{\text{fallback}} R ]

where (E) is the number of accepted errors and (R) is the number of rejected cases. The cost terms represent application-specific consequences. This simplified expression can be extended for class-dependent errors, latency, reviewer capacity, or multiple fallback stages.

The point is not to reduce every product decision to one number. It is to expose the fact that rejection has a cost as well as a benefit.

Check performance by class and subgroup

A single coverage number can hide uneven behavior.

Suppose the document system reaches 80% overall coverage, but the breakdown is:

The global metric can look healthy while contracts are sent to review more than half the time. That may be acceptable if contracts are rare and costly. It may be a serious service problem if contract automation is the main product goal.

Apply the same reasoning to relevant user or data subgroups when such analysis is appropriate and lawful. Different scanners, languages, document templates, acquisition channels, or time periods can produce different score distributions.

Do not assume one global threshold is automatically appropriate. Per-class or per-segment thresholds can be justified when the operational costs differ, but they add complexity and need enough validation data in each segment.

Distribution shift can break the rejection rule

An abstention threshold is selected from observed relationships between scores and errors. Deployment data can change those relationships.

A new document template may produce confident mistakes. Image degradation may lower scores for correct predictions and flood the review queue. A new class that the classifier was never designed to represent may be mapped confidently onto an existing class.

This creates two separate monitoring needs:

1. Traffic behavior: coverage, rejection rate, score distributions, and fallback volume.
2. Outcome quality: selective risk or a suitable proxy once verified labels become available.

A sudden coverage change can be detected without labels and is often an early operational signal. Stable coverage is not proof of stable accuracy, however. Confident errors can leave coverage almost unchanged.

Thresholds should therefore be treated as versioned system parameters tied to a model, evaluation dataset, and decision policy. Re-evaluate them after material model changes or meaningful shifts in input data.

Common mistakes that weaken selective classification

Several implementation choices can create false confidence.

Reporting accuracy only on accepted cases. High accepted accuracy is incomplete without coverage. A system that accepts 5% of traffic has a different value proposition from one that accepts 90%.

Treating softmax output as guaranteed confidence. Softmax produces normalized scores, but normalization alone does not make them calibrated probabilities or reliable error detectors.

Selecting a threshold on the test set. This leaks evaluation information into the policy and makes the reported result optimistic.

Ignoring fallback errors. If rejected cases go to another imperfect process, report end-to-end outcomes rather than presenting abstention as if rejected cases disappear.

Using a threshold from another model version. Score scales and rankings can change after retraining, architecture changes, quantization, or other modifications. Validate the policy again.

Assuming rejection catches unfamiliar inputs. Low confidence can correlate with unusual inputs, but a classifier can also be confidently wrong on data outside its intended scope. Dedicated out-of-distribution checks may be needed when that failure mode matters.

When selective classification is a good fit

Selective classification is useful when errors have meaningful cost and a fallback exists. Document routing, moderation queues, medical support systems with qualified review, industrial inspection, and other assisted-decision workflows can fit this pattern when their domain requirements permit it.

It is especially attractive when the system can tolerate variable automation rates. Instead of demanding that one model handle every case, the architecture allocates easy cases to automation and difficult cases to another path.

A simpler classifier without abstention can be preferable when every request must receive an immediate label, fallback capacity is effectively zero, model scores do not separate errors well, or the cost of rejection is comparable to the cost of a wrong prediction.

Selective classification is also not a substitute for fixing a weak model. If accepted errors remain high even at low coverage, the acceptance score is not providing enough separation for the intended policy.

Treat abstention as a decision policy

The central idea is simple: a classifier can choose whether to act, not only which label to emit. That extra decision creates a controllable trade-off between automation coverage and error among accepted cases.

Build the policy on held-out data, inspect the full risk-coverage curve, and include the fallback in the evaluation. Then monitor both acceptance behavior and real outcomes after deployment.

A useful next step is to take an existing classifier, record one scalar acceptance score per validation example, sweep the threshold, and plot coverage against selective risk. That curve will show whether abstention offers a practical operating region before you add more machinery.