A classifier can be highly accurate on its test set and still behave confidently on inputs that are unlike anything it was trained to recognize. A product classifier trained on shoes, bags, and watches may receive a photo of a bicycle and still be forced to choose one of its known classes.

That creates a deployment problem: ordinary classification answers which known class looks most likely, but many systems also need to ask whether this input resembles the data on which the classifier was validated.

Out-of-distribution (OOD) detection adds that second decision. This article develops one practical approach for classifiers: deriving an energy score from their logits, choosing a threshold on separate data, and using the score as a signal for routing unfamiliar inputs. You will also see why an OOD score is not a correctness guarantee and how to evaluate it without contaminating the classifier’s normal test results.

Start with the decision you actually need

Suppose a model classifies support screenshots into three known categories:

billing
login
shipping

For an ordinary screenshot, the classifier might produce logits such as:

billing:   4.2
login:     1.0
shipping: -0.4

Logits are the model’s raw class scores before softmax converts them into probabilities. The largest logit selects the predicted class, so this example predicts billing.

Now imagine that the input is not a support screenshot at all but a landscape photo. A closed-set classifier still has to assign probability among its three known classes. Softmax does not provide an extra unknown class merely because the input is unfamiliar.

An OOD detector therefore adds a separate gate:

input -> classifier -> class prediction
                   \
                    -> OOD score -> accept or flag

The useful mental model is classification and unfamiliarity are different questions. The class prediction says what the model would choose among known alternatives. The OOD score estimates whether the input should be trusted to participate in that choice.

Turn logits into an energy score

For a classifier with logits z_1, ..., z_K, a commonly used energy score is

E(x) = -T * log(sum_i exp(z_i / T))

where T is a positive temperature. With T = 1, this becomes the negative logsumexp of the logits.

Consider two simplified three-class outputs:

input A logits: [5.0, 1.0, 0.0]
input B logits: [0.4, 0.2, 0.1]

At T = 1:

E(A) = -log(exp(5.0) + exp(1.0) + exp(0.0)) ~= -5.025
E(B) = -log(exp(0.4) + exp(0.2) + exp(0.1)) ~= -1.339

Input A has the lower energy. Under the usual convention for this score, lower energy is treated as stronger evidence for an in-distribution input, while higher energy is treated as more OOD-like.

The absolute numbers are not universal. Changing the model, its training procedure, the temperature, or even the scale of its logits can change the score distribution. An energy value such as -3 therefore has no portable meaning by itself.

In implementation, compute the expression with a numerically stable logsumexp operation rather than exponentiating large logits directly.

Why energy differs from maximum softmax probability

A simple OOD baseline is the model’s maximum softmax probability. If the largest class probability is low, flag the input as unfamiliar.

That can be useful, but softmax probabilities depend on relative differences among logits. Adding the same constant to every logit leaves softmax unchanged:

softmax([2, 1, 0]) = softmax([12, 11, 10])

Energy does change under that shift because it uses the absolute logit values through logsumexp. This gives it information that maximum softmax probability discards.

That difference does not imply that energy scores will outperform maximum softmax probability on every model or every OOD dataset. Treat both as scoring rules to validate on data representative of the deployment problem. If a simpler baseline meets the required operating point, additional complexity may not be justified.

A threshold turns a score into an action

A raw score is not yet a product decision. You need a threshold that maps scores to actions.

For the convention above, a simple policy is:

if energy <= threshold:
    accept the classifier prediction
else:
    flag the input as potentially OOD

The threshold should be selected using validation data, not guessed from a few examples. Keep the data used for threshold selection separate from the final evaluation set.

A useful validation collection contains at least two groups:

  • In-distribution (ID) examples representative of valid production inputs.
  • OOD examples representative of important unfamiliar inputs the system may encounter.

Suppose a threshold keeps 95% of ID validation examples. That operating point deliberately allows about 5% of those ID examples to be flagged. You can then measure how many relevant OOD examples the same threshold catches.

The trade-off is unavoidable. Moving the threshold to flag more OOD inputs usually also flags more legitimate inputs. The correct balance depends on what happens after a flag. Human review for a high-impact decision may tolerate more false alarms than an interactive feature where unnecessary rejection is costly.

Evaluate detection separately from classification

Classifier accuracy and OOD detection quality measure different behavior. Report them separately.

For the classifier, continue measuring task metrics on clean ID test data. For the OOD detector, compare score distributions for ID and OOD examples and evaluate the operating points that matter to the application.

Useful OOD measurements include:

  • the fraction of ID examples accepted at a chosen threshold;
  • the fraction of OOD examples detected at that same threshold;
  • false-positive and false-negative rates for the accept/flag decision;
  • threshold-independent ranking summaries such as AUROC, when appropriate.

AUROC measures how well a score ranks positive examples above negative examples across thresholds. It does not tell you whether a particular production threshold has an acceptable false-alarm rate. A system can have a reasonable AUROC and still be unusable at the operating point your application requires.

Be explicit about which class is considered positive when reporting rates. OOD literature and software packages do not all use the same score direction or label convention. A sign error can make an apparently sensible metric describe the opposite decision.

Choose OOD data that represents real failures

OOD is not one homogeneous category. A detector that separates handwritten digits from photographs has not necessarily learned to detect the unfamiliar cases that matter in your application.

For a support-screenshot classifier, useful OOD validation sets might include:

ID:
- billing screenshots
- login screenshots
- shipping screenshots

near-OOD:
- support screenshots from an unsupported product area
- screenshots from a redesigned interface not represented in training

far-OOD:
- landscape photographs
- scanned documents
- unrelated illustrations

Near-OOD examples are especially important because they can look similar to valid inputs while differing in a task-relevant way. Far-OOD examples are often easier to separate and can make a detector look stronger than it will be on subtle production shifts.

Do not tune a threshold on the same OOD examples used for the final claim. Otherwise the evaluation measures adaptation to those examples as well as general detection ability.

Energy is a signal, not a guarantee

An energy threshold can fail in both directions.

An unfamiliar input may receive low energy and pass the gate. Conversely, a valid but unusual ID example may receive high energy and be rejected. The detector has learned no universal definition of unfamiliarity; its behavior comes from the classifier, its training data, and any OOD-specific training objective that was used.

Several practical failure modes deserve attention.

Distribution shift can be gradual

Production data can drift rather than jump cleanly from ID to OOD. Camera changes, new user behavior, language changes, or a redesigned interface may move score distributions slowly. A threshold selected once can therefore become stale.

Monitor both the rate of flagged inputs and the score distribution over time. A shift is a reason to investigate; it is not automatically evidence that every newly flagged example is invalid.

Correctness and familiarity are not the same

An ID input can be misclassified with low energy. An OOD input can accidentally receive the correct label for the application’s purpose. OOD detection therefore should not replace ordinary error analysis, calibration, or task-specific evaluation.

If the real requirement is “send likely mistakes to a reviewer,” validate the routing rule against actual mistakes rather than assuming an OOD score is a direct error probability.

Thresholds depend on consequences

There is no universally correct acceptance rate. If flagging an input triggers an expensive fallback model, the threshold affects latency and cost. If it blocks an automated medical or financial action, the consequences of missed OOD cases may dominate instead.

Measure the complete policy, including fallback behavior, rather than optimizing the detector score in isolation.

Decide whether you need OOD detection at all

OOD detection is useful when the model operates in an open environment but was trained and evaluated on a bounded set of inputs, and when the application has a meaningful fallback for unfamiliar cases.

A fallback might be:

flag -> human review
flag -> request a clearer input
flag -> route to a broader model
flag -> decline automated action

Without a useful fallback, detecting unfamiliarity may add complexity without changing the system’s outcome.

A simpler rule may also be better when domain validity can be checked deterministically. If a model should only process images with known metadata, file types, or schema constraints, validate those conditions directly before invoking the model. Statistical OOD detection is most valuable for differences that cannot be captured reliably by straightforward input validation.

Keep the deployment loop measurable

A practical rollout can remain small:

1. Record classifier logits on held-out ID and relevant OOD validation data.
2. Compute one documented energy-score convention.
3. Plot or summarize the two score distributions.
4. Select a threshold from the application's acceptable trade-off.
5. Freeze the threshold before final evaluation.
6. Measure ID task quality and OOD routing quality separately.
7. In production, monitor score and flag-rate drift.

If you later retrain or replace the classifier, revalidate the OOD threshold. Because the score comes from model logits, a threshold is part of the model-specific deployment configuration, not a permanent property of the input domain.

Conclusion

Energy-based OOD detection adds a useful question to closed-set classification: not only “which known class wins?” but also “does this input look sufficiently familiar to trust that competition?”

The core mechanism is compact: compute an energy score from logits, validate its direction and distribution, choose a threshold on separate ID and relevant OOD data, and route flagged inputs to a meaningful fallback. The difficult part is not the formula. It is defining realistic OOD cases and choosing an operating point whose false alarms, missed detections, latency, and downstream costs fit the application.

Use energy as evidence of unfamiliarity, not proof of correctness or safety. That distinction keeps the detector useful without asking it to guarantee something it was never designed to guarantee.

Artificial Intelligence Machine Learning Model Evaluation Out-of-Distribution Detection