A feature vector can sit close to a reference mean in Euclidean distance and still be unusual for the distribution that produced the reference data. Mahalanobis distance accounts for this by scaling displacement according to covariance. Directions with little observed variation contribute more to the score than directions in which the reference data naturally spreads out.

That behavior makes the distance useful as a compact outlier score for model features or embeddings, provided the reference statistics are meaningful and the covariance estimate is numerically usable.

Covariance changes the geometry of distance

For a feature vector x, reference mean mu, and covariance matrix Sigma, squared Mahalanobis distance is:

d^2(x) = (x - mu)^T Sigma^-1 (x - mu)

If Sigma is the identity matrix, this reduces to squared Euclidean distance from the mean. A diagonal covariance instead scales each coordinate according to its variance. A full covariance also accounts for correlations between coordinates.

Consider two feature directions. The reference data varies widely along the first but stays tightly concentrated along the second. Equal raw displacement in both directions does not have equal significance under the fitted distribution. The inverse covariance downweights movement along the high-variance direction and amplifies movement along the low-variance direction.

This geometry can also be viewed through whitening. If a linear transform maps the reference covariance to the identity, Mahalanobis distance in the original space equals Euclidean distance in that transformed space. The score therefore measures displacement after covariance structure has been normalized.

Class-conditional references preserve multiple centers

A single global mean is a poor reference when valid features form several separated groups. Classification models naturally provide one possible partition: estimate a mean for each class, then compare an input feature to the nearest class reference.

With class means mu_c and a shared covariance Sigma, a simple score is:

score(x) = min_c (x - mu_c)^T Sigma^-1 (x - mu_c)

Small values indicate that the feature lies near at least one class center under the covariance-adjusted geometry. Large values indicate distance from every fitted center.

A shared covariance pools information across classes and requires fewer covariance parameters than fitting a separate full matrix for each class. Separate class covariances can represent class-specific shapes, but their estimates become unstable sooner when feature dimension is large relative to the number of reference samples.

The choice is not purely mathematical. A class-conditional score assumes that the selected feature representation organizes relevant inputs into regions that can be summarized by those statistics. If features have curved, multimodal, or strongly non-elliptical structure within a class, one mean and covariance can compress away structure that matters for outlier detection.

Covariance estimation is often the limiting detail

A full covariance matrix for D features contains on the order of D^2 entries. In high-dimensional representations, the empirical covariance can be singular or poorly conditioned, especially when the reference set is not large compared with D.

Direct matrix inversion is then a fragile implementation choice. A regularized estimate can add a positive diagonal term:

Sigma_reg = Sigma + lambda * I

with lambda > 0. This shifts covariance eigenvalues away from zero and makes the linear system better conditioned. The value of lambda changes the geometry, so it is part of the detector configuration rather than a neutral numerical patch.

A diagonal covariance is another option. It discards cross-feature correlations but reduces estimation and storage costs. This can be a reasonable constraint when a full covariance cannot be estimated reliably, though it represents a different distance function.

Code also does not need to form Sigma^-1 explicitly. Solving a linear system is generally preferable:

delta = x - mu
solve Sigma_reg * v = delta
d2 = delta^T v

Factorizations such as Cholesky are suitable when the regularized covariance is symmetric positive definite. Reusing a factorization across many queries avoids repeating the same matrix decomposition.

The score is not a probability by itself

Mahalanobis distance produces a scalar ordering of feature vectors relative to fitted statistics. Turning that number into a probability requires additional assumptions or calibration.

Under an exact multivariate Gaussian model with known parameters, squared Mahalanobis distance has a chi-squared relationship with the feature dimension. Model features rarely come with a guarantee that their class-conditional distribution is exactly Gaussian, and estimated parameters add another layer between the textbook result and deployed behavior.

For that reason, a practical threshold is better treated as an empirical decision boundary fitted and evaluated on representative held-out data. The threshold can be chosen against a target operating condition, such as an acceptable false-rejection rate on in-distribution inputs, when suitable validation data exists.

A threshold selected for one model checkpoint or feature layer is not automatically portable to another. Representation scale, covariance structure, and class geometry can all change even when the external label set stays fixed.

Feature choice determines what counts as unusual

The detector only sees the representation supplied to it. Two inputs that are far apart in raw input space can map to nearby features, while visually or semantically similar inputs can separate in a representation optimized for a specific task.

Using an internal feature layer therefore defines outliers relative to that layer’s geometry. A representation trained to separate known classes may suppress variation that is irrelevant to classification, including variation that an outlier detector would otherwise want to retain.

This creates a boundary on interpretation. A large Mahalanobis score indicates mismatch with the fitted feature distribution; it does not identify the cause of that mismatch. Corruption, domain shift, a novel class, preprocessing changes, or ordinary tail examples can all produce elevated scores.

The detector should consequently be evaluated against the kinds of shifts that matter to the application rather than against one convenient outlier source alone.

Distance comparisons require a stable pipeline

Reference statistics and query features must come from the same representation pipeline. Changing normalization, tokenization, image transforms, model weights, pooling, or feature layer can invalidate the stored means and covariance even if the output vector has the same dimension.

Numerical precision can matter as well. Covariance estimation combines many products and differences, and an ill-conditioned matrix magnifies error in low-variance directions. Regularization helps conditioning, but it does not make mismatched or insufficient reference data representative.

Monitoring the distribution of scores on known in-distribution traffic can expose drift in the feature space, though score drift alone does not identify its source. When a model version changes, recomputing reference statistics and reevaluating the operating threshold keeps the detector tied to the representation it actually measures.

Mahalanobis distance is most useful when its assumptions stay visible: the feature space must carry relevant structure, the reference data must represent intended inputs, and covariance estimation must be stable enough to define a meaningful geometry. Under those conditions, the method offers more information than raw distance from a centroid without pretending that a covariance-adjusted score is a universal measure of novelty.

Artificial Intelligence Out-of-Distribution Detection Embeddings Evaluation