A classifier trained with one-hot targets is rewarded for moving probability mass toward the labeled class. Cross-entropy keeps decreasing as the model assigns that class a probability closer to one, even after the predicted class is already correct. Label smoothing changes this pressure by assigning a small amount of target mass to the other classes.

That change is easy to treat as a minor detail in the loss function. It is not minor when an application consumes the model’s probability values. The smoothed target changes the optimum encouraged by the training objective, so confidence scores from a smoothed model should not be interpreted as if they came from the same objective as ordinary one-hot training.

The target distribution stops being one-hot

For a classification problem with K classes, a common label-smoothing convention mixes the one-hot target with a uniform distribution. With smoothing parameter ε, the target distribution is

q = (1 - ε) * one_hot(y) + ε / K

Under this convention, the labeled class receives

1 - ε + ε / K

and every other class receives

ε / K

Some libraries and papers use a different convention that distributes the smoothing mass only across incorrect classes. Those definitions produce different target values for the same ε, so the exact implementation belongs in any comparison of training runs.

Cross-entropy with target distribution q is

L = -Σ q_i log p_i

where p_i is the model probability for class i. With a one-hot target, only the labeled class contributes directly to this sum. With label smoothing, every class has nonzero target mass under the uniform-mixture convention.

The resulting gradient with respect to a softmax logit z_i is

∂L/∂z_i = p_i - q_i

This makes the changed optimization pressure explicit. Once the predicted probability for the labeled class rises above its smoothed target value, its logit receives pressure in the opposite direction rather than being pushed indefinitely toward probability one.

Correct classification and extreme confidence become separate objectives

A model only needs the labeled class to have the largest score to classify an example correctly. One-hot cross-entropy, however, continues rewarding larger separation between that class and the alternatives as its probability approaches one.

Label smoothing weakens that incentive. It does not merely add noise to labels, and it does not randomly replace them. The target used by the loss is deterministic for a given class and smoothing rule.

This distinction affects the geometry of the output layer. Consider a three-class problem with ε = 0.1 under the uniform-mixture convention. The target for the labeled class is approximately 0.9333, while each alternative receives approximately 0.0333. A prediction such as

[0.97, 0.02, 0.01]

still selects the correct class, but it is more concentrated than the smoothed target. The loss therefore does not treat extra concentration as an unconditional improvement.

The practical consequence is not that every smoothed model emits probabilities near its target values. Shared parameters, finite data, regularization, optimization, and conflicting examples prevent that simple interpretation. The target distribution describes the objective’s local pressure, not a guaranteed output distribution.

Confidence calibration is not implied by smoothing

A lower maximum softmax probability can look better calibrated when an unsmoothed model is overconfident. That observation is not a guarantee that label smoothing calibrates probabilities for a particular application.

Calibration concerns the relationship between predicted confidence and empirical outcomes. For example, among predictions assigned confidence near 0.8, a calibrated classifier would be expected to be correct at roughly that rate under the evaluated data distribution and confidence definition. Label smoothing changes training targets; it does not directly enforce this empirical relationship on held-out data.

The distinction becomes especially relevant under distribution shift. A model can have moderate confidence values and still be poorly calibrated when the deployment distribution differs from the evaluation distribution. Conversely, an unsmoothed model can be calibrated after a separate calibration procedure.

Metrics also answer different questions. Negative log-likelihood evaluates the probability assigned to observed labels. Accuracy evaluates the top decision. Calibration errors summarize agreement between confidence and observed correctness using a particular estimator or binning scheme. A change in one does not establish a corresponding change in the others.

The smoothing convention belongs with the model artifact

Recording only ε = 0.1 is ambiguous. The value is incomplete without the rule used to construct targets.

Two implementations can both claim label smoothing of 0.1 while assigning different mass to the labeled class. With K classes, the uniform-mixture convention gives the labeled class 1 - ε + ε/K. A convention that reserves all smoothing mass for incorrect classes gives it 1 - ε.

The difference becomes small as K grows, but it is still part of the objective. Reproducing or comparing a run therefore requires the smoothing convention, class count, loss implementation, and any class weighting that changes target contributions.

Class weighting deserves particular care. Multiplying per-class loss terms by weights can interact with soft targets in implementation-specific ways. A library’s documented loss definition should be checked rather than assuming that smoothing and weighting compose identically across frameworks.

Label smoothing and knowledge distillation can both replace one-hot targets with distributions that contain nonzero mass on multiple classes, but the source of that mass differs.

Uniform label smoothing assigns alternatives according to a fixed rule. A teacher distribution in distillation can express structure among alternatives. For an image classifier, for example, a teacher might assign different probabilities to two incorrect classes because its logits distinguish their compatibility with the input. Uniform smoothing cannot encode that input-dependent relation.

This means smoothing should not be described as a generic substitute for soft teacher targets. Both alter the target distribution, but they carry different information and create different gradients.

Temperature-scaled distillation adds another distinction. Dividing teacher and student logits by a temperature before softmax changes the relative softness of the distributions used in the distillation term. That mechanism is separate from choosing a fixed label-smoothing parameter.

Evaluation should preserve the deployment decision

If a classifier is used only for top-class selection, accuracy or another task-specific decision metric may be the primary concern. If downstream code thresholds a probability, ranks cases by confidence, rejects uncertain predictions, or combines probabilities with other signals, the probability behavior itself becomes part of the interface.

A useful comparison keeps the deployment decision rule fixed and evaluates smoothed and unsmoothed models on the same held-out examples. Inspect the metrics that correspond to that interface rather than inferring probability quality from the training loss alone.

Thresholds deserve revalidation after changing the objective. A threshold chosen for an unsmoothed model does not retain the same operational meaning merely because the architecture and class labels remain unchanged. The score distribution can move even when top-class accuracy changes little.

The same caution applies when replacing a model checkpoint in an existing service. Label smoothing is a training-time choice, but its effects can cross the serving boundary whenever consumers depend on logits or probabilities. Treating the smoothing definition as model metadata makes that dependency visible and keeps confidence-sensitive behavior testable.

Artificial Intelligence Neural Networks Model Training Classification