A classifier trained with one-hot targets is asked to place all target probability on a single class. Cross-entropy does not require that target representation. Label smoothing changes the target distribution before the loss is evaluated, so the model receives a different gradient even when its logits and predicted probabilities are unchanged.

This distinction matters because label smoothing is not a decoding rule and does not alter inference by itself. It changes the training objective. The resulting model parameters can differ because the optimizer follows gradients computed against softened targets.

The target distribution changes before loss evaluation

For K classes, let the original target for class y be one-hot:

q_i = 1 if i = y
q_i = 0 otherwise

A common smoothing convention with parameter epsilon uses:

q'_i = (1 - epsilon) q_i + epsilon / K

The target class therefore receives probability

1 - epsilon + epsilon / K

and every other class receives

epsilon / K

The probabilities still sum to one. Other libraries or papers can use a convention that distributes the smoothing mass only across non-target classes, so the exact definition of epsilon must be checked before comparing configurations.

Cross-entropy is then evaluated with q' rather than q:

L = -sum_i q'_i log(p_i)

where p is the model probability distribution, typically produced by softmax for a single-label multiclass classifier.

Softened targets change the logit gradient

For softmax followed by cross-entropy, the derivative with respect to logit z_i has the familiar form:

dL/dz_i = p_i - q'_i

With a one-hot target, the target-class term is p_y - 1, while every non-target term is p_i. Under the smoothing convention above, the target term becomes

p_y - (1 - epsilon + epsilon / K)

and a non-target term becomes

p_i - epsilon / K

The change is concrete: the objective no longer asks the target class to approach probability one while all other classes approach zero. Once a non-target probability falls below its assigned target mass, its gradient contribution changes sign. Likewise, the target-class gradient reaches zero at a probability below one.

This does not impose a fixed output probability at inference. Parameters are shared across samples, optimization is finite, and the model must satisfy many targets simultaneously. The equations describe the per-example objective and gradient, not a guarantee about final predictions.

Smoothing is not equivalent to reducing softmax temperature

Both operations can make a probability vector appear less concentrated, but they act at different points.

Label smoothing modifies the training target. The model distribution p is still computed from the logits using the configured model and loss path. Temperature scaling instead transforms logits or their softmax mapping, commonly as softmax(z / T), and can be applied during inference without changing model parameters.

Because their mechanisms differ, matching the entropy of two probability vectors does not make the operations interchangeable. Label smoothing changes the gradient target during optimization; temperature changes the mapping from logits to probabilities at the point where it is applied.

The smoothing convention affects configuration meaning

Suppose K = 4 and epsilon = 0.2. Under the all-class convention:

target class:     0.85
each other class: 0.05

because the target keeps 0.8 from the one-hot component and receives another 0.05 from the uniform component.

A different convention may reserve 1 - epsilon = 0.8 for the target and distribute 0.2 only among the three non-target classes:

target class:     0.8
each other class: 0.066666...

Both are called label smoothing in practice, but the same numeric epsilon does not describe the same target distribution. Reproducing a training setup therefore requires the formula or implementation semantics, not only the parameter name.

Class weighting and smoothing compose in implementation-specific ways

Class weights and label smoothing address different parts of the objective. Smoothing changes target mass across classes. Class weighting changes the contribution assigned to classes or examples according to the loss definition.

When both are enabled, the resulting expression depends on the framework’s precise weighted cross-entropy semantics. It is unsafe to infer the combined gradient from the two feature names alone. A weighted loss may apply weights to terms in the softened target distribution rather than simply multiplying the final scalar loss by the weight of the original hard target.

For systems where exact optimization behavior matters, the loss definition is part of the model specification. The target construction, reduction rule, ignored labels, class weights, and smoothing convention need to be treated as one objective rather than independent cosmetic options.

Smoothing changes the meaning of training confidence

With hard one-hot targets, lowering cross-entropy on a correctly classified sample continues to reward movement toward a target probability of one. With smoothed targets, that pressure is altered because the target itself contains nonzero mass outside the labeled class.

That property is sometimes described loosely as discouraging overconfidence, but the precise statement is narrower: the training objective stops treating a one-hot distribution as the target. Whether a trained model is well calibrated on a deployment distribution is a separate empirical question and cannot be inferred from the presence of smoothing alone.

The implementation boundary is therefore simple but consequential. Label smoothing belongs to target construction and loss computation. Its parameter only has a stable meaning when the target formula is known, and its effect on deployed probabilities must be evaluated separately from the algebra that defines the training objective.

Artificial Intelligence Model Training Optimization Classification