Cross-entropy gives every classified example a loss determined by the probability assigned to its target class. When a training batch contains many examples the model already classifies with high confidence, their individual losses may be small yet their aggregate contribution can still occupy a substantial part of the objective. Focal loss changes that balance with a confidence-dependent multiplier.
The mechanism is not a new classifier head or sampling strategy. It modifies the loss so that examples with high target-class probability are attenuated more strongly than examples with low target-class probability.
The modulating factor depends on target confidence
For a labeled example, let p_t denote the probability assigned to the target class. Ordinary cross-entropy is:
CE(p_t) = -log(p_t)Focal loss adds a factor controlled by gamma >= 0:
FL(p_t) = -(1 - p_t)^gamma * log(p_t)When gamma = 0, the factor is one and the expression reduces to ordinary cross-entropy. For positive gamma, the factor approaches zero as p_t approaches one. A confident correct example therefore contributes less loss than it would under plain cross-entropy.
The effect is milder when p_t is low. If p_t = 0.1 and gamma = 2, the multiplier is 0.81. If p_t = 0.9, the same multiplier is 0.01. These values describe the multiplicative term only; the complete focal loss also contains -log(p_t).
This probability dependence is the defining feature. Focal loss does not simply assign a fixed weight to a class or example category.
Modulation changes gradients as well as reported loss
Multiplying cross-entropy by (1 - p_t)^gamma changes the derivative used for parameter updates. The modulating factor itself depends on the model output, so its derivative contributes to the final gradient.
An implementation that computes cross-entropy and then multiplies its detached numeric value by a focal factor would not represent the same objective if the focal factor is also detached from the computation graph. The intended loss requires gradient flow through the probability-dependent modulation.
This distinction matters when custom loss code is assembled from intermediate tensors. A scalar loss value can look plausible while its gradient differs from the mathematical expression being implemented.
For numerical stability, implementations commonly derive the required probabilities from stable log-softmax or binary cross-entropy computations rather than applying logarithms to probabilities that have already rounded to zero. The exact implementation depends on whether the task uses a mutually exclusive softmax output or independent sigmoid outputs.
Class weighting solves a different allocation problem
Focal loss is often used in settings with severe class or example imbalance, but focal modulation and class weighting are separate mechanisms.
A class weight such as alpha_t changes the contribution according to the target class:
FL(p_t) = -alpha_t * (1 - p_t)^gamma * log(p_t)alpha_t can increase or decrease the contribution of a class regardless of current model confidence. The focal term instead changes continuously with p_t.
Combining them can be appropriate when both effects are desired, but the parameters should not be treated as interchangeable. Increasing gamma does not directly specify a class ratio, and changing class weights does not reproduce confidence-dependent attenuation.
Sampling policies are distinct again. Oversampling a minority class changes which examples enter optimization and how often. Loss weighting changes the objective contribution of examples that are already present. Those choices can interact, so applying all of them without accounting for their combined effect can produce much stronger reweighting than intended.
Large gamma values narrow the active region
As gamma increases, confident examples are suppressed more aggressively. This shifts a larger fraction of the loss toward examples that currently have lower target-class probability.
That shift does not imply that every low-confidence example is informative. A mislabeled sample, ambiguous input, or outlier can also have low p_t. Focal loss has no mechanism for distinguishing a difficult valid example from a corrupted label solely from target confidence.
A large gamma can therefore concentrate optimization on a smaller set that includes both useful hard cases and problematic data. The appropriate value depends on the model, data, label quality, and optimization setup rather than on class counts alone.
This also makes loss inspection useful during experiments. Aggregate focal loss can decrease while the remaining gradient mass becomes concentrated on a small subset of examples. Examining the distribution of p_t and per-example loss can expose that concentration more directly than a single batch average.
Multiclass and multilabel forms require different probabilities
In single-label multiclass classification, softmax produces a distribution whose class probabilities sum to one. p_t is the softmax probability of the labeled class, and increasing one class probability necessarily changes the others.
Multilabel classification commonly uses an independent sigmoid for each label. Each output is a binary decision, so the focal construction is applied to each label target using the probability of the observed binary outcome. For a binary target y, that target-aligned probability can be written as:
p_t = p when y = 1
p_t = 1 - p when y = 0The same focal expression can then operate on each binary term. Treating multilabel logits as a single softmax distribution would impose mutual exclusivity that the task does not have.
Library APIs may expose focal loss with different parameter names, reduction rules, weighting conventions, or logits expectations. Those details are implementation-specific. The mathematical behavior should be checked against the API in use rather than inferred from the function name alone.
Probability calibration is not the focal objective
Focal loss deliberately changes the training objective according to model confidence. That does not provide a guarantee that resulting probabilities are calibrated estimates of event frequency.
A classifier can improve a task metric while its probability calibration changes in either direction. If downstream code consumes confidence thresholds, expected costs, or probability estimates, calibration should be evaluated separately on held-out data.
The same separation applies to accuracy. Focal loss can change optimization pressure without guaranteeing a higher top-1 accuracy. Its useful claim is narrower: it reduces the relative contribution of examples that the current model assigns high probability to the target, with the strength controlled by gamma.
That narrow mechanism gives focal loss a clear design boundary. It is most relevant when confidence-dependent allocation of gradient is the intended intervention. If the core problem is incorrect labels, a mismatched class prior, poor probability calibration, or inadequate model capacity, focal modulation does not directly repair that source of error.