A classifier trained on a long-tailed dataset can see thousands of examples from common classes and only a handful from rare ones. Ordinary empirical risk minimization gives the common classes more influence simply because they appear more often. A tempting fix is to weight each class by the inverse of its example count, but that can make a tiny class disproportionately influential, including any mislabeled examples it contains.

Class-balanced loss based on the effective number of samples provides a smoother way to derive class weights. Instead of treating every additional example as equally informative, it models diminishing returns within a class and weights classes according to an adjusted, or effective, sample count.

This article develops the calculation from a small example, explains what the smoothing parameter controls, and shows how to evaluate the method without confusing class reweighting with threshold tuning or data-quality fixes.

Raw frequency creates an optimization imbalance

Consider a three-class classifier trained on:

common:  10,000 examples
medium:   1,000 examples
rare:       100 examples

With an unweighted per-example cross-entropy loss, every training example contributes according to the same loss rule. Because the common class supplies 100 times as many examples as the rare class, it can contribute far more total gradient signal over an epoch.

That is not automatically wrong. If deployment traffic has the same class distribution and every error has the same cost, frequency-weighted training may be appropriate. The problem appears when the product objective values minority-class performance more than raw training frequency does.

A common response is inverse-frequency weighting:

weight_c proportional to 1 / n_c

For the counts above, the rare class receives 100 times the weight of the common class. This fully compensates for the count ratio at the class-total level, but the correction can be aggressive. One mislabeled rare example can then produce a large weighted loss.

Effective sample counts introduce a tunable middle ground between no count correction and inverse-frequency-style correction.

The effective number grows with diminishing returns

For a class containing n examples, define its effective number as:

E(n) = (1 - beta^n) / (1 - beta)

where:

0 <= beta < 1

The corresponding class-balanced factor is the reciprocal:

w(n) = (1 - beta) / (1 - beta^n)

Weights are usually normalized afterward so their overall scale is convenient for the chosen loss reduction.

The formula becomes easier to understand if we expand the geometric series:

E(n) = 1 + beta + beta^2 + ... + beta^(n-1)

The first example contributes 1 unit to the effective count. The next contributes beta, the next beta^2, and so on. When beta is close to one, later examples still add substantial effective count. When beta is smaller, their marginal contribution decays more quickly.

This is a modeling choice, not a claim that the literal information content of every dataset follows a geometric series. The useful property is that the effective count grows more slowly than the raw count, so very large classes receive diminishing additional credit.

Work through a small numerical example

Use beta = 0.9 and compare classes with 1, 2, and 10 examples.

For one example:

E(1) = (1 - 0.9^1) / (1 - 0.9)
     = 1

For two examples:

E(2) = (1 - 0.9^2) / 0.1
     = 1.9

For ten examples:

E(10) = (1 - 0.9^10) / 0.1
      ~= 6.513

The ten-example class has ten raw observations but an effective count of about 6.513 under this setting.

The unnormalized reciprocal weights are therefore approximately:

n = 1   -> 1 / 1.000 = 1.000
n = 2   -> 1 / 1.900 = 0.526
n = 10  -> 1 / 6.513 = 0.154

Compare that with raw inverse-frequency weights:

n = 1   -> 1.000
n = 2   -> 0.500
n = 10  -> 0.100

For this teaching example, effective-count weighting still favors the smallest class, but the ten-example class is not downweighted as strongly as it would be under exact inverse frequency.

Production datasets often use beta values much closer to one than 0.9; the smaller value here only keeps the arithmetic readable.

Beta controls how strongly counts saturate

The parameter beta determines the shape of the correction.

At beta = 0:

E(n) = 1

for every non-empty class. Every class therefore receives the same unnormalized weight. At the class-weight level, this is a strong correction because raw class size no longer changes the weight assigned to each example through this formula.

As beta approaches one, for fixed finite n:

E(n) -> n

and therefore:

w(n) -> 1 / n

So values near one make the formula approach inverse-frequency weighting for fixed class counts. Intermediate values produce a smoother saturation curve.

This limit is easy to misinterpret. A larger beta does not mean “less reweighting” in every informal sense. What matters is the resulting relative weights for the actual class counts. Calculate and inspect those weights rather than reasoning from the parameter name alone.

A practical diagnostic is a small table before training:

class       count       raw weight       normalized weight
common      10000       ...              ...
medium       1000       ...              ...
rare          100       ...              ...

If the rare-to-common ratio is far more aggressive than intended, change the weighting recipe before spending a training run.

Normalize weights deliberately

Multiplying every class weight by the same constant does not change their relative importance, but it can change the numerical scale of the reported loss and, depending on the exact reduction and optimizer setup, the scale of gradients.

A common convention is to normalize C class weights so their mean is one:

w_normalized[c] = C * w[c] / sum_j w[j]

Then:

mean class weight = 1

This makes the scale easier to compare with an unweighted baseline, but it does not make optimization behavior identical. Batch composition changes the average weight of the examples present in each update, and adaptive optimizers, gradient clipping, regularization, or learning-rate schedules can interact with the changed gradients.

The loss implementation matters too. Some framework functions that accept class weights normalize a mean loss using the weights of the observed targets rather than simply averaging already weighted per-example losses. Others expose several reduction modes. Check the API contract before assuming a particular denominator.

Apply the weight to the training objective

For a single-label classifier with target class y, ordinary cross-entropy is:

L = -log p_y

A class-balanced version applies the target class’s weight:

L_CB = w_y * (-log p_y)

Suppose the model assigns the true class probability p_y = 0.2. The unweighted loss is:

-log(0.2) ~= 1.609

If the target belongs to a rare class with normalized weight 2.0, its weighted contribution becomes:

2.0 * 1.609 ~= 3.219

If a common class has weight 0.5, the same prediction error contributes:

0.5 * 1.609 ~= 0.805

The weighting changes training pressure, not the model architecture. It tells the optimizer that errors on some target classes should contribute more strongly to the objective.

That distinction is useful when debugging. Class-balanced loss cannot create features that are absent from the input, repair incorrect labels, or guarantee that a rare class has enough variation to generalize.

Another way to change class influence is to alter how examples are sampled. Oversampling a rare class causes its examples to appear in more updates; loss weighting keeps the sampling stream unchanged but scales their contributions when they appear.

These approaches can produce different optimization behavior even when their expected class contributions look similar. Oversampling repeatedly exposes the model to the same minority examples, which may increase overfitting when the class is tiny. Weighting avoids literal duplication but can produce high-variance gradients if rare examples receive large weights.

You can also combine sampling and weighting, but doing so can accidentally correct the same imbalance twice. For example, if a rare class is heavily oversampled and also receives a large inverse-count weight, its effective influence may become much larger than intended.

Track the complete pipeline:

raw class frequency
-> sampling probability
-> per-example loss weight
-> batch reduction
-> optimizer update

Reasoning about only one stage can hide the actual class contribution.

Class-balanced loss is not focal loss

Class-balanced weighting and focal loss address different signals.

Class-balanced weighting derives a factor from class frequency. Two examples from the same class receive the same class factor even if one is easy and the other is difficult.

Focal loss derives a modulation from the model’s confidence on an example, reducing the relative contribution of examples the model already handles confidently. It can also include class weighting, but the focusing term itself is about prediction difficulty rather than class count.

The two ideas can be combined, but that creates a stronger intervention. Before combining them, establish which failure you are trying to fix. If the model performs poorly on rare classes because they are underrepresented, class weighting is directly relevant. If a huge number of easy examples dominates the objective even within a class, focal-style modulation addresses a different mechanism.

Evaluate the metrics that motivated reweighting

If the reason for reweighting is poor minority-class behavior, overall accuracy is an incomplete evaluation. A model can improve rare-class recall while losing some common-class accuracy and leave the aggregate metric almost unchanged.

Inspect per-class precision and recall, a macro-averaged metric when appropriate, and the confusion patterns that matter for the application. If error costs differ by class, evaluate those costs directly rather than assuming a balanced metric represents them.

Also evaluate on a dataset whose distribution matches the question you are asking. A class-balanced training objective does not require a class-balanced test set. If you need expected production performance, preserve or reconstruct the relevant production distribution in evaluation. If you need equal attention to each class, report a metric that explicitly gives classes equal influence.

Probability calibration deserves separate attention when downstream code interprets scores as probabilities. Reweighting the training loss changes the objective and can change score calibration. A classifier with better minority recall is not thereby guaranteed to emit probabilities that match deployment frequencies.

Watch for rare-class label noise

Large weights amplify both useful and harmful gradients. This becomes especially visible when a rare class contains mislabeled, duplicated, or ambiguous examples.

Imagine a class with only 20 examples. If two are mislabeled, they represent a meaningful fraction of the class before any weighting is applied. Giving the class a much larger loss weight also magnifies those incorrect targets.

Before increasing minority weights aggressively, inspect rare classes manually when feasible, check duplicate patterns, and verify the labeling process. More sophisticated weighting cannot compensate for a class whose supervision is unreliable.

The same caution applies to extremely small classes. If a class has one example, the main limitation may be lack of coverage rather than optimization balance. Collecting representative data can be more valuable than making that one example dominate more updates.

Empty classes need an explicit policy

The formula assumes n > 0. For an empty class:

1 - beta^0 = 0

so the reciprocal weight is undefined.

An empty training class cannot be learned from ordinary supervised examples in the first place. Decide whether the class should be removed from the current label space, supplied with data, or handled by another mechanism. Do not silently add a tiny epsilon to the denominator and pretend the underlying data problem has disappeared.

Also distinguish a class that is absent from one mini-batch from a class that is absent from the training dataset. Class weights should normally be derived from the intended training population or a documented sampling scheme, not recomputed from each mini-batch. Batch-level counts would make the objective fluctuate with incidental batch composition.

When effective-count weighting is worth trying

This method is a reasonable candidate when the training distribution is strongly long-tailed, minority classes matter to the product objective, and exact inverse-frequency weighting appears too aggressive or unstable.

It is less useful when class frequencies already reflect the desired error trade-off, when the main problem is a decision threshold rather than learned representation, or when rare-class data are too noisy or sparse to support generalization. For binary systems in particular, threshold tuning after training can sometimes reach the required precision-recall operating point without changing the training objective. Test that simpler option when it matches the problem.

Treat beta as a hyperparameter tied to the class-count distribution, not as a universal constant. Compare against an unweighted baseline and simpler alternatives such as explicit class weights or resampling. Keep the evaluation protocol fixed so you can tell whether the extra machinery buys anything.

Make the weighting policy visible

Class-balanced loss is easiest to use well when the weighting policy is treated as part of the model specification rather than a hidden training detail. Record the class counts used to compute weights, the beta value, the normalization rule, the sampler, and the exact loss reduction.

Then inspect the resulting weight ratios before training and evaluate the finished model on both aggregate and per-class behavior. Effective sample counts give you a smooth way to control how raw frequency translates into optimization pressure, but they do not decide what errors your application should value. That decision still belongs in the evaluation and product requirements.

Artificial Intelligence Machine Learning Classification Model Training