Neural networks are usually trained with gradient descent, which depends on small changes in parameters producing informative changes in the loss. A discrete operation can break that assumption. Rounding a value, choosing a binary gate, or selecting a quantized level may be exactly what the forward computation needs, yet its derivative can be zero almost everywhere or undefined at transition points.
A straight-through estimator (STE) is a practical way to keep training in that situation. The forward pass uses the discrete operation, while the backward pass substitutes a simpler derivative so that a gradient can flow through it. The important consequence is easy to miss: the backward signal is generally not the true derivative of the discrete forward computation. It is a deliberately chosen surrogate.
This article builds that idea from a one-variable example, shows how the surrogate changes optimization, and explains when an STE is useful, what can go wrong, and what to measure before trusting a model trained with one.
Start with the gradient problem
Consider a scalar parameter w that is rounded before it is used:
q = round(w)
loss = (q - 3)^2Suppose w = 1.7. The forward pass gives:
q = round(1.7) = 2
loss = (2 - 3)^2 = 1We would like training to move w so that its rounded value eventually becomes 3.
The problem is the derivative of round. Between rounding boundaries, changing w slightly does not change q. For example, values from 1.5 up to but not including 2.5 round to the same integer under the usual nearest-integer rule, apart from implementation-specific tie handling at exact half values. Inside that interval, the derivative of the rounded output with respect to w is zero.
Applying the ordinary chain rule away from the discontinuities therefore gives:
d(loss)/dw
= d(loss)/dq * dq/dw
= 2(q - 3) * 0
= 0The loss is nonzero, but the parameter receives no local gradient telling it to move toward a better rounding region. At a rounding boundary, the operation is discontinuous, so an ordinary derivative is not available there either.
This is not a failure of automatic differentiation. It reflects the mathematical operation in the forward pass.
The mental model: different rules for forward and backward
An STE keeps the discrete forward computation but changes the derivative used during backpropagation.
For the simplest identity-style STE, pretend during the backward pass that:
dq/dw ≈ 1The forward result remains q = round(w). Only the backward rule changes. For w = 1.7 and q = 2:
d(loss)/dq = 2(q - 3) = -2Using the surrogate derivative:
d(loss)/dw ≈ -2 * 1 = -2A gradient-descent update with learning rate 0.1 becomes:
w_new = w - 0.1 * (-2)
= 1.9The rounded forward value is still 2, so the loss does not improve after this one step. But repeated surrogate-gradient updates can move w across the next rounding boundary. Once that happens, the discrete forward value changes.
That is the core purpose of an STE: provide an optimization direction through a forward operation whose exact local derivative is not useful for ordinary gradient descent.
An STE is a gradient estimator, not a hidden derivative
It is tempting to describe the identity STE as if round somehow had derivative 1 during training. That is misleading. The forward operation has not changed, and its mathematical derivative has not become 1.
Instead, training uses two related computations:
forward: q = discrete(w)
backward: use a chosen surrogate for dq/dwThe resulting update direction is often called a surrogate gradient or coarse gradient. Its usefulness depends on whether that direction helps optimize the actual objective induced by the discrete forward computation.
This distinction matters because different backward surrogates can produce different training behavior while computing exactly the same forward outputs. An STE is therefore part of the optimization design, not merely an implementation trick.
A binary gate makes the trade-off clearer
Suppose a network learns a gate from a real-valued score a:
g = 1 if a > 0 else 0
output = g * branch(x)The hard gate is useful in the forward pass because it can switch a branch completely on or off. But the step function is constant on either side of zero, so its derivative is zero almost everywhere.
One possible STE uses an identity derivative:
backward surrogate: dg/da ≈ 1Another uses a clipped surrogate:
dg/da ≈ 1 when |a| <= 1
dg/da ≈ 0 otherwiseBoth produce the same binary gate in the forward pass. They differ in which scores receive gradients.
The identity rule allows gradients even when a is far from the decision boundary. That can make optimization easy, but it ignores how insensitive the hard gate actually is in those regions. The clipped rule limits updates to a neighborhood around the boundary, which may better reflect where changing a can affect the gate, but it can also stop a badly positioned score from receiving a useful signal.
There is no universal surrogate that is correct for every discrete operation and objective. The choice is an optimization assumption that should be evaluated.
How frameworks usually express the idea
Deep-learning systems can implement an STE with a custom backward rule, but a common teaching pattern uses a stop-gradient operation. Let stop(x) return the value x in the forward pass while contributing zero derivative in the backward pass.
For a discrete function d(x), define:
y = x + stop(d(x) - x)In the forward pass:
y = x + d(x) - x = d(x)In the backward pass, stop(d(x) - x) contributes no derivative, so:
dy/dx = 1The expression therefore behaves like d(x) forward and like the identity function backward.
Equivalent syntax appears in several autodiff systems under names such as stop-gradient or detach. The exact API is framework-specific, so the important reusable idea is the computational structure rather than a particular function name.
This pattern also makes the approximation visible in code: the forward value and backward derivative have intentionally been decoupled.
Where straight-through estimators are useful
STE-style training is most relevant when the deployed or modeled computation genuinely needs a hard or discrete choice but gradient-based optimization is still desirable.
Quantized neural networks
Quantization maps continuous values to a restricted set of levels. A simplified binary weight rule might be:
q(w) = +1 if w >= 0 else -1Optimizing only the quantized value with its exact derivative provides little useful local gradient information. A training procedure can instead keep an underlying continuous parameter and use a quantized version in the forward pass while applying a surrogate derivative backward.
The continuous parameter acts as optimization state. The discrete value is what the forward computation sees.
Real quantization-aware training systems may use more elaborate fake-quantization operators, clipping ranges, scale parameters, and custom gradient rules. The simple binary example explains the mechanism but is not a complete production quantization recipe.
Hard gates and conditional computation
A model may need a binary decision about whether to execute a component. Soft probabilities are differentiable, but they do not reproduce the same forward behavior as an actual on/off gate.
An STE can let the forward pass exercise the hard decision during training while still sending an approximate learning signal to the gate parameters. This can reduce the mismatch between training with a purely soft gate and deploying with a hard one, although the surrogate gradient introduces a different approximation of its own.
Discrete latent representations
Some models map continuous encoder outputs to discrete codes. The selected code is not a smooth function of the encoder representation, so direct backpropagation through the selection can fail to provide a useful gradient to the encoder.
A straight-through path can copy a gradient from the selected representation back toward the continuous encoder output. This lets downstream reconstruction or prediction losses influence the encoder, but it does not make discrete code selection differentiable in the ordinary mathematical sense.
Understand the bias you are introducing
A useful gradient estimator can be biased: its expected value does not have to equal the true gradient of the objective.
For an identity STE through rounding, the mismatch is obvious. The true local derivative is zero almost everywhere, while the surrogate is 1. The method intentionally trades gradient fidelity for a signal that may be easier to optimize with.
Bias alone does not tell you whether training will succeed. What matters operationally is whether the surrogate updates tend to improve the objective and whether they remain stable in the region the optimizer visits.
This leads to a practical rule: do not judge an STE only by whether training loss decreases. Also evaluate the model using the actual discrete forward path that will matter at inference or deployment.
If a surrogate drives continuous parameters into a region that looks favorable only under its backward approximation, training metrics can hide weaknesses in the discrete system.
Watch the scale and range of surrogate gradients
Because an STE replaces a derivative, it directly changes gradient magnitude.
Suppose the downstream gradient arriving at a binary gate is large. An identity STE passes that gradient through unchanged regardless of how far the gate score lies from its threshold. A clipped surrogate can suppress it outside a chosen interval. A smooth surrogate, such as the derivative of a sigmoid-like approximation, can attenuate it according to the score.
Those choices affect:
- how quickly parameters cross discrete boundaries;
- whether extreme values continue moving;
- the effective step size seen by upstream layers;
- sensitivity to the optimizer and learning rate;
- training stability near decision thresholds.
Treat the surrogate rule and its scale as hyperparameters with optimization consequences. If changing the STE requires a major learning-rate change, that is evidence that the backward rule is materially changing the optimization problem.
Separate boundary crossing from value improvement
Discrete objectives often have flat regions. That creates a behavior that looks strange if you inspect only the forward loss after every step.
Return to the rounding example. Moving w from 1.7 to 1.9 does not change round(w), so the forward loss stays at 1. The surrogate gradient can still be useful because it moves the continuous parameter toward a boundary. Improvement appears only after the boundary is crossed.
This means short-term forward-loss changes can be a poor diagnostic for an individual parameter. At model scale, many discrete decisions may cross boundaries at different times, making aggregate loss smoother, but the underlying issue remains.
When debugging, inspect both sides of the mechanism:
continuous parameter or activation
-> discrete forward value
-> downstream loss
surrogate gradient
-> update to continuous valueIf the continuous values move but almost no discrete decisions change, the learning rate, surrogate range, parameterization, or thresholds may be preventing useful boundary crossings.
Common mistakes
Forgetting which computation runs at inference
If deployment uses quantized weights or hard gates, validation should exercise those same forward decisions. Evaluating only a continuous shadow representation can overstate quality.
The STE is normally a training mechanism. It defines a backward rule; it does not require a backward pass at inference.
Treating the surrogate as mathematically exact
An STE does not restore the ordinary chain rule through a discontinuous operation. Describing the surrogate explicitly makes it easier to reason about bias, compare alternatives, and debug optimization.
Applying an STE when a smooth formulation is sufficient
If the application does not require a discrete forward decision during training, a smooth parameterization can be simpler. For example, a soft gate may be appropriate when fractional weighting is meaningful and deployment does not require an exact binary choice.
Using an STE adds an approximation that needs justification. Do not add it merely because a discrete formulation is possible.
Ignoring dead regions in clipped surrogates
Clipping can prevent unrealistic gradients far from a threshold, but it can also create regions where the surrogate derivative is zero. If parameters enter those regions, they may stop receiving the signal needed to return.
Track the distribution of pre-quantized values or gate scores, not just the final task metric.
Assuming all STEs behave alike
Two implementations can have identical forward code and different backward rules. When reproducing a method, the surrogate derivative is part of the algorithm. Record it alongside the discrete operator, optimizer, and other training settings.
How to evaluate an STE in practice
A useful evaluation separates model quality from optimization behavior.
First, measure the task metric with the true discrete forward computation. For a quantized classifier, that means evaluating the quantized model rather than a continuous proxy. For a hard routing system, exercise the actual routing decisions.
Second, inspect optimization diagnostics. Useful signals include gradient norms around the discrete operation, the distribution of continuous pre-discrete values, the fraction of values near decision boundaries, and the rate at which discrete assignments change during training.
Third, compare against a simpler baseline. Depending on the problem, that might be a fully continuous model, a smooth relaxation, or a model trained without the discrete component. An STE is worthwhile only if the hard computation provides a benefit that justifies the added optimization approximation.
Finally, test sensitivity. If small changes to the surrogate range, gradient scale, initialization, or learning rate cause large quality swings, the training procedure may be relying on a fragile gradient approximation.
When to use an STE and when not to
An STE is a reasonable candidate when three conditions hold:
- the forward computation needs a genuinely discrete or hard operation;
- the exact local derivative is unusable for standard backpropagation;
- you can validate the surrogate empirically on the actual discrete objective.
Consider alternatives when those conditions do not hold. A differentiable relaxation may be preferable when soft decisions are acceptable. Other gradient estimators can be appropriate for stochastic discrete variables when their statistical properties match the objective better, although they may introduce variance or additional computation. In some systems, reformulating the model to avoid the discrete training bottleneck is simpler than engineering a surrogate gradient.
The choice is not between a “real” gradient and a universally correct STE. It is between optimization strategies with different assumptions, bias, variance, computational cost, and forward-behavior requirements.
Conclusion
Straight-through estimators solve a specific training problem: a model needs a discrete forward operation, but that operation blocks the local gradient signal required by ordinary backpropagation. The STE keeps the hard forward behavior and substitutes a backward derivative that can move the underlying continuous parameters.
The key mental model is to keep the two passes separate. The forward operation defines what the model actually computes. The backward surrogate defines how the optimizer is encouraged to change it. Because those rules are intentionally different, the surrogate must be treated as part of the optimization algorithm and evaluated rather than assumed correct.
Use an STE when the discrete behavior is genuinely valuable, make the surrogate rule explicit, validate with the real forward path, and monitor whether the approximation produces stable boundary-crossing behavior. That turns a convenient gradient trick into a deliberate engineering choice.