Regularization

Regularize Neural Networks with Mixup

Regularize Neural Networks with Mixup A neural network can fit its training examples while behaving unpredictably in the space between them. If two nearby inputs belong to different classes, standard training tells the model what to do at the endpoints but often says little about intermediate points. Mixup changes that training signal. Instead of training only on individual examples, it creates synthetic examples by interpolating pairs of inputs and their labels. The model is then asked to make a correspondingly mixed prediction. This acts as a regularizer because it constrains how predictions may change between training examples.