Synthetic Data

Filter Synthetic Training Data with Rejection Sampling

Generating synthetic examples is easy; generating synthetic examples that are worth training on is harder. A language model can produce thousands of candidate answers, but blindly adding them to a training set can reinforce factual errors, weak reasoning, unwanted style, or artifacts of the generator itself. Rejection sampling provides a simple mental model for controlling that pipeline: generate one or more candidates, evaluate each candidate with an acceptance rule, and keep only candidates that pass. The acceptance rule might use deterministic checks, a learned reward model, another language model, human review, or a combination of signals.