Structured Output

Constrain LLM Output with Grammar-Guided Decoding

Asking a language model to return JSON, SQL, or another structured format creates a failure mode that ordinary prompting cannot remove: the model can understand the requested format and still generate a token that makes the output syntactically invalid. For applications that immediately parse model output, one missing quote or delimiter can turn an otherwise useful answer into an error. Retrying helps, but it spends more inference time without guaranteeing that the next attempt will parse.