A prompt can explain a task with instructions, but sometimes examples communicate the desired behavior more precisely. Few-shot prompting places a small number of input-output demonstrations in the model’s context before the real input.
This technique is useful when a task has a specific output format, subtle classification boundary, naming convention, or transformation rule that is difficult to describe completely in prose. The model is not retrained by these examples. Instead, it uses the demonstrations as part of the current context when generating the next response.
What few-shot prompting changes
Consider a simple intent-classification task. An instruction-only prompt might say:
Classify each message as billing, account, or technical.
Return only the label.
Message: I cannot sign in after resetting my password.The instruction defines the task, but it leaves some decisions implicit. Adding examples can make those decisions clearer:
Classify each message as billing, account, or technical.
Return only the label.
Message: Why was I charged twice?
Label: billing
Message: Please change the email on my profile.
Label: account
Message: The application crashes when I upload a file.
Label: technical
Message: I cannot sign in after resetting my password.
Label:The demonstrations communicate both the classification behavior and the required output shape. They can reduce ambiguity without changing the model’s parameters.
Few-shot prompting is not fine-tuning
Few-shot prompting and fine-tuning can both influence model behavior, but they operate differently.
With few-shot prompting, examples are included in every relevant request. They consume context tokens and disappear when they are no longer present in the conversation or request.
Fine-tuning updates model parameters through a training process. Its learned behavior persists without repeating the training examples in every prompt.
This distinction matters operationally. Few-shot prompting is usually faster to test and easier to revise, while fine-tuning can be more appropriate when a stable behavior must be learned across many requests and the economics justify training and maintenance.
Choose examples for coverage, not quantity
More examples are not automatically better. A useful demonstration set should represent the decisions the model actually needs to make.
For a classifier, include examples near important category boundaries rather than several nearly identical easy cases. For structured extraction, include representative variations such as missing fields, multiple values, or unusual ordering. For text transformation, show the expected tone and formatting consistently.
A practical set often includes:
- a common, straightforward case;
- one or two cases that distinguish easily confused outcomes;
- an edge case that should still follow the same rule;
- the exact output format expected in production.
Each example should earn the context space it consumes.
Keep demonstrations internally consistent
Conflicting examples weaken the signal. If one demonstration returns a lowercase label and another returns a sentence explaining the label, the model receives competing patterns.
Consistency should cover more than formatting. Check that examples apply the same semantic rule. For example, if password resets are classified as account in one example and technical in another without a clear distinction, the prompt teaches an unstable boundary.
Before adding an example, ask two questions:
- Would a human reviewer assign this output confidently under the stated rules?
- Does this example reinforce the same convention as the other demonstrations?
If either answer is no, fix the example or the instruction first.
Match the production input distribution
Demonstrations work best when they resemble the kinds of inputs the application will receive. Examples written in polished documentation language may provide weak guidance if production inputs are short, noisy support messages.
This does not mean copying user data into prompts. Instead, construct representative examples that preserve relevant characteristics while avoiding sensitive information.
For a multilingual application, evaluate examples in the languages the system must support. For long documents, test realistic lengths. For extraction tasks, include the formatting variations that occur in actual inputs.
Separate instructions from examples clearly
The model should be able to distinguish task rules, demonstrations, and the new input. Stable delimiters and repeated labels help.
For example:
Task: Convert each request into JSON with "action" and "resource" fields.
Example 1
Request: Restart the search service.
Output: {"action":"restart","resource":"search service"}
Example 2
Request: Disable the nightly export.
Output: {"action":"disable","resource":"nightly export"}
New request
Request: Enable the audit worker.
Output:The exact delimiter style matters less than using it consistently. Clear structure also makes prompts easier for developers to inspect and maintain.
Do not let examples override safety or application rules
Examples should demonstrate how to perform the task, not become an alternative channel for hidden instructions. In systems that combine user input, retrieved content, and demonstrations, keep trusted application instructions structurally separate from untrusted data.
When examples contain arbitrary text, delimit that text as data. Avoid demonstrations that accidentally teach the model to follow instructions embedded inside fields that should only be classified, summarized, or extracted.
This is especially important when few-shot prompting is used inside larger retrieval or agent workflows.
Account for context-window cost
Every demonstration consumes input tokens. Longer prompts increase the amount of context processed for each request and can increase latency and cost depending on the model and serving setup.
There is also an opportunity cost: tokens used by demonstrations cannot simultaneously hold conversation history, retrieved evidence, or the user’s document when the request approaches the model’s context limit.
Start with a small, high-value example set. Add examples only when evaluation shows that they improve a failure mode enough to justify their context cost.
Evaluate examples as part of the system
A few examples that look convincing during manual testing may fail across a broader input set. Treat the demonstration set as a versioned component and evaluate it against representative test cases.
Useful measurements depend on the task. Classification can use accuracy or per-class precision and recall. Structured generation can measure schema validity and field correctness. Extraction can compare exact values or task-specific matching rules.
When changing examples, compare the new prompt with the previous version on the same evaluation set. Look for regressions as well as improvements. An example added to fix one edge case can shift behavior on another.
When zero-shot instructions may be enough
Few-shot prompting adds complexity and token usage, so it should solve a real problem. Clear zero-shot instructions may be sufficient when the task is common, the output format is simple, and evaluation already shows reliable behavior.
Prefer adding demonstrations when they clarify a boundary or pattern that instructions alone do not communicate reliably. If many examples are required to achieve acceptable performance, consider whether the task needs better instructions, retrieval, a different model, or fine-tuning instead.
A practical workflow
A disciplined few-shot workflow is straightforward:
- Write the clearest instruction-only prompt you can.
- Build a small evaluation set that includes normal and difficult cases.
- Identify recurring failure patterns.
- Add a minimal number of demonstrations that clarify those patterns.
- Keep example semantics and formatting consistent.
- Re-run the same evaluation set and compare results.
- Remove examples that add tokens without measurable value.
Few-shot prompting is most effective when examples are treated as deliberate specifications rather than decorative prompt content. A small set of representative, consistent demonstrations can communicate task boundaries efficiently while remaining easy to inspect, revise, and evaluate.