Writing Prompts That Generate Consistent Output

One of the hardest parts of using AI in production is consistency. Ask the same question twice, get two different answers. Ask the API for JSON and sometimes get markdown. Here are the prompt techniques I use to pin down output so the downstream code can rely on it.

Specify the Output Format Explicitly

'Respond in JSON' is not enough. The model might wrap JSON in markdown code fences, add explanatory text, or miss fields. I spell it out:

Respond ONLY with valid JSON matching this schema:
{
  "summary": "string, 2 sentences",
  "key_points": ["string"],
  "sentiment": "positive | negative | neutral"
}
Do not include markdown code fences. Do not add commentary.

The more specific the schema, the more consistent the output.

Use Examples (Few-Shot)

One example beats ten rules. I include a sample input and expected output:

Example input: 'The product launched on time with minor bugs.'
Example output:
{
  "summary": "The product shipped on schedule. Minor issues remain.",
  "key_points": ["on-time launch", "minor bugs"],
  "sentiment": "positive"
}

The model pattern-matches the example format and applies it to new inputs. Consistency shoots up.

Lower Temperature for Structured Tasks

Temperature controls randomness. For creative tasks (0.7-1.0) you want variety. For structured output (0.0-0.3) you want repeatability. I default to 0.0 for anything that will be parsed by code.

Validate and Retry

Even with the best prompt, the model sometimes returns malformed output. My pipeline always validates:

from pydantic import ValidationError
 
for attempt in range(3):
    response = call_model(prompt)
    try:
        parsed = ResponseSchema.model_validate_json(response)
        break
    except ValidationError:
        prompt += "\nYour last response was invalid JSON. Try again."

Three attempts catches 99% of parse failures. Beyond that, the prompt needs fixing.

Use Tool Calling When Available

Modern APIs have a tool/function calling feature that forces structured output. Instead of hoping for JSON, I define a function signature and the model fills it in:

tools = [{
    'name': 'extract_summary',
    'input_schema': {
        'type': 'object',
        'properties': {
            'summary': {'type': 'string'},
            'sentiment': {'enum': ['positive', 'negative', 'neutral']}
        },
        'required': ['summary', 'sentiment']
    }
}]

Tool calling is 10x more reliable than 'please respond in JSON.' Use it whenever the API supports it.

The Meta-Pattern

Treat the prompt like a function signature. Specify the inputs, the output type, and the constraints. The model is the implementation. If the signature is vague, the implementation varies. If the signature is strict, the implementation converges.

See the Anthropic tool use documentation for the structured output pattern that replaces most prompt-only JSON attempts.