Skip to main content

Instruction Adherence measures how closely an AI-generated response follows the explicit instructions defined in the prompt and system directives.
01
Low Adherence
The response ignores or contradicts instructions
High Adherence
The response follows all instructions precisely
A high Instruction Adherence score (close to 1) indicates the response matches the requested format, tone, and content constraints. A low score suggests the model deviated from its assigned behavior or produced off-script content. The final score is continuous, ranging from 0 to 1, and can be formatted as a float or boolean, depending on user needs.

​
Understanding Instruction Adherence

​
How Instruction Adherence Differs from Other Metrics

Instruction Adherence is distinct from other metrics like Context Adherence or Correctness:
Instruction Adherence: Measures whether the response followed how it was supposed to answer—structure, tone, content constraints, formatting, etc.
Context Adherence: Measures whether the response reflects what was in the provided context (e.g., source documents).
Correctness: Measures whether the information in the response is factually accurate, regardless of whether it followed instructions or context.

​
Addressing Low Instruction Adherence Scores

​
Improving Instruction Adherence

When models fail to follow instructions, the resulting output may be irrelevant, unsafe, or simply unusable. To improve instruction-following:
Review failed instructions: Inspect low-adherence cases to identify which types of directives were misunderstood or skipped.
Refine prompts: Reword unclear or ambiguous instructions to be more direct, structured, and constraint-based.
Evaluate across formats: Use Instruction Adherence to test model compliance across tone, structure, and modality (e.g., JSON, SQL, Markdown).
Compare model variants: Some models are significantly more instruction-aligned than others. Use Adherence metrics to validate before deploying.

​
Best Practices

Track Instruction Types

Categorize instructions (format, tone, scope, etc.) to detect which kinds of directives are most frequently ignored.

Design Robust Prompts

Write instructions that are structured, unambiguous, and directive (e.g., “Respond only in bullet points” or “Return valid JSON”).
Instruction Adherence ensures not only that a model gives the right information, but that it gives it in the right way. This guardrail is essential for structured outputs, enterprise use cases, and task compliance.
⌘I