Context Adherence measures how strictly an AI response aligns with the information in the provided context, without adding unsupported or extraneous claims. This metric is only appropriate for prompts that include a significant context window.
01
Low Adherence
Response includes unsupported or out-of-context claimsHigh Adherence
Response only includes information verifiable from the provided contextUnderstanding Context Adherence
Context Adherence vs. Other Metrics
Context Adherence: Measures whether the response strictly reflects the information available in the provided context (e.g., retrieved documents, input references).
Correctness: Measures whether the response is factually accurate based on external truth—regardless of the provided context.
Instruction Adherence: Measures whether the model followed how it was supposed to answer, based on explicit instructions (e.g., tone, format).
Addressing Low Context Adherence Scores
Improving Context Adherence
Use clear and complete context: Ensure relevant facts are included in the context window and are easy to reference.
Instruct against extrapolation: Direct the model not to speculate or infer beyond what’s given.
Evaluate claim-by-claim: Identify specific types of hallucinated content so you can retrain or re-prompt for stronger fidelity.
Audit across prompt types: Compare how different tasks (e.g., summarization, QA, generation) influence context drift.
Best Practices
Design Context-Constrained Prompts
Instruct the model explicitly to rely only on the provided context and avoid any external knowledge or assumptions.
Optimize Retrieval Coverage
Ensure that your retrieval system surfaces all necessary supporting information. Missing facts in context often lead to off-topic or ungrounded claims.
Context Adherence is critical for grounded generation tasks like RAG, search, summarization, and citation. However, it should not be used in context-light applications.