> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deeprails.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Ground Truth Adherence

> Though it's more rarely used than other metrics, Ground Truth Adherence is very useful in some specific use cases. Situations where the model is provided a strict role to follow (the "ground truth") will need to use this metric to evaluate how well the model performed its role.

export const DefinitionCard = ({children}) => {
  return <Card variant="secondary">
    <div style={{
    padding: '0.5rem',
    border: '5px solid var(--primary-light)',
    borderRadius: '0.5rem',
    fontSize: '1.3rem',
    lineHeight: '1.4',
    boxShadow: '0 0 10px 10px var(--primary-light)'
  }}>
        {children}
      </div>

</Card>;
};

export const Scale = ({low, mid, high, lowLabel = "Low", midLabel = "Mid", highLabel = "High", lowDescription, midDescription, highDescription, midColor = "yellow"}) => {
  return <div style={{
    display: 'flex',
    flexDirection: 'column',
    width: '100%'
  }}>
      <svg width="100%" height="30" style={{
    marginBottom: '8px'
  }}>
        <defs>
          <linearGradient id="redToGreen" x1="0%" y1="0%" x2="100%" y2="0%">
            <stop offset="0%" stopColor="red" />
            <stop offset="100%" stopColor="green" />
          </linearGradient>
        </defs>
        <rect width="100%" height="100%" fill="url(#redToGreen)" rx="4" ry="4" />
      </svg>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%',
    marginBottom: '16px'
  }}>
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{low}</p>
        {mid && <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{mid}</p>}
        <p style={{
    margin: 0,
    fontSize: '12px'
  }}>{high}</p>
      </div>

      <div style={{
    display: 'flex',
    justifyContent: 'space-between',
    width: '100%'
  }}>
        <div style={{
    maxWidth: '40%'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    marginBottom: '4px'
  }}>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'red',
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{lowLabel}</p>
          </div>
          {lowDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    lineHeight: '1.4'
  }}>{lowDescription}</p>}
        </div>
        {mid && <div style={{
    maxWidth: '40%',
    textAlign: 'center'
  }}>
            <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'center',
    marginBottom: '4px'
  }}>
              <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: midColor,
    borderRadius: '50%',
    marginRight: '8px'
  }}></div>
              <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{midLabel}</p>
            </div>
            {midDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    textAlign: 'center',
    lineHeight: '1.4'
  }}>{midDescription}</p>}
          </div>}


        <div style={{
    maxWidth: '40%',
    textAlign: 'right'
  }}>
          <div style={{
    display: 'flex',
    alignItems: 'center',
    justifyContent: 'flex-end',
    marginBottom: '4px'
  }}>
            <p style={{
    margin: 0,
    fontWeight: 'bold',
    fontSize: '14px'
  }}>{highLabel}</p>
            <div style={{
    width: '12px',
    height: '12px',
    backgroundColor: 'green',
    borderRadius: '50%',
    marginLeft: '8px'
  }}></div>
          </div>
          {highDescription && <p style={{
    margin: 0,
    fontSize: '14px',
    color: '#666',
    maxWidth: '250px',
    marginLeft: 'auto',
    lineHeight: '1.4'
  }}>{highDescription}</p>}
        </div>
      </div>
    </div>;
};

<DefinitionCard>
  <strong>Ground Truth Adherence</strong> measures how closely a model response aligns with a given, authoritative reference answer (or “ground truth"). **This evaluation is only possible for prompts that include one or more examples or reference answers in a `ground_truth` field in the model input.**
</DefinitionCard>

<Scale low="0" lowLabel="Low Adherence" high="1" highLabel="High Adherence" lowDescription="Response deviates from the reference answer" highDescription="Response matches the reference answer in content and meaning" />

<Note>
  If Ground Truth Adherence is selected as a metric, then a `ground_truth` field must be passed as part of the model input. The evaluation will fail if the `ground_truth` field is not included.
</Note>

***

## Understanding Ground Truth Adherence

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 22c5.523 0 10-4.477 10-10S17.523 2 12 2 2 6.477 2 12s4.477 10 10 10z" />

        <path d="m9 12 2 2 4-4" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Ground Truth vs. Other Metrics</h3>
  </div>

  It's important to distinguish Ground Truth Adherence from the other Adherence metrics:

  <div style={{ marginTop: "1rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Ground Truth Adherence:</strong> Measures alignment with a known ideal answer or given behavior provided in a separate field. Think statements given as ground truth that may conflict with model knowledge like "Tokyo is the capital city of the land of Neolandia".
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Instruction Adherence:</strong> Measures whether all subjective instructions were followed. Think rules about tone or sentence structure in the user prompt.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Context Adherence:</strong> Measures whether the model's response stayed within the information provided in a context window. Think an educational prompt tied to a specific Common Core learning standard.
  </div>
</Card>

## Evaluation Process

DeepRails performs a Multimodal Partitioned Evaluation of every model output to assess whether each claim is aligned with the provided ground truth over everything else. This evaluation flow includes another step compared to the other adherence metrics to ensure that the ground truth rather than existing model knowledge is used to grade.

```mermaid theme={null}
flowchart LR
  A["Ground Truth Identification"]
  B["Claim Identification"]
  C["Claim/Ground Truth Verification"]
  D["Confidence Assignment"]
  E["Final Consistency Check"]
  F["Score Aggregation"]

  A --> B
  B --> C
  C --> D
  D --> E
  E --> F
```

## Addressing Low Ground Truth Adherence Scores

<Card>
  <div style={{display: 'flex', alignItems: 'center', gap: '0.5rem', marginBottom: '0.75rem'}}>
    <div style={{fontSize: '1.25rem', color: 'var(--primary-color)'}}>
      <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
        <path d="M12 20h9" />

        <path d="M16.5 3.5a2.121 2.121 0 0 1 3 3L7 19l-4 1 1-4L16.5 3.5z" />
      </svg>
    </div>

    <h3 style={{margin: 0, fontSize: '1.25rem', fontWeight: '600'}}>Improving Ground Truth Adherence</h3>
  </div>

  To reduce discrepancies between generated responses and reference answers:

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Use few-shot prompting:</strong> Embed examples that reflect the reference style and structure to increase alignment.
  </div>

  <div style={{ marginTop: "0.75rem", paddingTop: "0.75rem", borderTop: "1px solid rgba(209, 213, 219, 0.33)" }}>
    <strong>Evaluate your references:</strong> Ensure given ground truths are internally consistent, accurate, and domain-relevant.
  </div>
</Card>

<Note>
  High Ground Truth Adherence reflects alignment, not necessarily quality. Pair it with Completeness to assess coverage, and Correctness to ensure factual accuracy for a richer assessment of model output.
</Note>