Skip to content

EX-17 · Measure position sensitivity for real

Difficulty: 🔴 · label difficulty: 3 Stuck? Ask in the clinic thread for EX-17 — one long-running thread per exercise, so the answers accumulate where the next person will look. Submit: post it in Show and Tell titled [solution · EX-17] ..., with your numbers and your interval.

Notebook 05 · ~2 h · Skill: measuring rather than citing

The offline reader has no position sensitivity. Point the same harness at a real model (Bedrock or Claude) and measure the U-curve on this eval set.

Acceptance criteria

  • The gold chunk forced into position 1, mid, and last — identical evidence set
  • The spread, with an interval
  • Whether edge-interleaving recovers it, measured
  • Cost of the experiment, reported