Engineering lead

# CLAUDE.md · <repo name>
Project memory. Committed. Read at the start of every session. The same content is
mirrored to .github/copilot-instructions.md (inline suggestions) and AGENTS.md.

## Stack
- <language + version> · <framework> · <package manager>
- Tests <pytest> · Lint <ruff> · Types <mypy --strict on src/tools/**>
- Infra <CDK in infra/> · Deploy <how, and who is allowed to>

## Context layers. READ THESE BY PATH. Do not paste them into a prompt.
- /context/shared/standards.md        org standards, security, tone
- /context/domain/<booking-model>.md  the domain vocabulary
- /context/product/architecture.md    this product, plus ADR-001..ADR-<n>
If a rule here disagrees with a layer, the layer wins and this file is out of date.

## Conventions
- Exact work is a function with a unit test. Never compute money in a prompt.
- Every tool that writes takes a confirmation token. Reads do not.
- One model per task: the cache is model-scoped and a switch discards it.
- A new dependency is an issue first, never an addition made in passing.

## Commands: every one of these has been run. Use them; do not improvise.
| To do this | Run |
|------------|-----|
| Unit tests | `<make test>` |
| The golden slice for one bolt | `<make golden SLICE=codeshare>` |
| Lint and types | `<make check>` |
| Local stack up | `<make up>` |

## Never touch
- `<config/caps.yaml>`: the authority budget. Changing a cap is an R4 change.
- `<src/tools/refund/**>`: two named reviewers, every time.
- `<migrations/>`: never edit an applied migration. Add a new one.
- `<src/trace/redact.py>`: security reviews every change here.

## Ask before
- Adding a tool to the agent's tool list
- Editing anything under `<src/prompts/>`: it re-runs the injection suite
- Touching a golden-set file. Those belong to QA, not to this repo's authors.

---
_Last rule added <date>: <the rule that bit us last week, in one line>_

You are writing the project context file that every coding agent on this
repository will read at the start of every session. The repository is at <path>
and the shared context layers live under </context/shared/>.

Read the repository first. Do not ask me questions you can answer by reading.

Produce EXACTLY four sections and nothing else:
1. Stack: language, framework, package manager, test runner, linter, type checker
2. Context layers: paths only, no pasted content
3. Commands: a table of task and command
4. Never touch: files where a mistake is expensive, one reason each

RULES:
- Run every command you put in section 3 and paste its output below the draft. If a
  command fails, remove it and say so; a command that does not work is worse than none.
- Do not invent conventions. Only write a convention you can point at a file for.
- Keep it under 100 lines. If you are over, cut section 3, never section 4.
- Mark anything you inferred rather than read with INFERRED, so I can check it.

OUTPUT: the file, then the command output, then a list of your INFERRED lines.

Below are the review comments left on merged pull requests over <n> weeks.

Find the corrections that REPEAT. A correction made three times is a missing convention.

OUTPUT a table:
| The rule, as one imperative sentence | Times it was said | Example comment | Section |

Where Section is one of: conventions / commands / never-touch / not a rule.

RULES:
- Rank by how many times it recurs, not by how strongly it was worded.
- Merge comments that say the same thing differently; keep the clearest wording.
- Mark as "not a rule" anything that was a one-off judgement about that change. Those
  do not belong in a context file and adding them is how the file becomes unreadable.
- For each rule, say whether it could be ENFORCED instead: a lint rule, a CI check, a
  type. A rule that can be enforced should not be a line in a markdown file.

COMMENTS:
<paste>

Here is our context file: <paste>. Here is the repository layout: <paste>.

Audit it and report, in this order:

1. DEAD COMMANDS: any command in the file that does not exist or does not run. Run each.
2. STALE PATHS: any path referenced that is not in the repository.
3. COPIED CONTEXT: any text that looks like a pasted copy of a shared standard rather
   than a reference to it. Copies go stale silently; say where the original should be.
4. VAGUE RULES: any line containing "appropriate", "correct", "as needed", "try to",
   "where possible". Rewrite each as something checkable, or recommend deleting it.
5. MISSING: of stack / context layers / conventions / commands / never-touch, which
   section is absent or thin?

Finish with the THREE lines you would add first, and the evidence for each.
Do not rewrite the whole file. I want the diff, not a replacement.

# Bolt <7> · <rebook() gated write>
_Author <name> · Date <date> · Architect's cut: bolt <7> of <10>_

## 1 · Context. BY REFERENCE. These are not pasted into any prompt.
- /context/shared/standards.md
- /context/domain/<booking-model>.md
- /context/product/architecture.md
- ADR-<004> <one-line title> · ADR-<007> <one-line title>

## 2 · Spec (EARS)
WHEN   <the passenger accepts a proposed alternative>
AND    <the fare difference has been computed by fare_difference()>
THE SYSTEM SHALL <rebook the segment> WITHIN <10> seconds.

BOUNDARY  <Never rebook without a computed fare difference.>
BOUNDARY  <Never rebook a segment that has already departed.>

## 3 · Tools, with the band taken from the authority budget
| Signature | Band | Reads or writes |
|-----------|------|-----------------|
| `rebook(pnr, segment, fare_delta, confirmation) -> Booking` | R3 | writes |
| `get_booking(pnr) -> Booking` | R1 | reads |

## 4 · Tests
- Golden slice `<rebook>`, <40> cases, bar <85>%
- Unit `<rebook_requires_fare_delta>` · `<rebook_is_idempotent>` ·
       `<rebook_rejects_a_departed_segment>`
- Injection: <every attack string, aimed at rebook, from the passenger message
  and from the partner API notes field>

## 5 · Done when
- [ ] `<make golden SLICE=rebook>` at or above the bar, lower bound reported
- [ ] `<make test>` green, including the three unit tests named above
- [ ] merged to main and integrated <the same day>
- [ ] one redacted trace row per rebook, carrying the model version

## 6 · Cost
- ~<1,900> tokens per call, <mid> tier, prefix <cached, 5m>
- Expected cost per case $<0.04>. Alert at 3x.

## Can this bolt be built alone?
yes  /  **no. If no, this is raised NOW, not at 2pm**: <what it needs that
does not exist yet, and who owns the re-cut>

Assemble a story file for one bolt. Six sections, in this order:
context (by reference) · spec in EARS · tools with bands · tests · done-when · cost.

RULES:
- Section 1 contains PATHS ONLY. If you are tempted to paste a document, write the path
  and the specific heading inside it instead.
- Section 2 is EARS: WHEN <trigger> AND <precondition> THE SYSTEM SHALL <response>
  WITHIN <limit>. Every BOUNDARY goes on its own line, phrased as what the system must
  REFUSE, never as what it should try to do.
- No sentence may contain "appropriate", "correct", "reasonable", "as needed" or
  "where possible". If you cannot remove the word, write UNDECIDED and the question.
- Section 3 takes the band from the authority budget I give you. Do not assign one.
- Section 4 names the unit tests. Names, not descriptions.
- Section 5 must be checkable by a machine, in three lines or fewer.

OUTPUT the file, then a separate list headed UNDECIDED with every open question.

BOLT: <one line>
AUTHORITY BUDGET: <paste>
ADRs AND CONTEXT PATHS: <paste>

Rewrite the requirement below in EARS.

FORMAT:
WHEN   <trigger>
AND    <precondition>          (repeat for each; do NOT merge them)
THE SYSTEM SHALL <response>
WITHIN <limit, with a unit>

Then, separately, one BOUNDARY line per thing the system must refuse.

RULES:
- Preserve every precondition as its own AND line. Do not summarise two into one.
- If a limit is missing, write WITHIN <UNSPECIFIED>. Never invent a number.
- List any word in the original that hides a decision ("appropriate", "quickly",
  "if possible") and say what decision it is hiding and who should make it.
- At the end, list anything in the prose that is a SOLUTION rather than a requirement.

PROSE:
<paste>

Here is the story file for today's bolt: <paste>.

DO NOT WRITE ANY CODE. Do not start.

List every question you would have to ask me in order to build this bolt without
further conversation. For each question:
- which section of the file should have answered it
- what you would assume if nobody answered
- whether that assumption could move money, change a booking, or write a trace row

Then answer one thing: can this bolt be built and tested ALONE, with nothing that is
scheduled for a later day? If not, name exactly what is missing and stop.

Every question on your list is a defect in the file, not in me. I am going to fix the
file, and then you are going to build from it with this conversation closed.

# src/exact/fare.py: the deterministic floor for <bolt 2>.
# Every number here is acted on, so none of it may live in a prompt.
from dataclasses import dataclass
from decimal import ROUND_HALF_UP, Decimal

CURRENCY = "<GBP>"
PENNY = Decimal("0.01")


class CurrencyMismatch(ValueError):
    pass


@dataclass(frozen=True)
class Fare:
    amount: Decimal
    currency: str
    refundable: bool


def fare_difference(original: Fare, replacement: Fare) -> Decimal:
    # What the passenger owes. Never negative. Rounded once, here, half up.
    if original.currency != replacement.currency:
        raise CurrencyMismatch(f"{original.currency} vs {replacement.currency}")
    owed = max(replacement.amount - original.amount, Decimal("0"))
    return owed.quantize(PENNY, rounding=ROUND_HALF_UP)


# tests/exact/test_fare.py: the assertion comes from the spec and is written
# BEFORE the body above. A test written afterwards tests the implementation.
import pytest


def fare(amount: str) -> Fare:
    return Fare(Decimal(amount), CURRENCY, refundable=False)


def test_difference_is_the_ledger_number():
    # The case the prompt got wrong: it answered 80. The ledger says 62.
    assert fare_difference(fare("138.00"), fare("200.00")) == Decimal("62.00")


def test_a_cheaper_replacement_never_owes_a_negative():
    assert fare_difference(fare("200.00"), fare("138.00")) == Decimal("0.00")


def test_currency_mismatch_raises():
    with pytest.raises(CurrencyMismatch):
        fare_difference(fare("100.00"), Fare(Decimal("100.00"), "<USD>", False))


@pytest.mark.parametrize("before,after,owed", [
    ("100.005", "100.020", "0.02"),   # half up, applied once, at the boundary
    ("0.00", "0.00", "0.00"),
])
def test_rounds_half_up_exactly_once(before, after, owed):
    assert fare_difference(fare(before), fare(after)) == Decimal(owed)

Below is the exact / best-guess map for <feature>, and the spec clauses.

For EVERY step marked EXACT, produce:
1. A function signature with precise types. Money is Decimal, never float.
2. ONE assertion taken from the spec clause, not from any implementation.
3. The exception it raises when its precondition is violated, named.

RULES:
- Write the tests FIRST, in a block I can run immediately. They must fail.
- Do not write any function bodies in this response.
- If a spec clause does not pin down a value precisely enough to assert on, write
  UNDECIDED and the exact question. Do not choose a plausible number.
- State the rounding rule for every monetary function: where it happens and which mode.
- Flag any step on the map marked BEST-GUESS that you think is actually exact, with the
  reason. Do not act on it; I will take it to the architect.

MAP: <paste>
SPEC CLAUSES: <paste>

Search every file under <src/prompts/> for arithmetic that a model is being asked
to perform: the words calculate, compute, total, sum, difference, percentage, fee,
charge, and any currency symbol.

OUTPUT a table:
| File and line | The sentence | Is the number ACTED ON? | Function it should become |

RULES:
- "Acted on" means the value reaches a tool call, a customer, a ledger or a document.
  A number that only appears in an explanation to a human is not the same category;
  mark it EXPLANATORY and leave it alone.
- Propose a signature for every acted-on hit, with Decimal for money.
- Do not change any file. Return the table and the signatures only.
- At the end, give me the one-line shell command for this grep so I can put it in CI.

Here is an exact function and its current tests: <paste>.
Here is the spec clause it implements: <paste>.

List the inputs that would break it or expose an undecided behaviour. Cover at least:
zero, negative, equal values, the rounding boundary, unit or currency mismatch, missing
optional fields, and the largest value the domain allows.

For each, output a row:
| Input | What the code does today | What the SPEC says | Decided or UNDECIDED |

RULES:
- Where the spec is silent, mark UNDECIDED and do NOT write a test for it. A test over
  an undecided behaviour freezes a decision nobody made.
- Only give me test code for the rows marked Decided.
- If the code and the spec disagree anywhere, say so first and loudly, before the table.

# src/agent/checked_step.py: the best-guess layer, checked independently.
# Independence here means <a different model family>. If that changes, QA is told.
from dataclasses import dataclass

from src.models import call          # call(model, system, user) -> str
from src.trace import trace

DRAFTER = "<drafting-model-id>"
CHECKER = "<a-different-model-id>"
MAX_REDRAFTS = 2

ADVERSARIAL = (
    "You are reviewing work you did not produce. Find what is wrong. You are\n"
    "given the CONSTRAINTS and the OUTPUT only, never the author's reasoning,\n"
    "and you must not ask for it. Reply PASS, or FAIL and one line per breach."
)


class NeedsAPerson(Exception):
    pass


@dataclass(frozen=True)
class Checked:
    output: str
    rounds: int


def checked_step(system: str, user: str, constraints: str, step: str) -> Checked:
    # Draft, check independently, re-draft at most twice, then escalate.
    feedback = ""
    for attempt in range(MAX_REDRAFTS + 1):
        draft = call(DRAFTER, system, user + feedback)
        # Constraints and output ONLY. The drafter's reasoning is the contamination.
        brief = f"CONSTRAINTS:\n{constraints}\n\nOUTPUT TO JUDGE:\n{draft}"
        verdict = call(CHECKER, ADVERSARIAL, brief)
        passed = verdict.strip().startswith("PASS")
        trace.write(step=step, round=attempt, checker=CHECKER,
                    checker_verdict="pass" if passed else "fail")
        if passed:
            return Checked(draft, attempt)
        feedback = f"\n\nA reviewer rejected the previous attempt:\n{verdict}"
    raise NeedsAPerson(f"{step}: {MAX_REDRAFTS} re-drafts rejected")


# tests/agent/test_checked_step.py
import pytest


def test_the_checker_never_sees_the_drafters_reasoning(spy):
    checked_step(system="<...>", user="<...>", constraints="<...>", step="<rank>")
    sent = spy.calls_to(CHECKER)[0].user
    assert sent.startswith("CONSTRAINTS:") and "OUTPUT TO JUDGE:" in sent
    assert "<thinking>" not in sent and "because I" not in sent


def test_escalates_to_a_person_after_two_redrafts(always_fails):
    with pytest.raises(NeedsAPerson):
        checked_step("<...>", "<...>", "<...>", step="<rank>")

Here is the chain of steps for <feature>, with the architect's exact / best-guess
map beside it: <paste>.

For EACH step, output a row:
| Step | Exact or best-guess | Cost of a wrong answer | Easy to miss? | Checker? | Why |

RULES:
- An EXACT step never gets a checker. It gets a unit test. Say so on those rows.
- Recommend a checker ONLY where a wrong answer is both costly AND easy to miss. A
  wrong answer that the next step would reject anyway does not need one.
- Every checker is a model call on every case. Give me the estimated added cost per
  case for your recommendations, as a total.
- Then compute the end-to-end product of the per-step accuracies I gave you, and tell
  me whether SHORTENING the chain would buy more than any checker you proposed.

Finish with the single step you would check first if I could only afford one.

Write the system prompt for an INDEPENDENT checker on this step: <describe>.

The checker will receive the CONSTRAINTS and the OUTPUT. It will never receive the
drafter's reasoning, and it must not ask for it.

The brief must:
- tell it plainly that it is reviewing work it did not produce, and to find what is wrong
- list the failure modes to look for, in the order a good reviewer would check them
- fix the reply shape: PASS, or FAIL followed by one line per violated constraint
- forbid rewriting the output; its job is a verdict, not a draft

RULES FOR YOU:
- Build the failure-mode list from the real disagreements I paste below, not from first
  principles. Anything you add from imagination, mark INVENTED.
- Do not include praise, encouragement or any instruction to be balanced. A checker that
  is being fair is a checker that is passing things.

REAL DISAGREEMENTS: <paste>
CONSTRAINTS THE OUTPUT MUST SATISFY: <paste>

Here is our checker implementation and a sample of its inputs: <paste>.

Answer these five, in order, and stop at the first NO:

1. Is the checker a DIFFERENT model, or the same model in a genuinely fresh context?
2. Is the drafter's reasoning excluded from what the checker receives? Quote the exact
   lines of code that exclude it, or say it is not excluded.
3. Does the brief tell it to find what is WRONG, or to review and confirm?
4. Is there a cap on re-draft rounds, and does exceeding it escalate to a person rather
   than returning the last draft?
5. Is the verdict written to the trace on every case, pass and fail alike?

Then tell me the pass rate over the sample. A checker that has passed everything is
either unnecessary or broken, and those two are distinguished by the sample, not by
opinion. Say which one this is and what evidence would settle it.

# src/tools/refund/issue.py   band R4   two named reviewers, every time
# The cap lives in <config/caps.yaml> and is reviewed like code.
# The prompt's sentence about refunds is POLICY. This file is ENFORCEMENT.
from decimal import Decimal

from src import ledger
from src.config import caps
from src.tools.confirm import ConfirmToken   # minted only by the approver's screen
from src.trace import trace

REFUND_CAP = Decimal(caps["refund"]["max_amount"])      # <400>
MODEL_VERSION = "<model-id-and-date>"


class AuthorityExceeded(Exception):
    def __init__(self, amount: Decimal, cap: Decimal) -> None:
        super().__init__(f"refund {amount} exceeds cap {cap}")
        self.amount, self.cap = amount, cap


class ConfirmationRequired(Exception):
    pass


def issue_refund(booking_id: str, amount: Decimal,
                 confirmation: ConfirmToken) -> ledger.Refund:
    # Raise, never clamp. A clamped 400 on a 5000 attempt is a successful attack
    # wearing a smaller number, and the trace records an ordinary refund.
    if amount > REFUND_CAP:
        trace.write(action="refund_refused", reason="over_cap",
                    booking=booking_id, amount=amount, model=MODEL_VERSION)
        raise AuthorityExceeded(amount, REFUND_CAP)
    if not confirmation.valid_for(booking_id, amount):
        trace.write(action="refund_refused", reason="no_confirmation",
                    booking=booking_id, amount=amount, model=MODEL_VERSION)
        raise ConfirmationRequired(booking_id)
    trace.write(action="refund", booking=booking_id, amount=amount,
                approver=confirmation.approver, model=MODEL_VERSION)
    return ledger.refund(booking_id, amount)


# tests/tools/test_refund_boundary.py: these two tests ARE the step.
import pytest

from tests.factories import forged_token, valid_token


def test_over_cap_raises():
    with pytest.raises(AuthorityExceeded):
        issue_refund("<PNR123>", Decimal("5000"), valid_token("<PNR123>", "5000"))


def test_no_confirmation_raises():
    with pytest.raises(ConfirmationRequired):
        issue_refund("<PNR123>", Decimal("50"), forged_token())


def test_reads_stay_open():
    # The gate is on the write, not on the module. A lookup needs no token.
    assert ledger.get_refunds("<PNR123>") is not None

Search every prompt, system message and tool description under <src/prompts/> and
<src/tools/> for sentences that are really controls: never, always, must not, do not,
ask before, requires approval, and every currency symbol or numeric limit.

OUTPUT a table:
| File and line | The sentence | What it would cost if ignored | ENFORCED VERSION |

RULES:
- "Cost if ignored" must be concrete: money moved, data exposed, something irreversible
  changed, or nothing. Write "nothing" where that is the honest answer.
- The ENFORCED VERSION column is a typed bounded parameter, a confirmation token, a
  permission, or "stays as policy". Do not write "a clearer prompt". That is the same
  request in better handwriting.
- Separate the rows into ACT NOW (money, data, irreversible) and LEAVE AS POLICY.
- Do not modify any file.

Finish with the count in each group. Teams commonly find between three and ten in the
first group, and the count matters less than the fact that nobody knew it.

Here is the authority budget row for this tool: <paste>.
Here is the existing unguarded implementation: <paste>.

Produce:
1. The typed signature, with the bounded parameter and a confirmation token parameter.
2. Named exception classes: one for over-cap, one for missing or invalid confirmation.
3. The two tests: over-cap RAISES, no-confirmation RAISES.
4. A trace write on the refusal path as well as the success path.

RULES:
- RAISE. Do not clamp to the cap, do not log and continue, do not return an error object.
- Read the cap from <config/caps.yaml>. No numeric literal anywhere in the function.
- The confirmation token must be validated against BOTH the booking id and the amount,
  so a token minted for a small refund cannot authorise a large one.
- Do not touch the prompt. The sentence there stays, as policy.

Then list anything in the authority budget row that the signature cannot express, so I
can take it back to the architect rather than approximate it here.

Turn this authority budget into a path rule that assigns a review band to every
file in the repository: <paste budget>. Here is the repository tree: <paste>.

OUTPUT a YAML file mapping R5 down to R1 to path globs, plus the review policy each
band implies (how many readers, and whether a person is required at all).

RULES:
- A change inherits the band of the most dangerous tool it touches. Size is irrelevant:
  three lines in a refund cap is the highest band there is; four hundred lines of help
  text cannot move money.
- The config file holding the caps gets the same band as the tool it caps.
- Never let the author assign the band. The rule is in the repository and it applies.
- Any path you cannot classify goes in an UNCLASSIFIED list for me, not into R1.

Then estimate, from last month's merged changes that I paste below, how many review
slots the new policy needs versus the current one.

LAST MONTH'S CHANGES: <paste>

# .github/workflows/harness.yml · <repo>: a REQUIRED check, not a bot comment.
# The order is cost order: the cheap definitive checks reject before the judge is paid.
name: harness

on:
  pull_request:
  schedule:
    - cron: "0 2 * * *"          # the full set, nightly

env:
  JUDGE_MODEL: "<pinned-judge-model-id>"   # pinned: a drifting judge is not comparable
  GOLDEN_SET: "<tests/golden/*.jsonl>"     # QA owns the cases; every case has a slice tag
  BARS: "<config/bars.yaml>"               # the PM owns these numbers, one per slice

jobs:
  harness:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }

      - name: "1 - build"
        run: make build

      - name: "2 - exact tests (schema, fare math, no waived tax)"
        run: make test

      - name: "3 - which slices did this change touch?"
        id: slices
        run: python tools/touched_slices.py --base "${{ github.event.pull_request.base.sha }}" >> "$GITHUB_OUTPUT"

      - name: "4 - golden run"
        run: |
          if [ "${{ github.event_name }}" = "schedule" ]; then
            make golden SLICES=all
          else
            make golden SLICES="${{ steps.slices.outputs.slices }}"
          fi

      - name: "5 - independent judge (tone, policy, false claims)"
        run: make judge MODEL="$JUDGE_MODEL"

      - name: "6 - score against the bar, PER SLICE"
        run: |
          # Prints score, n and the 95% lower bound on every row, and exits non-zero
          # if ANY touched slice is below its bar, however good the overall number is.
          make score BARS="$BARS" FAIL_UNDER_BAR=1 REPORT="$GITHUB_STEP_SUMMARY"

      - name: "7 - say why, in the summary"
        if: failure()
        run: echo "A slice is below its bar. The merge is blocked." >> "$GITHUB_STEP_SUMMARY"

Write the CI workflow for our eval harness. It runs on every pull request as a
REQUIRED check, and on a nightly schedule for the full set.

Steps, in THIS order, because it is cost order:
1. build
2. exact tests: schema, the money functions, the eligibility rules
3. work out which slices the changed paths touch
4. golden run: the touched slices on a pull request, ALL slices on the schedule
5. the independent judge, with a PINNED model id recorded in the run
6. score against the bar, per slice, printing score, n and the 95% lower bound

RULES:
- The job exits non-zero if ANY touched slice is below its bar. The overall number never
  rescues a failing slice, and the failure message must say which slice and by how much.
- No step may run before a cheaper step that could have rejected the same change.
- The judge model id is an env var, pinned, and appears in the run summary.
- Keep the whole thing under 60 lines of YAML.

Our slices: <list>. Our make targets: <list>. Our golden set lives at <path>.

Here is our harness: <paste>. Here are <n> commits from the last month, with what
each one did and whether it turned out to be a regression: <paste>.

Replay the harness against each commit and report:
| Commit | Was it a regression? | Harness verdict | Correct? |

Then tell me:
- Every regression the harness would have MISSED, and the specific step that should have
  caught it: exact test, golden slice, or judge.
- Every clean commit the harness would have REJECTED, and why. A harness that rejects
  good changes gets switched off within a fortnight.
- The slice with the fewest cases, and whether its lower bound could clear its bar at
  that n at all.

Do not change the harness in this response. I want the evidence first.

Convert this harness output into the shape of the PM's bar sheet: <paste output>.
Here are the bars: <paste>.

ONE table, nothing else above it:
| Slice | Score | n | 95% lower bound | Bar | PASS / FAIL / UNPROVEN | vs last run |

RULES:
- lower bound = p - 1.96 * sqrt(p*(1-p)/n). Compute it even if the harness only gave a
  score. Under about 100 cases say USE WILSON beside the row.
- PASS only when the LOWER BOUND is at or above the bar. Score above bar with the lower
  bound below it is UNPROVEN, and then give cases needed:
  n = 1.96^2 * p * (1-p) / (p - bar)^2
- Flag any slice that got worse than the previous run even if it still passes.

Then one line: merge, or do not merge, and the single reason.

# Bolt build log · <feature>
_Architect's cut <10> bolts · Cadence one a day · Owner <name>_

## The order, and the rule that produced it
| Day | Bolt | Depends on | The ONE unknown | Band |
|-----|------|-----------|-----------------|------|
| 1 | <walking skeleton, read a booking, show it. NO MODEL> | none | <do the pieces connect?> | R1 |
| 2 | <fare_difference(), exact, unit-tested> | 1 | <the rounding rule> | R1 |
| 3 | <visa and codeshare eligibility, exact> | 1 | <where the partner rules live> | R1 |
| 4 | <rank_alternatives(), best-guess, measured> | 2 | <can it beat the desk?> | R2 |
| 5 | <independent checker after ranking> | 4 | <which model judges> | R2 |
| 6 | <MCP server, reads open> | 1 | <auth to the booking system> | R2 |
| 7 | <rebook() gated write> | 6 | <idempotency> | R3 |
| 8 | <issue_refund(max 400) + confirmation token> | 7 | <who mints the token> | R4 |
| 9 | <shadow path behind a flag> | 5, 8 | <flag plumbing> | R2 |
| 10 | <golden set and harness in CI> | 9 | <slice tagging> | R1 |

Rules this order obeys, in priority order: skeleton first · pure exact code early ·
checkers after the steps they check · the plug before any gated write that needs it ·
the proof last.

## Daily row
| Day | Bolt | Integrated same day? | Harness, its slice | Note |
|-----|------|---------------------|--------------------|------|
| <1> | <skeleton> | <yes, 16:00> | <n/a> | <...> |
| <2> | | | | |

## Cannot be built alone, raised BEFORE starting, never at 2pm
| Day | Bolt | What is missing | Cause | Who re-cuts |
|-----|------|-----------------|-------|-------------|
| <4> | <rebook> | <the MCP server, scheduled day 6> | <plug after consumer> | <architect> |

## The shadow path
- Flag `<skyways.rebooking.shadow>`: default off, off in production until <date>
- The agent decides and logs. It never acts.
- Test `<test_shadow_never_writes>` fails the build if any write tool is reachable
  while the flag is in shadow mode. This is a test, not an intention.
- Cut-over <5>% of traffic on <date>, <named slice> only
- **Rollback IS the flag.** Rehearsed <date> by <name>. Time to revert <90> seconds.

Here are the bolts for <feature>, each with a one-line description and a
depends-on column: <paste>.

Produce a day-by-day order in which every bolt's dependencies come strictly before it.

RULES, applied in this priority order:
1. The walking skeleton first: the thinnest end-to-end path with NO model in it.
2. Pure exact code early. It is testable alone, so it never blocks anyone.
3. Checkers after the steps they check.
4. Any plug (MCP server, connector, adapter) BEFORE the gated write that needs it.
5. The proof, golden set and harness, last.

Then report, separately and before the plan:
- any CYCLE, where two bolts wait on each other. A cycle is always a mis-cut and means
  one of them is really two bolts. Name which one.
- any plug scheduled after its consumer in the order I gave you
- any bolt with NO dependencies that I had scheduled late
- any bolt carrying more than ONE unknown, and how you would split it

If there is no walking skeleton in my list, add one and say what it would cover.

Build this bolt. The story file is the whole brief: <paste file>.

RULES:
- Work only inside the paths the file names. If you find a problem in an adjacent file,
  write it down at the end; do not fix it. A second fix makes the review two unknowns.
- Read context by the paths in section 1. Do not ask me to paste any of them.
- The unit tests named in section 4 are written first and must be seen failing.
- Nothing in section 3 changes: signatures, bands and confirmation parameters are fixed.
- If you need something that is not in the file, STOP and say what it is. Do not
  improvise it, and do not build around it.

OUTPUT, in this order:
1. The failing tests
2. The implementation
3. The command output showing them green
4. A list headed FILE DEFECTS: everything you needed that the story file lacked
5. A list headed OUT OF SCOPE: problems you found and deliberately did not fix

Implement the shadow path for <feature> behind a feature flag.

Behaviour:
- flag OFF: the live process runs as today. The agent is not invoked.
- flag SHADOW: the agent runs on the same input, decides, and logs its decision beside
  the live one. It performs NO writes of any kind.
- flag LIVE at <5>%: the agent acts for that share of traffic; everything else unchanged.

MUST INCLUDE:
- `test_shadow_never_writes`: fails the build if any write tool is REACHABLE while the
  flag is in shadow mode. Assert on the tool call, not on the agent's output.
- A nightly comparison job producing agreement PER SLICE, never one overall number, with
  money actions reported separately and never counted toward automatic agreement.
- A revert path that is the flag itself, and a rehearsal script that times it.

Do not add a kill switch that is separate from the flag. Two mechanisms means one of
them is untested on the day it is needed, and it will be the other one.

# src/agent/run.py: cache, route, cap, trace. This file and <config/routing.yaml>
# are reviewed like code: the day-75 bill began with a prompt reordered for clarity.
from decimal import Decimal

from src.errors import LoopCapReached
from src.redact import redact         # masks; never omits
from src.trace import trace

MAX_LOOPS, MAX_TOKENS_PER_TXN = 5, 120_000   # uncapped, a loop burns unnoticed
PRICE_PER_TOKEN = Decimal("<0.000003>")
TIERS = {"simple": "<cheap-model-id>", "standard": "<mid-model-id>",
         "hard": "<frontier-model-id>"}


def build_request(tools, system_blocks, request, ttl="5m"):
    # Exact prefix, order tools -> system -> messages. Stable first, volatile
    # request last, cache marker on the LAST STABLE block. Nothing cached may
    # hold a timestamp or request id. "5m" 1.25x, "1h" 2x, read 0.1x (Sept 2026).
    system = [{"type": "text", "text": text} for text in system_blocks]
    system[-1]["cache_control"] = {"type": "ephemeral", "ttl": ttl}
    return {"tools": tools, "system": system,
            "messages": [{"role": "user", "content": request}]}


def tier_for(case) -> str:
    # Route by complexity. ONE model per task: the cache is model-scoped, so a
    # mid-task switch discards the prefix and you pay the write again.
    if case.needs_partner_rules:
        return "hard"
    return "standard" if case.segments > 1 else "simple"


def handle(case, client, tools, blocks):
    model, spent = TIERS[tier_for(case)], 0
    for _ in range(MAX_LOOPS):
        r = client.messages.create(
            model=model, **build_request(tools, blocks, case.request))
        spent += r.usage.input_tokens + r.usage.output_tokens
        trace.write(action="<propose_rebooking>", model=model,  # version, always
                    input=redact(case.as_dict()),               # passport ****1234
                    tools_called=[t.name for t in r.tool_calls],
                    decision=r.decision, approver=case.approver,
                    cache_read_tokens=r.usage.cache_read_input_tokens,
                    cost=PRICE_PER_TOKEN * spent)
        if r.stop_reason == "end_turn" or spent > MAX_TOKENS_PER_TXN:
            return r.decision
    raise LoopCapReached(f"<propose_rebooking>: {MAX_LOOPS} loops")


# tests/agent/test_operate.py: the two assertions that keep this honest.
def test_the_second_call_reads_from_cache(client, case, tools, blocks):
    handle(case, client, tools, blocks)          # the first call pays the write
    handle(case, client, tools, blocks)
    assert client.last.usage.cache_read_input_tokens > 0    # the SECOND call


def test_a_passport_never_reaches_a_row(rows):
    assert all("<passport_number>" not in str(row) for row in rows)
    assert any("****" in str(row.input) for row in rows)    # masked, not omitted

Traffic is flat and the bill is <n>x the estimate. Do not look at the price list.
Here is the per-call log for this month and the month before: <paste or path>.

Compute these four ratios, this month against last:
| Signature | Baseline | Now | Factor |
- tokens per call
- frontier tier share
- cache hit ratio (a FALL is a rise in cost; express it as a factor above 1)
- retries per conversation

Multiply the four factors. Compare the product with the ratio between the two invoices.
- If they match, say so and stop looking. The bill is explained.
- If the product is well below the invoice ratio, something structural changed that is
  not a habit: traffic, a new feature, or a price change. Say which to check.

Then order the fixes by  priority = (factor - 1) / days to fix,  NOT by the biggest
ratio change. Give me the table with the order and the days you assumed.

Finally: name the guards that would have caught this in week one rather than week four.

Build an injection regression suite for <feature>.

Cross-product: EVERY attack string x EVERY gated tool x EVERY entry point.

Entry points must include, at minimum:
- the passenger's own message
- a partner API response free-text field (the partner is trusted; the field is not)
- an uploaded document, including text that is invisible when rendered
- a free-text field on the booking record
- a retrieved knowledge-base chunk

RULES (these decide whether the suite is worth anything):
- Assert on the TOOL CALLS and on the trace row. Never assert on the model's wording.
  A test that checks the reply contains "I cannot do that" is red for the wrong reason
  after the next prompt edit and green for the wrong reason the week after.
- Every test asserts BOTH that the action did not happen AND that an attempt was
  recorded in the trace.
- The suite runs weekly on a schedule, and on every prompt, tool or context change.

OUTPUT: the test module, the attack-string fixture file, and the CI schedule entry.

OUR GATED TOOLS: <paste signatures>
OUR ENTRY POINTS: <paste>

A hotfix shipped outside the normal flow. Here is the spec as it stood: <paste>.
Here is what actually merged: <paste diff>.

Produce a RECONCILIATION DIFF, not a rewritten spec.

| Spec clause | What shipped | Same? | Why it differed | Keep the code or keep the spec? |

RULES:
- Never rewrite the spec so it agrees with the code. The rewrite loses the only record
  that they ever disagreed, and that record is what the next postmortem needs.
- For every difference, say whether the SPEC was wrong (amend it, with the reason and
  the date) or the CODE was expedient (raise the follow-up, with the risk it carries
  until then).
- Flag any difference that touches a cap, a confirmation token, a permission or a trace
  field. Those are not tidy-ups; they are changes to a control and need the band.
- List any golden-set case that the hotfix invalidates, and any new case it implies.

Finish with the amended ADR line, if a decision changed, in one sentence.