A game: run a ninety‑day AI project. Ninety Days, the SkyWays simulator

The game needs script to run. Without script, here are its thirteen decisions as text: what has happened on each day, the question, and every option with its price in days and what it leads to. The calculators and the case in depth are in the workbench.

Day 1. Six airline managers have given the project team 31 requirements for the AI assistant. Design starts this week.

It is the first week, and the project team has met the six managers twice.

Arjun: Six people gave us 31 requirements. Four of them are one requirement, said four ways.

Arjun: And compliance has hardly spoken.

What should the team do with the 31 requirements?

• Start the design from the 31 as written (no days). Design starts this afternoon. Coach's note: Thirty-one lines from six people will overlap and contradict each other. Designing from them moves those arguments into design reviews, where they cost the most. A day of merging first shows who asked for what. Later: The first design review became an argument about which line wins. The quiet stakeholder held the rule that mattered.
• Merge them into one list that keeps who asked (1 day). 31 lines become 12 requirements and 9 quality targets, with the limits still unsorted. Each keeps its source. Compliance has no line yet, and now the gap shows. Coach's note: A merged list with a name on every line is something six people can check. The names are what let each manager see their words survived. The missing compliance line is the find: a gap seen now costs a conversation.

Day 4. Six airline managers must approve one list of their 31 requirements. No manager has seen the list yet.

The project team wrote the list after two meetings with the six managers.

Arjun: I have the list. The six of them have not seen it.

Priya: Each of them will look for their own words first.

What should the team send to the six managers?

• The full minutes of both meetings (1 day). Eleven pages go out. One of the six replies. Coach's note: Eleven pages ask six busy people to find their own words. Most will not, and silence then passes for agreement. Agreement you can rely on needs a short list each person can say yes to. Later: Silence was taken as agreement. In week five it turned out that it was not.
• The list, each line credited to who raised it (1 day). Four people see their words merged into one line, with their names beside it. All six confirm by Friday. Coach's note: People confirm what they recognise. Their own words, merged and credited, take minutes to check. Six written yeses now are worth more than any meeting later.

Day 6. Three things set limits on the AI assistant: a refund law, a slow booking system and a partner airline contract.

The six managers have not yet agreed any target for speed, safety or cost.

Compliance officer: Every refund over $400 needs a named approver. That one is the law.

Arjun: The booking system is old and slow. And our partners cap how often we can call them.

When should the team set the three limits against the targets for speed, safety and cost?

• After the targets are agreed, in one pass (no days). The limits go in a folder for later. Coach's note: Limits decide which targets are even possible. Agree the targets first, and the room can promise what the old booking system cannot deliver. Sorting the limits takes a day; running the workshop twice takes more. Later: The room agreed a response time the old booking system cannot deliver. The workshop has to run again.
• Now, sorted by kind, before anything is agreed (1 day). The limits come out of the folder before the workshop. Six lines were called a must in the meetings, and some of them are only wishes. Coach's note: A limit nobody can negotiate belongs on the table before anyone promises a number. Sorting by kind separates the law and the old system from preferences. Some of the six musts will turn out to be wishes.

Which of these can move? All six were called a must in the meetings. Tick what nobody in this building can change inside ninety days. Everything left unticked goes to the Day 9 workshop to be ranked.

• Refunds over $400 need a named approver. A limit: the law. Refunds: a named approver signs anything over $400.
• It must answer in five seconds. A wish. It goes to the workshop to be ranked.
• The booking system is slow and is not being replaced. A limit: the system. Speed: 30 seconds for 95 in 100 requests, which the old system can meet.
• It must never hand a passenger to a person. A wish. It goes to the workshop to be ranked.
• Partner airlines cap how often we may call them. A limit: a contract. Partner flights: partner seats cached inside the cap.
• It must cost less per case than a phone call. A wish. It goes to the workshop to be ranked.

Day 9. The AI assistant has nine quality targets, such as speed and cost. The nine targets cannot all come first.

Six airline managers have one afternoon to put the nine targets in order.

Arjun: Nine quality targets, six people, one afternoon.

Finance controller: They cannot all come first.

How should the six managers decide which target comes first?

• Dots on a wall: everyone votes, and the count decides (1 day). The room is done by three o'clock. Cost per case gets two dots. Coach's note: A dot vote measures what is popular in the room that afternoon. Cost rarely wins a popularity contest, and cost is what ends projects. A ranking needs each person to judge importance and difficulty on their own. Later: Nobody set an alert on cost, because the vote put it last. The bill grew for weeks before anyone looked.
• Each person ranks what matters and what is hard; merge the six (2 days). Nine targets, signed. Three score high on both importance and difficulty. Those three are where a design decision is owed. Coach's note: Judging each target on importance and difficulty finds the few that need real design work. The three that score high on both are where a decision record is owed. Signed targets also become the yardstick for later choices.

Day 12. The AI assistant's design has three hard decisions. Each decision gains one thing and gives up another, and no decision is written down.

Design has started, and the team has never recorded a design decision in writing.

Arjun: Three of our decisions cut both ways. The team has never written one down.

Sam: So how many do we write?

How many of its design decisions should the team record in writing?

• One for every design choice the team has made (4 days). Forty records by Friday. Coach's note: Records are for trade-offs someone will question later. Forty of them bury the three that matter, and nobody reads a pile. Write one where something real was given up, and keep the rest in a design note. Later: Nobody read the forty-first record. The three that mattered are buried, so the framework choice starts from nothing.
• One for each real trade-off; the rest go in a design note (1 day). Two are written today and the third is dated for week three. Each names what was given up. Coach's note: A decision record earns its place when it names what was given up. Written while the reasons are fresh, three of them will answer next month's questions. The one dated for week three is a promise, so give it an owner.

Day 15. Today the product manager signs off what the AI assistant must do. The engineers will build exactly what is signed.

The requirements run to thirty pages, and the engineers' coding agent builds whatever it is given.

Priya: The requirements document is thirty pages.

Sam: My coding agent will read none of it. It builds what it is given, to the letter.

Which document should the engineers build the AI assistant from?

• The thirty pages, with a summary on top (no days). The build can start on Monday. How much the assistant may do alone is left to whoever writes the code. Coach's note: A coding agent builds exactly what it is given, gaps included. Thirty pages leave the big questions to whoever writes the code. The riskiest of them here is how much the assistant may do alone.
• One page: what it may do, how right it must be, who signs (2 days). Five things turned out to be undecided: the model's role, what it may do alone, how right it must be, the fallback, the records. Each is settled with its owner. Coach's note: This is the sign-off that matters most: the engineers will build exactly this page. Five open questions surfaced, and each now has an owner. Two days here cost less than any of them surfacing in production.
• Stop everything until every open question is closed (5 days). Eleven meetings. Eight of those questions could have run beside the build, each with an owner and a date. Coach's note: Some questions block the build and some can run beside it. Stopping for all of them spends days the project cannot get back. Settle what the build depends on, and give the rest an owner and a date.

Day 20. The AI assistant needs a software framework to run on. The team can build a framework, buy one or borrow a free one.

The engineering lead already favours one framework, and the choice will last for years.

Sam: We can build the agent framework, buy one, or borrow an open one. I know which I like.

Arjun: Liking it is not a reason we can defend in a year.

How should the team choose the framework?

• Go with the most senior engineer's choice (no days). Chosen in ten minutes. Coach's note: Seniority is a fine tiebreaker and a poor reason. Without written reasons, the first serious problem reopens the choice. Scoring against the signed targets leaves a reason anyone can check. Later: Nobody can say why this framework was chosen, so the incident reopens the choice.
• Score the three on the signed targets and the cost over three years (2 days). The criteria come from the signed targets, so the weights are already agreed. The record names what reversing the choice would take. Coach's note: The signed targets make this quick: the criteria and their weights are already agreed. Three-year cost catches a cheap start that turns expensive. Writing down what reversing would take gives you an exit plan.
• Pause the build until all three are trialled (6 days). Three trials, six working days. The answer matches the scoring. Coach's note: Trials feel thorough. Here they cost six working days and gave the answer the scoring gave. Trial when the scores are close, or when a criterion cannot be judged on paper.

Day 30. The engineers start building the AI assistant today. Seven things about the build are still unknown.

Design is finished, and the first two weeks of building are already booked.

Sam: Seven things we do not know yet, and a two-week sprint already on the calendar.

What should the engineers build first?

• The model call, drafting a reply to one real case (1 day). A draft reply appears on day two. It is wrong, and it is not clear why. Coach's note: Starting with the model feels like progress, because a reply appears on day two. When it is wrong, the fault could be in any part around it. Prove the plumbing first, so a wrong answer can only be the model's. Later: When the draft was wrong it could have been the prompt, the booking adapter or the search tool. Three days went on telling them apart.
• The thinnest version that runs end to end, with no model in it (1 day). A booking shows on a screen by four in the afternoon. The plumbing is proven, and each day after retires one unknown. Coach's note: A thin version with no model proves the plumbing: the booking, the screen, the hand-off. Each day after it retires one unknown. When the model goes in, any new fault has one place to look.

Day 45. On its first test the AI assistant got 82.4 percent of 500 real cases right. The team promised 80 percent.

The assistant is due to start answering real passengers on Monday.

Maya: 500 real cases. 412 right. That is 82.4 percent.

Priya: And we promised 80. So we pass?

What should the QA lead report about the score?

• A pass: 82.4 beats the 80 we promised (no days). Everything goes to 5 percent of passengers on Monday. Coach's note: A score from 500 cases is a range, and 82.4 could really be 79. A bare pass sets an expectation the live number may miss by chance. Say the low end of the range, then judge each kind of case. Later: The live rate read 80. It was always inside the margin. The team spent a fortnight hunting a fault that was not there.
• 82.4, at least 79.1, then each kind of case on its own (2 days). Probably above the bar, and not yet proven. Saying so is what makes the next number believable. Coach's note: Stating the margin is what makes the next number believable. An average can hide one kind of case that fails badly. Reading each kind on its own shows where the assistant is ready and where it is not.
• Hold every kind of case until it clears one bar of 98 (5 days). Nothing ships. Same-day changes were ready weeks ago and wait with the rest. Coach's note: One bar for every kind of case treats a same-day change like a refund. The simple changes then wait behind the hardest cases. Set each bar by what a mistake costs in that kind of case.

Day 60. Nine changes to the AI assistant's code are waiting to be reviewed. Two of the nine changes alter how refunds are paid.

Two senior engineers read every change, and the queue is four days long.

Sam: Nine changes are waiting for review. The queue is four days long.

Maya: Two of the nine touch money.

How should the team review the nine changes?

• Ask the reviewers to go faster until the queue clears (no days). They do, for a week. Coach's note: Asking people to read faster lowers attention where it is needed most. The two changes that move money deserve the slowest reading of the nine. Speed comes from routing, so careful eyes go where the risk is. Later: The change that touched the refund limit got a ninety-second reading on a Friday. It has to be pulled back and read again.
• Route by risk: two readers, one, or the automatic checks alone (1 day). The two money changes are read first, by people who are no longer tired. Coach's note: Most changes carry little risk and can go through on the automatic checks. That frees two fresh readers for the changes that move money. Review effort should follow what a mistake would cost.
• Keep two senior readers on every change (no days). Nothing changes. Every change still waits for the same two people. Coach's note: Two senior readers on everything sounds careful. In practice the queue grows, and urgent fixes wait behind trivial ones. Match the reading to the risk, and senior time goes where it counts. Later: The queue kept growing. Three fixes sat behind a typo correction.

Day 75. The bill for the assistant's AI model is 4.4 times the estimate. Passenger numbers are flat.

The assistant now answers real passengers, and the funding decision is fifteen days away.

Lena: The model bill is 4.4 times the estimate. Traffic is flat.

Finance controller: Then what changed?

What should the team look at first to explain the bill?

• A comparison of cheaper models (5 days). Three weeks of trials. The habits behind the bill carry on at a lower unit price. Coach's note: A cheaper model lowers the unit price and leaves the habits that drive the bill. Without the log of every model call, nobody can say what changed. Find the cause first; the cheapest saving is usually a habit. Later: Finance asks why the bill is still more than twice the estimate. Nobody can say which habit is behind it.
• The log of every model call (1 day). Four things moved, and they multiply: 1.6 × 1.5 × 1.3 × 1.41 is 4.4. That is the whole bill. Coach's note: A bill that grows while traffic is flat is a behaviour problem, so look at behaviour. The log shows four habits that multiply into the whole 4.4. Fix the habits, and the price question often answers itself.
• Switch the assistant off until it is understood (no days). The assistant is off for a week. The contact centre goes back to the phones. Coach's note: Switching off stops the spending and the learning together. The bill is a puzzle the logs can solve while the assistant keeps serving passengers. Keep it on, find the cause, then change one habit at a time. Later: A week back on the phones. The contact-centre head brings it up in the room, and the bill was never explained.

Day 82. The AI assistant tried to refund $2,000 to a passenger who was owed $180. The refund tool refused the payment.

By law, a refund over $400 needs a named person to approve it.

Contact-centre head: A passenger was owed $180. The assistant tried to refund $2,000.

Maya: The tool refused it at $400. The limit is in the code.

What should the team do about the refund that was refused?

• Close the case as blocked (no days). The case is closed. Nobody asks why the assistant wanted $2,000. Coach's note: A refusal that worked is still a near miss worth reading. Why did the assistant try to pay $2,000? Each near miss written down is an incident you will not have. Later: Ines asks what was learned from the refund that was refused. Nothing was written down.
• Add the case to the test set, and review what it may do alone (1 day). Six cases like it join the test set. The limit held, and now there is a test that says so every week. Coach's note: The limit held, so this lesson is cheap to learn today. As a test, the case makes the limit prove itself every week. Reviewing what the assistant may do alone keeps the next attempt smaller.

Day 82. The AI assistant paid $2,000 to a passenger who was owed $180. Nothing stopped the payment.

By law, a refund over $400 needs a named person to approve it.

Contact-centre head: A passenger was sent a $2,000 refund that was not owed.

Maya: The $400 limit was written in the prompt. The refund tool accepts any amount.

What should the review of the wrong payment ask first?

• Who wrote that prompt, and who approved it? (1 day). A name is found and a training session is booked. Coach's note: Finding a name feels like accountability. It leaves the refund tool as it was, so the same payment can happen again. Ask what control would have made this impossible, then build it. Later: The refund tool still accepted any amount. It happened again in week thirteen.
• What limit in the code would have made this impossible? (2 days). A limit and a confirmation step go into the tool itself. The assistant may do one level less alone. Six cases like it join the test set. Coach's note: A limit in the code holds whatever the prompt says. A rule the model only reads can be talked around, as this case showed. Moving the limit into the tool turns a lesson into a control.

Day 90. The executive who sponsors the project asks what ninety days achieved and what the work cost.

A committee of airline executives decides today whether to keep funding the assistant.

Ines: Ninety days. What did we get, and what did it cost?

The slide for the committee has room for three lines. Which three should go on the slide?

• What was saved, in person-days and in money.
• What it cost: the model bill and the review time.
• The net for the cycle, and one target for the next.
• The score on the test set.
• Any money paid out in error.
• How many documents are on file.

Leave the cost off the slide and finance finds it three weeks later.