Sessions 23–24 · Movement 03
Training &
Handoff
The engagement — test, install, hand over
Sessions 17–28 · The engagement

Twelve sessions. One real deliverable.

17–19

Scoping the Engagement

Scope with the business and lock the plan.

20–22

Building

Sprint the build; adjust as reality lands.

23–24

Training & Handoff

Test it, train them, and set it up in their accounts.

25–28

Buffer & Close

Held in reserve — then review, present and grade.

Sessions 23–25 · The gate

No feature passes without its evidence.

What every AI feature ships with

3+ unit-style assertions · ONE validated binary judge · a one-page error-analysis memo. Binary pass/fail, never 1-to-5 — range scores are nearly impossible to align with human preference. (The non-AI majority of your build proves itself the class-09 way: tests green, checklist rows demonstrated — this gate covers the parts that can be wrong in new ways.)

Validate the judge itself

An LLM judge is a meta-eval — it needs to be evaluated. Measure its true-pass and true-fail rates against held-out human labels before trusting it. (Class 14 gave you the loop; this block runs it on the partner build.)

Criteria drift — the one-hour exercise

You can’t fully specify quality until you grade real outputs — and grading changes the criteria (UIST 2024). So: write criteria → grade 20 real outputs → document what changed. That memo is part of the deliverable.

Compendium Part V — “Eval Engineering and Non-Determinism” · UIST 2024, “Who Validates the Validators?”
Sessions 23–25 · Acceptance

The owner accepts — formally.

Acceptance, executed (class 13 wrote it — now run it)

Deliverable + written criteria + deadline → a 5–10-business-day review window → written acceptance, with a cure period for anything short. The calendar makes the order strict: criteria walkthrough at visit 3, the window closes before the visit-4 install, and anything short gets cured in the remote stretch — documented, the fix demoed over video.

Visit 4 is the install — and the last on-site.Then Thanksgiving week: no visit, no students in the building — the involuntary proof that the tool, the runbook and the named owner can run a week without you. If it breaks that week, the runbook’s first pagealready says who to call: the on-call student pair (remote), the instructor, and the rollback the owner rehearsed at install. Alerts go to the owner and the team channel until handoff completes.
Sessions 23–25 · Year three

Agents made the cheap part cheaper.

The lifecycle split

Development is 20–40% of a tool’s lifetime cost; maintenance is 60–80% — and three-quarters of that is enhancement, not bug-fixing. The part agents accelerated was the small part.

And the long tail is still human

83.2% of maintenance commits on AI-generated files are still performed by humans (2026 study — 1,016 files across 100 repos). The orphaned-app death of the 4GL era runs on the same mechanism: nobody on staff understands the code well enough to touch it.

The escape hatch is regeneration: the durable artifact is the spec + the test suite — the code is a disposable build product a future agent can regenerate. No spec, no tests, no regeneration. And custody is already settled (class 15): spec, tests, repo, and the ops bill all live in the partner’s accounts, on the partner’s card.
The ops floor — 30 minutes, graded, $0–20/month:uptime monitorerror trackingtested backupsdependency alertsthe difference between “the tool is down” and “the tool has been silently down for three weeks.”
Compendium Part VII — “The Year-3 Question” · 2026 maintenance study, dated on-slide
Sessions 23–25 · The honest number

The year-3 talk goes in the handoff doc.

Total cost of ownership, honestly

A $10K-equivalent custom tool loses roughly 3:1 to a $200/month SaaS over three years — unless it replaces three or more subscriptions, kills per-seat pricing, or does what no vendor sells. Write that arithmetic into the handoff doc, with this tool’s actual numbers.

And a named exit path — one of three, chosen with the owner:retainer — market norm 15–25% of build cost per year: for a build a firm would price at $10K, that’s $1,500–2,500/yr — write thistool’s number in the docregeneration — the spec + tests let any future builder rebuild itplanned sunset — a date and a migration, decided now, not discovered“Shipping an unmaintained tool to a small business is shipping a liability.” We don’t.
Compendium Part VII — the year-3 conversation · Part II — “Pricing the Work” §retainers
Connect.AI logoConnect.AI
Sessions 23–24 · Training & Handoff01 / 06
1 / 6
Checking access…