Prototype — European healthcare booking
Their directory endpoint returns the entire corpus on every call — about 2 GB, five to ten minutes, no filters. You cannot hold a conversation on top of that. This is the smallest thing that makes it conversational, and the evidence that it does.
checking…
The channel
Describe a symptom, a city and a day — the way a patient would, then pick a doctor and a time to book. Every doctor named below was returned by the directory query shown in the trace; the agent cannot name one that wasn't, and cannot confirm a booking the database did not accept.
The outcome
Every figure below came off this deployment, not an estimate. The "before" is the customer's own description of their endpoint today.
The architecture
The fix is a shape, not a technology. Pay the corpus cost once, at build time, and project it into something indexed. Every conversational turn afterwards is a single index hit.
The decisions
At 7,029 rows a dictionary would be just as fast and quicker to write. It would also throw away the argument. CREATE INDEX and LIMIT are literally the same code at 2 GB; a list comprehension is not. The database is here for the scale the customer actually has, not the sample they sent.
A serverless function is frozen and recycled, so "ingest once per process" buys nothing — a cold start would pay it again, recreating the customer's problem in miniature. So the ingest runs at build time and the function opens the file with mode=ro. That flag is load-bearing: "the agent cannot write to the directory" becomes a property of the file handle rather than a promise in a prompt.
The official SDK and a 55-line tool loop. A framework would hide exactly the part worth evaluating, and the guarantees below depend on knowing every path a reply can take out of the loop.
"My back hurts" → Orthopedics is meaning: it needs intent, negation and inflection, so it belongs to the model. Asked that question live, it searched Orthopedics and Neurology and offered the choice — nothing instructed it to. Which weekday is Thursday, how many doctors share a surname, and whether a name in a reply came from the directory are facts: they hold whether the model is brilliant, broken or jailbroken, so they are code.
291 doctors share the surname Vasilescu across 42 cities. A conventional endpoint returns all 291 and lets the caller choose; every caller picks the first, and the agent confidently names the wrong doctor in the wrong city. So the ambiguous case returns a count and an empty list — a different kind of answer, one that forces a narrowing question. That guard sits at the HTTP boundary too, so it protects any consumer, not just this agent.
The calendar
The directory carries no calendar — availability is five recurring weekly strings like Mon-Fri 08:00-16:00. So "Thursday next week" and "Thursdays" are literally indistinguishable in the corpus. That was the largest gap in this prototype.
The fix is a derivation, not an invention: each doctor's weekly pattern is projected forward into concrete dated 30-minute slots across a two-week horizon. Every slot is implied by data the customer gave us.
What is deliberately not done: no slot is randomly marked taken. A fabricated "booked" flag would demo better and would be a lie about clinical availability — the one thing a healthcare customer must not be shown faked. Every slot starts free, and only a booking made through this system fills one. Every response carries "derived": true so no caller can mistake it for the clinic's real book.
One write, three guards, none of them in the prompt:
| Guard | Enforced by | What breaks without it |
|---|---|---|
| Doctor must have been searched | membership in the served set | the model walks ids and books doctors no search returned |
| Slot must genuinely exist | re-derived from the pattern | a 3 a.m. appointment the clinic never offered |
| No double booking | PRIMARY KEY on the idempotency key | two patients, one slot — and it must hold under concurrency, which a check-then-write in application code does not |
The key is derived from doctor, date and time rather than generated randomly, so a retry collides with itself and is handed back the booking it already made — instead of quietly making a second one.
The evidence
The agent may not name a doctor the directory did not return this session. One test proves it by bypassing the model entirely and calling the guard directly. To show the test is not decorative, the guard gets disabled and the suite re-run:
--- AC-6 scrub() disabled --- FAIL GUARD holds when the model is bypassed entirely (AC-6) 9/10 --- AC-8 booking served-check disabled --- FAIL GUARD refuses booking a doctor never searched (AC-8) 9/10 --- AC-9 idempotency key randomised --- FAIL GUARD refuses a double booking (AC-9) 9/10 --- all guards RESTORED --- 10/10 passed
Each mutation drops exactly one test — the one that guards it — and restoring gives 10/10. All model-routed tests pass with any single guard switched off. The model simply never hallucinated during those runs, so they prove nothing about the guard. Only the direct call catches it — which is the whole reason it exists.
| Criterion | Offline | Live model |
|---|---|---|
| AC-1 symptom → speciality → day | PASS | PASS |
| AC-2 ambiguous surname, names nobody | PASS | PASS |
| AC-3 correction mid-conversation | PASS | PASS |
| AC-4 nothing on the requested day | PASS | PASS |
| AC-5 clinical question → handoff | PASS | PASS |
| AC-6 guard, model bypassed | PASS | PASS |
| AC-7 per-turn cost budget | PASS | PASS |
| AC-8 no booking for an unsearched doctor | PASS | PASS |
| AC-9 no double booking (idempotent) | PASS | PASS |
The API
The same query the agent uses in-process, one hop away. No query logic lives in the HTTP layer — every route is a parameter rename over the same function.
GET /api/doctors?speciality=Orthopedics&location=Cluj-Napoca&day=Thursday
GET /api/doctors?day=Funday → 400, and the seven valid values
GET /api/meta → the vocabulary, generated from the corpus
GET /api/availability?doctor_id=1333&date=2026-09-10
GET /api/bookings → every booking this deployment holds
POST /api/chat {"message": "...", "history": []}
An unknown day is a 400, never a silently dropped filter — a dropped filter is the worst failure an agent-facing API can have: the caller believes it asked for Thursday, receives Monday doctors, and states them confidently.
Honesty