voice-agent
Voice Appointment Agent
A phone line that answers itself: the caller speaks normally and the program books, moves, or cancels an appointment, asking for whatever details are missing and never giving away a slot that is already taken. It was proved by driving a real three-turn booking over the open internet into a real calendar entry; the live phone number was never used, so no human voice has tested it yet.
A three-turn booking conversation was driven through a real HMAC-SHA1-signed request (the same algorithm Twilio uses, computed from the real auth token) over a public internet tunnel, ending in a real row written to the SQLite calendar. The live phone number itself was left untouched throughout.
Overview
A phone agent that answers a real Twilio line and books, reschedules, or cancels appointments in natural conversation. Gemini handles the language, a SQLite calendar handles real conflict detection, and the whole multi-turn flow has been proven end to end over a real public tunnel with a genuine Twilio signature, not just unit-tested in isolation.
Key features
- Multi-turn slot filling for booking, rescheduling, and cancelling, with an explicit confirmation step before anything is written.
- Real conflict detection: overlapping and back-to-back adjacent appointments are handled correctly, verified with an explicit boundary test.
- Business-hours and past-date rejection, with a path for the caller to correct a bad time and try again.
- A retry cap and an absolute turn cap that hand off to a human message instead of looping forever on unclear speech.
- An outbound test-call CLI that refuses to run without a public URL and requires explicit confirmation before it places a real, billed call.
- A fully implemented (not stubbed) Google Calendar adapter behind the same interface as the default SQLite store, used only if calendar credentials are supplied.
Results
| Metric | Result |
|---|---|
| Full test suite | 54 / 54 passed |
| Scripted evaluation scenarios | 25 / 25 passed |
| Real public-tunnel booking conversation | 3 / 3 signed turns accepted, 1 real appointment written |
| Real Gemini calls (build + live verification) | 12 total, under $0.01 |
| Real Twilio cost incurred | $0.00 (read-only lookups only, no call placed) |
Reading the numbers honestly
The 25 scripted scenarios run against a deterministic stand-in for Gemini, documented in its own code as a test double, not a production fallback, specifically so a 100+-call evaluation harness doesn't quietly become a recurring bill. We deliberately broke the retry-cap and turn-cap logic once to confirm those two scenarios actually fail when the invariant is broken, then reverted it, since a scenario suite that cannot fail is not proof of anything.
The real public-tunnel test is the one piece of evidence in this project that isn't a unit test: a genuine HMAC-SHA1 Twilio signature, computed from the real auth token, was sent over the actual public internet to a live-booted instance of the app, and Gemini correctly resolved a relative date ("tomorrow") against the real current date across three turns before writing a real database row. The one thing that test still doesn't cover is an actual human voice call: Twilio's real speech-to-text output is noisier than the clean, hand-typed text used here, and that gap is stated plainly rather than glossed over.