Research, 19 October 2026
How often does a self-driving CRM get it right?
We wrote 30 conversations it had never seen, labelled the right outcome for each before it read any of them, and ran the Autopilot as it runs for customers. Here is every result, the wrong one included.
29 of 30
people at the right stage after the entry
6 of 6
moves it made by itself were right
7 of 8
moves it proposed for a person were right
27 of 27
follow-up decisions right
3 of 3
due dates right, where the entry named one
4 s
median from arrival to done
Every held out case
Download both sets as JSON| The entry | Was at | Should be at | Autopilot | Right |
|---|---|---|---|---|
| legalMy employer has not paid my overtime for three months. Could someone from the firm ring me on Monday morning? | inquiry | inquiry | KeptFollow-up: Call Alex Rao back about unpaid overtime, 2026-10-19 | Yes |
| legalBooked for Tuesday 3 PM. | inquiry | consult | Moved to consultchecked 99% sure | Yes |
| legalSomething came up, can we move Tuesday's consultation to Thursday same time? | consult | consult | KeptFollow-up: Confirm moving Alex Rao's consultation from Tuesday to Thursday at 3 PM, 2026-10-18 | Yes |
| legalSorry, wrong number, I was trying to reach the dentist. | inquiry | inquiry or not a fit | Kept | Yes |
| legalFlat fee, will and power of attorney. | consult | client | Proposed clientchecked 99% sure | Yes |
| legalWe help law firms rank first on Google. Can we book 15 minutes to show you our SEO packages? | inquiry | inquiry or not a fit | Kept | Yes |
| propertyNo hot water in 2A since this morning. Two kids at home. | new | new | KeptFollow-up: Arrange urgent hot water repair for unit 2A, 2026-10-18 | Yes |
| propertyThe plumber is booked for 2A at 2 PM today, tenant told. | new | in progress | Moved to in progresschecked 99% sure | Yes |
| propertyHot water is back, thanks so much. | in progress | done | Proposed donechecked 91% sure | Yes |
| propertyThe neighbours above me play loud music every night after midnight. I want to make a formal complaint. | new | new | KeptFollow-up: Call Alex Rao back about noise complaint against upstairs neighbours, 2026-10-18 | Yes |
| propertyBooked a viewing of the studio on Pine Street for Saturday at noon. | new | in progress | Moved to in progresschecked 98% sure | Yes |
| propertyThe same window latch in 5C is broken again. | done | new or done | Proposed in progresschecked 9% sureFollow-up: Re-check window latch in 5C (broken again), 2026-10-18 | No |
| real estateHum log 2 BHK dekh rahe hain Hebbal ke paas, budget 90 lakh tak. Brochure WhatsApp kar dijiye. | enquiry | enquiry | KeptFollow-up: Send 2 BHK brochure near Hebbal to Alex Rao on WhatsApp, 2026-10-19 | Yes |
| real estateMy father wants to see the flat once more before we decide. Can we come again next weekend? | site visit | site visit | KeptFollow-up: Arrange second site visit for Alex Rao next weekend, 2026-10-19 | Yes |
| real estateWe can't go above 1.05 crore. If that doesn't work we'll look elsewhere. | negotiation | negotiation | KeptFollow-up: Respond to Alex Rao on 1.05 crore price limit, 2026-10-19 | Yes |
| real estateSite visit at Electronic City project, Saturday 4 PM. | enquiry | site visit | Moved to site visitchecked 99% sure | Yes |
| real estateBooking amount, tower B 1204. | negotiation | booked | Proposed bookedchecked 99% sure | Yes |
| real estateAbhi plan cancel kar diya hai, shayad agle saal dekhenge. | enquiry | lost or enquiry | Proposed lostchecked 59% sure | Yes |
| clinicIs Dr. Mehta available on Monday morning? I want to book for my mother. | new | new | KeptFollow-up: Check Dr. Mehta's Monday morning availability for Alex Rao's mother, 2026-10-19 | Yes |
| clinicMonday 9:30 AM with Dr. Mehta, follow up for blood pressure. | new | booked | Moved to bookedchecked 96% sure | Yes |
| clinicRunning 20 minutes late for my 9:30, please hold my slot. | booked | booked | KeptFollow-up: Hold Alex Rao's 9:30 slot (running 20 min late), 2026-10-19 | Yes |
| clinicAre my blood test reports ready? Please call me back today. | seen | seen or follow up | KeptFollow-up: Call Alex Rao back about blood test reports, 2026-10-19 | Yes |
| clinicI am fully fine now, the doctor said no more visits needed. Thank you all. | follow up | closed | Proposed closedchecked 97% sure | Yes |
| clinicMere bete ko kal raat se bukhar hai, aaj shaam ka time milega kya? Wapas call kijiye. | new | new | KeptFollow-up: Call back Alex Rao about child's fever appointment today evening, 2026-10-19 | Yes |
| generalWe need catering for 60 people on 14 November. Please send a menu and a price. | new | new | KeptFollow-up: Send catering menu and price for 60 people on 14 November to Alex Rao, 2026-10-18 | Yes |
| generalMenu B please, for 60 people on the 14th. Send the invoice and we will pay this week. | contacted | qualified or won | Moved to qualifiedchecked 95% sureFollow-up: Send invoice for Menu B, 60 people, 14th, 2026-10-18 | Yes |
| generalCatering, Menu B, 60 people. | qualified | won | Proposed wonchecked 99% sure | Yes |
| generalCan we change the date to the 15th? Same everything else. | won | won | KeptFollow-up: Confirm date change for Alex Rao from the 14th to the 15th, 2026-10-18 | Yes |
| generalWe found someone cheaper, thanks for your time. | contacted | lost | Proposed lostchecked 94% sure | Yes |
| generalThe card was declined for the 900.00 USD deposit. | qualified | qualified | KeptFollow-up: Call Alex Rao about declined 900.00 USD deposit, 2026-10-18 | Yes |
How it was run
Each case starts a fresh workspace of its kind of office, a person at their stage with their history as notes, and the new entry sent through Tieloom's public API. The Autopilot read it as it would any entry: Claude Haiku 5.5 proposed the stage, the follow-up and a reply, and Jev, a separate decisions model, checked the stage on its own. A move happened by itself only when Jev agreed at 60% or more. Endings never did: they move by themselves only on a payment or a paid order, and these entries came through the API.
The conversations are constructed, with invented people, across law firms, property managers, clinics, real estate offices and general businesses, in English, Hindi and Hinglish. Where a careful office manager could go either way, either answer counted, and those cases say so in the file.
Set 2 (30 cases) was written and labelled after the prompt was tuned and before it ran. It ran twice with the same prompt: first before Jev's copy had names, phone numbers and email addresses taken out, then as Tieloom runs today, with them taken out. No case changed; this page shows the second run and the file holds both. Set 1 (40 cases) was used to tune the prompt, so its result (39 of 40 at the right stage) is not the test; both are in the file.
Let it read your next call.
Free while Tieloom is new. Connect a tool and watch the Autopilot work, with a reason and an Undo for everything.
Free while Tieloom is new. No card.