Ten minutes a week

A customer assistant is not a thing you set up once. It is a thing you tend, a little, regularly. The good news is that "a little" turned out to mean about ten minutes a week, once the admin grew three pages that do the looking for you. This post walks through that loop as it exists today.

1. Read what it could not answer

The most valuable page in the admin is not the analytics. It is Unanswered: every turn in the last thirty days where the assistant admitted it did not know, next to the visitor question that caused it.

The detection is deliberately simple. There is no model judging the answers; a small set of patterns matches the phrasings the bot uses when it punts ("I don't have that information", "please ask our staff"), tested against real production transcripts. Polite scope refusals are excluded on purpose: a visitor asking for a poem and being told no is the bot working, not a gap.

Each row has an Add fact shortcut straight to that business's knowledge base. The loop is: read the question, write a short section that answers it, save. The next visitor with that question gets a real answer. No restart, no deploy. Every row you clear is a question answered properly forever after, which is why this is the highest-leverage ten minutes in the whole product.

2. Check what it is repeating

Fresh questions that closely paraphrase a recent one are answered from a cache, instantly and for free. That is lovely until the cached answer is wrong. It happened for real: a cached "what's coming up?" served February events as upcoming for most of a day, after the events bug itself had been fixed. The cache had outlived the fix.

The cache self-heals (rows expire after 24 hours and the whole thing is wiped whenever knowledge or settings change), but the owner needed to see what was being served and evict one bad row without ceremony.

The hit counts turned out to be a second feature for free. Sorted by hits, the cache is a ranked list of what visitors ask most in their own words. If the top row's answer is merely fine, it is worth making it good.

3. Get pinged when it matters

Reading pages is a habit; a nudge is better. Each tenant can have a Slack webhook, and with one configured every new conversation lands in Slack as it starts, with the visitor's first message. The owner reads the first line of a chat during opening hours and can walk over to the counter if it is someone in the shop. A second ping fires if the daily message cap is reached, which is the difference between "busy day" and "something is hammering the chat".

The field teaches you how to get a webhook, right under the input, because the first owner to see it had no idea what one was. And it only accepts URLs on Slack's own host, which is not fussiness: a tenant-supplied URL that the server will POST to is the classic way to turn a helpful feature into a way of reaching internal services. Pinning the host is the whole security model of that field.

4. Decide how long to keep things

Conversations are kept to improve the service, and the privacy notice says so. To make that sentence true rather than aspirational, each tenant has a delete conversations after N days setting. There is no cron job; a lazy sweep runs on chat traffic and removes what is older than the limit, transcripts included. Blank keeps everything, and ninety days is a sensible number for a cafe.

The habit

Monday, coffee in hand: Unanswered first, fix two or three rows. Glance at the cache's top hits. Skim the Slack channel for anything odd over the weekend. Done. Ten minutes, and the assistant is measurably better than it was last week, because every one of those minutes was spent on a question a real person actually asked.