Support Line AI Assistant

Tech stack

For engineers: the path from gateway to outcome, signed integrations, a guard on model answers, waves and switches, memory, and decisions with costs.

~22 min read · 4406 words

TL;DR

Support Line AI Assistant is a Python 3.12 and FastAPI backend with PostgreSQL 16 and Redis, the GPT-5.2 model answering in a strict schema, and a guard behind it. WhatsApp messages arrive from the messenger gateway as signed requests, and the website chat runs over WebSocket. The bot reads the customer's orders through the store platform's API, passes questions to the driver and changes an order after the customer says "yes". Each class of answers opens behind its own flag: in September 2026, 15 waves opened over 10 days. Every rule in the instructions is explained by one of 134 cards, and a test fails if the instructions change without their card. The suite has 3,928 tests with no network access and runs in 21 seconds. There have been 207 automated deploys since February, and restarting the container takes about 2.4 seconds.

Diagram

A message from WhatsApp and a message from the website chat go through the same pipeline: recognise the customer, gather live data, ask the model, check the answer and pick an outcome.

Component	What it does	Technology
Messenger gateway	receives WhatsApp messages, runs the job queue and the operators' feed in Telegram, carries out the bot's decisions	Python, FastAPI, aiogram, PostgreSQL
Bot backend	the decision on every message on the line, the website chat, customer memory	Python 3.12, FastAPI, asyncpg, Redis; 29 modules, 33k lines
Model	answers, topic, photo descriptions, memory summaries, web lookup on products	OpenAI GPT-5.2
Knowledge base	25 topics, 3,006 sample questions from the line	text-embedding-3-small at 512 dimensions, neighbour search in NumPy
Store platform	orders, statuses, order changes, questions for the driver	its own API; see Last-mile Delivery Platform
Website chat	loader, an iframe on another domain, WebSocket	plain JavaScript, no framework
PostgreSQL 16	decision log, memory, chat sessions	15 tables, native on the server
Redis	switches, rate limits, replay protection, idempotency	native on the server

The bot backend runs in one Docker container on the host network; PostgreSQL and Redis run natively on the same server, and the service faces the outside through nginx. Voice is transcribed by Whisper: on the line the gateway does it, in the website chat the backend does.

Integrations

The messenger gateway and the bot

The gateway and the bot talk through signed requests: for every job, for the second step of a web lookup and for every line a person writes. The signature covers the path as well, so a signed request for one route cannot be sent to another, and a captured request cannot be replayed. A repeat of the same job does not send the answer twice: the bot recognises it by message ID. Any failure inside the bot comes back to the gateway as a "hold" outcome with a reason rather than an error. A 50-second time budget; the gateway gives up at 60. Inside it: the model 20 seconds, a photo 30, the platform API 5, an order change 15, a web lookup 25.

The gateway retries network errors, 408, 429 and 5xx five times with pauses of 5, 30, 120 and 600 seconds, then marks the job failed and alerts the owner. Since 4 October the gateway decides nothing itself: the bot determines what to answer, when and to whom, and the gateway passes facts on and carries out send, draft, hold and the hand-over to the driver.

The bot and the store platform

The bot calls the platform's API with an integration key. Reads: the customer by phone number, their orders and profile, an order by number, the catalogue. Writes are a closed list of five order-change routes: add, edit items, cancel, reschedule, change the address. Reads wait 5 seconds, changes 15. If the API does not respond, the decision gets a reason such as orders_unavailable and a "hold for a person" outcome. Reasons like that are never replayed from the cache; they are recomputed on a retry.

On the platform's side an order change goes through the same mechanism as a dispatcher's buttons: one transaction with a lock on the order, an idempotency key, a repeat with the same key returns the stored answer, and a different body gets a 409. A question for the driver the bot only proposes: a text of up to 1,000 characters, the order and the customer. The gateway makes the call to the platform with its own key and its own idempotency, and fetches the driver's replies with a separate request.

The integration is described by a contract with 90 file:line references into the platform's code, and a platform test checks that none of them have gone stale. My contract with the gateway is a separate document, with each rule dated and measured.

The answer pipeline

The main function is _answer_claimed in office_ai.py, a 10.7k-line module that holds all the rules. In order:

Admission. Text or a photo, the master switch, an empty message, a photo without bytes.
A claim in Redis and a check for repeats.
A person has already replied after this message: outcome "stay silent".
Rate limits per customer number and for the whole line.
Fast paths before the model: an order number the customer gave earlier; a "yes" or "no" to a proposed order change; "thanks" after a question nobody has answered yet.
The pre-gate pre_gate: 8 reasons why a message cannot be handled as usual (a complaint, a legal question, a business proposal, a request for a person and others), plus "ok" and emoji, a bare link, and a question about cancelling. A complaint, a legal question or a business proposal is held for people. Since 6 October a request for a person gets a fixed text with the office phone number. Questions about jobs, timing and cancelling continue to the model.
Topic. The message is embedded, its 5 nearest neighbours in the knowledge base are found, and a majority of three decides. The model names a topic of its own as well, and the rules look at both: for example, which order a question is about is decided by whichever of the two topics concerns an order at all.
The customer's orders through the platform API.
Fixed paths: how to pay, "nothing is on its way right now", how to message the driver yourself.
Live store data: whether it is open, whether it delivers to the address, what is on the shelf, the product behind a link the customer sent.
A verbatim answer from the knowledge base, if similarity is at least 0.80 and the question involves no live data.
Building the prompt. The system part is constant: style, store facts, rules, topics, 31 line rules and the blocks for step 3, order changes and lookup, 51,708 characters in all. Everything live goes into the user message: order blocks, store data, the knowledge-base answer, the customer card, the lookup result. That keeps the system prefix identical for all requests with the same set of flags, so OpenAI's prefix cache works. On average a request uses 17.8k prompt tokens and 162 answer tokens.
The model: GPT-5.2, temperature 0.1, up to 400 tokens, a strict JSON schema. Fields: topic (one of 24), answer, clarifying question, whether a person is needed, confidence, reason. Step 3 adds a hand-over to the driver, lookup adds a search request, order changes add 9 change fields.
If the model asks for a person, separate paths take over: pass it to the driver, ask for the order number, the order is closed, the complaint is too late, or a second call to the model saying "there is no person on this line".
The guard, guard: 18 kinds of error, 3 more for what cannot be promised without a person (an interview, "we'll get back to you", "we'll pass it on"), and 1 for repeating the same answer. Among them are time_promise, invented_time, tracking_link, external_url, unknown_phone, claims_person, unasked_cancel and too_long.
One rewrite. If there is time left, the model gets its answer back with the reason text from GUARD_REPAIRS. The rewritten answer goes through the whole guard again. If it fails, a fixed line goes out (the payment rule, "I'm the automated assistant", the order status) or the outcome is "hold for a person".
The outcome in _finish. send is set only if the wave for that class is open and nothing stops it: a person wrote in the last 15 minutes (60 for classes beyond status, payment and rules), a burst of messages, the same template twice in a row, an unclosed hand-over to a person, or the master send switch.

Over the three days before the snapshot, processing inside the bot took 4.2 seconds at the median and 7.5 at the 95th percentile. The customer waits longer, about 29 seconds at the median, because the gateway spends 20 of them waiting in case the customer adds something.

Waves and switches

Every answer the bot sends on its own carries a wave label. The set of open waves is built at start-up from environment flags, so changing a flag means recreating the container, about 2.4 seconds. Waves w0–w4 are always open; each of the others has its own flag.

Wave	Opened	What it opened
`w1`, `w2a`, `w2b`	21.09	greetings; order status and "where is the driver", with and without a link
`w2c`	21.09	topic not recognised, but the answer holds the link to this order's page
`w0`	24.09	a reply to a clear "thanks"
`w3`	24.09	how to pay, as fixed texts
`w4`	24.09	"nothing is on its way right now"
`w5`	28.09	the model's answer itself decides whether to send
`w6`	28.09	the model names the topic; up to three clarifying questions
`w7`	28.09	the customer messages the driver directly, the bot explains how
`w8`	28.09	number not recognised: ask for the order number
`w9`	28.09	a question for the driver or the dispatchers
`w10`, `w11`	29.09	everything except the "always a person" list; product lookup
`w12`	30.09	order changes after a "yes" (rescheduling and address on 1.10)

Every wave was preceded by a measurement on the log. Before w5, over 30 days the bot sent 4% of its decisions itself, 41% went out as drafts and 55% were held. Before w1/w2, 1,450 messages arrived in 14 days with zero automatic answers, and 504 were candidates for the wave. In its first three shifts the wave sent 10, 24 and 11 answers on its own. Wave w2c was expected to give 18 answers in 14 days and gave 0, 6 and 2 in three shifts.

Since 21 September the bot has sent 800 answers on its own. The most came from w5 (248), w2b (96), w3 (77) and w10 (69).

Switches come at three levels:

the master admission switch: off, and the bot does not answer at all;
the master send switch: off, and everything goes out as drafts, from the next message;
a class flag in the environment: turns off one wave once the container is recreated.

The order for opening a class: measure on the log of past messages (office_log_replay.py replays the log with a setting switched) → change → tests and mutations → a check against the live model → measure after → flag → rebuild the rules document.

Memory

Four tables: customer_memory is the customer card (summary, facts in jsonb, language, counters), conversations holds conversations (channel, start and end, summary, outcome, order numbers, topics), conversation_messages holds lines with the sender "customer", "person" or "bot" and a snapshot of the order state, and customer_aliases joins keys together. The customer key is their ID in the platform, or a hash of the number until there is one.

A conversation ends after 45 minutes of silence. Every 300 seconds a background task closes quiet conversations (up to 20 per pass), asks the model for a summary and updates the card. A summary is at most 400 characters and is cut at a sentence boundary. A card holds up to 12 facts, and the last three conversations go into the prompt.

Filters run before anything is written. A fact that mentions payment, a number of 7 digits or more, or an address is dropped. Payment details and IDs, email addresses and phone numbers are removed from summaries sentence by sentence. In the decision log, order-page links and payment details are masked on write. Closed conversations are kept for 365 days, cards for unknown numbers for 90, the decision log for 90. Probe messages are never written to memory.

On 19 October memory held 642 cards (417 linked to a platform customer), 2,967 conversations and 9,491 lines, and the database took 47 MB. Memory was switched on on 27 September and can be turned off with one environment line, with the data kept. The website chat writes to the same card only when sign-in has proven whose key it is.

The project has no vector database. The knowledge base is an .npz file of 3,006 vectors of 512 numbers each, built on 12 August from the line's history. It ships in the image and is loaded into the process's memory.

The website chat

The loader creates a container with its own styles on the store's page, and an iframe from the bot's domain. Inside the iframe is the chat app itself; it holds a WebSocket to the backend and reconnects up to five times. The visitor key lives in the browser. The site's styles and the chat's styles cannot see each other.

Since 15 October the chat answers with what the line uses: live store data, the knowledge-base answer, and the customer's orders if sign-in has proven they are the owner. The answer goes through the same guard plus two checks of the chat's own: it must not promise that a person will join, and it must not ask for contact details. Then one rewrite and a fixed line. The whole switch rolls back with one flag, WIDGET_BOT_KNOWLEDGE.

Instruments

office_replay.py: 15% of the knowledge base is held out and kept out of the vectors. The tool runs the real _answer_claimed over it with the transport stubbed, and a judge (the same GPT-5.2, temperature 0, a strict schema) gives one of six verdicts: correct, vague, wrong, harmful, rightly held, needlessly held. The judge sees the question, the real reply a person gave, the live data and the bot's answer. The weak point: the judge is the same model that answers.
office_log_replay.py replays the live log with a setting switched, without reaching customers. That is how the "before" is measured ahead of a new wave.
autosend_report.py and autosend_delivery.py: what the bot decided and what actually went out. The second joins the bot's log with the gateway's by request ID.
voice_report.py checks tone against the line's standard. Its check function is shared with the tests.
mute_scan.py checks the checks. It parses the tests into a syntax tree, finds every assert and pytest.fail, runs each file in its own process under sys.settrace, and prints the places execution never reached. It then compares its count with a normal run and labels a mismatch as a broken run, so it is never mistaken for silence. The --self-test flag plants a place that is known to be silent. On 2 September it found 4 assertions the tests never reached, and on 6 September a line in the tool itself that was printed without being checked.
The rules document. office_runbook.py builds 134 cards from the instructions: 66 situations, 29 facts, 16 guard rules, 12 style rules, 11 texts. Four tests hold the link: every rule in the instructions is explained exactly once, the instructions were not changed behind the document's back (hashes of 134 items against a reference file), and every pre-gate reason and every guard verdict has a card.
Reference files: 7 of them, from the model's instructions to the list of memory fact keys.
No network in tests: conftest.py replaces socket.connect and create_connection, so anything but a local address fails with an error.
Probe messages: their ID starts with probe. They are written to the log but not to memory, and every report drops them with one condition.

Deploys

A push to the main branch starts GitHub Actions.
Tests: compilation, the full suite, an import of the app, a scan of the tree for secrets.
An image built with the commit tag and latest in GitHub's registry. Inside the built image a check confirms that the knowledge-base files are present and load.
The deploy: SSH to the server, docker pull, docker compose up -d.
A check: an external request to /health, up to 10 attempts 6 seconds apart, or the run goes red.

The whole cycle takes about three minutes, and the container restart itself takes 2.4 seconds until the app is ready. Since 8 February there have been 207 successful deploys and 5 failed pushes. The pipeline has no rollback: old code comes back through git revert and a new push. The database schema needs no migration tool: at start-up the app runs idempotent CREATE TABLE IF NOT EXISTS (15 tables) and 28 ADD COLUMN IF NOT EXISTS. Since 21 September every push happens only with my approval.

Decisions and why

The gateway is only a channel

The bot decides what to answer, when, to whom and whether to answer at all; the gateway passes facts on and carries the decisions out. Until 4 October some send stops lived in the gateway: a burst of messages, the same template twice in a row, an unclosed hand-over to a person. The decision "answer or not" was being made in two places, and on 4 October I moved all of it into the bot. The cost: the bot has to know more about the channel, such as which people wrote and what really went out, and the gateway has to pass those facts in every job.

A bot error is a 200 and "hold"

The alternative was to return 5xx and rely on the gateway's retries. But a retry after a half-finished step could send a customer the same answer twice. So any failure inside the bot comes back as a 200 with a "hold" outcome and a reason, and the gateway retries only on network errors and explicit transient codes. The cost: failures do not show up in response codes, only in the reasons in the log, and the shift report counts them separately.

Each class of answers is a wave of its own

I could have switched automatic answers on all at once. Instead I opened classes one at a time, each with its own flag and a measurement before and after: 15 waves in 10 days. The "thanks" wave was switched off 14 minutes after it opened on 24 September, and the damage stopped at two messages. The cost: slower, and the code now carries a wave label on every answer and a list that a test checks against the reasons.

A strict schema and a guard after the model

The model answers in JSON with a strict schema, and every answer goes through a guard with 22 kinds of error. When it finds one, there is exactly one rewrite with the reason given, then a fixed line. The alternatives were to trust the model or to keep asking until it passed. One rewrite caps the answer time, and a fixed line cannot turn out to be untrue. The cost: the guard's regular expressions have to be maintained alongside the instructions, and each verdict has its card and its test.

Live values as they are, links only from order data

The bot takes the status, the arrival time and the order-page link from the platform and calculates nothing itself. The guard drops any link that is not in the order data, and any promised time that is not there either. The cost: if the platform does not provide something, the bot will not say it, and the gap shows only in measurements, such as the share of "when will it arrive" questions that have an answer.

The knowledge base is a file in the image

3,006 vectors of 512 numbers do not need a separate vector database: neighbour search runs in NumPy in the process's memory, and the file is built offline and ships in the image together with the code that reads it. The cost: adding questions means rebuilding the file and deploying the image. Nothing is learned on the fly.

Memory is a card and summaries; the full history is not needed

The prompt gets the customer's summary, up to 12 facts and the last three conversations. The full history would give more context, but it would quickly eat the prompt budget and it would still contain payment details and phone numbers. The cost: one model call per closed conversation, and filters before every write. Before memory was switched on, a review found 66 of 567 cards cut off at exactly 400 characters, 56 of them mid-word. Since then a summary is cut at a sentence boundary.

People's example answers are collected and switched off

The block of examples showing how the line used to answer is ready: 42 lines across 10 topics. On the 12 test questions where it appeared, the median answer length fell from 95 to 75 characters and the single harmful answer disappeared. I meant to switch it on to raise the model's confidence, but a measurement on 28 September showed that among the 180 answers it would have touched, none had been given with less than high confidence. The block would not have added a single answer without a person, so I left it off. The cost: the bot does not learn from people automatically; every improvement comes as a change to a rule.

The rules document is built from the instructions

Every item in the instructions is explained by a card, a script builds the document, and a test fails if the instructions change without a card. The alternative is documentation by hand, which falls behind the code within a week. The cost: every change to the instructions rebuilds the document and the reference file of hashes.

A database schema without a migration tool

Tables and columns are only ever added, and the app runs idempotent DDL at start-up. The alternative is Alembic, with versions and down-migrations. For 15 tables and changes of the "add a column" kind it would cost more than it gives. The cost of the decision: a column cannot be dropped or renamed this way, only by hand as a separate step.

A person in the chat: 15 and 60 minutes

The bot stays silent for 15 minutes after a person's line and for the next 45 answers only in narrow classes: status, payment, rules, a confirmed order change. The alternative was silence until the end of the conversation, but then a customer whom a person had answered once could wait until evening for anything more. The cost: the boundaries were tuned on shifts and not derived from a model of how people behave. In the shift of 18 October the 15-minute wait for a held answer happened four times and helped none of them. That is the next thing to measure.

Checking the checks outside CI

mute_scan runs every test file in its own process under tracing, which is slower than a normal run. So the tool is not in CI; it runs once per batch of changes, before a push. The cost: between runs a silent assertion can survive a few commits.

How it is checked

3,928 tests in 59 files, a full run in 21 seconds, no network allowed in tests.
2–3 mutations on the main condition of each change: I break the change and confirm that its own test is the one that fails.
mute_scan before every push.
Every change to the instructions is checked against the live model several times in a row. On 30 September the first wording of a rule worked in 1 run out of 6.
An AI agent reviews every shift on the line from the log, and I read the review and decide what to fix.
A new class of answers is switched on only after a measurement on the log.

What the bot does not do

There is no vector database; the knowledge base is a file in the image.
The store has no dashboard or settings: the developer changes flags and rules.
The model is not fine-tuned, and the bot does not learn from people's replies automatically.
It never writes to a customer first and sends no reminders.
It does not know whether a message was delivered or read: the gateway never gets that.
The deploy pipeline has no rollback and there is no migration tool, by design.

Links

The source is laid out by directory: backend/ holds the bot backend and the answer pipeline, widget/ the website chat, scripts/ the instruments and the knowledge-base build, tests/ the test suite and reference files, contracts/ the contract with the gateway, and docs/runbook/ the rules document.