Agent Conversation Flow

How a user's chat message becomes an agent reply: provisioning, queueing, the turn pipeline, and what keeps it correct when a worker dies mid-call.

Concepts and data model live in Agent Sessions; the conversation surface itself is Chat.

---

End-to-end flow

sequenceDiagram
    participant U as User (reviewer)
    participant API as Backend API
    participant DB as PostgreSQL
    participant Q as Celery (agents queue)
    participant W as Agent worker
    participant LLM as Gemini
    participant WS as WebSocket

    Note over API,DB: chat created
    API->>DB: create chat
    API->>DB: create agent session + stage greeting
    Note right of API: synchronous — the greeting exists<br/>before anyone can subscribe

    U->>API: POST message
    API->>DB: persist message (commit)
    API-->>WS: fan out message
    API->>Q: enqueue turn (only if the poster is a REVIEWER)
    API-->>WS: "Assistant is writing…"

    Q->>W: process turn
    W->>DB: claim lease (short committed tx)
    W->>DB: collect inputs after cursor
    W->>LLM: run turn (no tx open)
    LLM-->>W: reply + tool calls applied to state
    W->>DB: stage reply + token-gated write (state, history, cursor)
    W->>DB: commit
    W-->>WS: publish reply
    Note over W: loop until no inputs remain<br/>or the session completes

Two deliberate choices in that picture:

• The greeting is staged at chat creation, not on the first turn, so a canned opening can never collide with the first model reply. No live push is needed — nobody is subscribed yet, and the client sees it on its first fetch.
• Only reviewers converse with the agent. Any role may post in the chat, but a turn is enqueued only for a REVIEWER author (strict — superusers included). Other posts are saved and simply get no bot reply.

---

Turn pipeline

One iteration of the drain loop:

collect inputs after cursor        transport: user messages since the last processed one
  └─ raise the typing signal       best-effort, refreshed while the call runs
load state + bind runtime          rehydrate JSON state; inject org id, memory, subject access
apply structured inputs            a pressed option is applied deterministically, before any gate
run the turn
  ├─ turn-start hook               e.g. snapshot the finish handshake
  ├─ scope gate                    topicality or safety — fail-closed
  ├─ burst folding                 rapid messages folded into one labeled prompt
  ├─ media note + demand-gated vision
  └─ LLM run                       tools mutate the state
stage the reply (+ options offer)  persisted inside the turn's write transaction
token-gated write                  state, message history, cursor, status, result
apply buffered effects             memory writes and option locks, same transaction
commit → publish                   reply and option updates, best-effort

Notable behaviors:

• A burst is one turn. Several messages sent in quick succession are folded into a single LLM call, each labeled, with an instruction to address every one — so no message looks dropped.
• Structured input beats the gate. A pressed option is applied under the lease before the scope gate, so a selection can never be discarded as "off-topic".
• Completion messages can be deterministic. A kind may replace the model's closing sentence with fixed text; the raw model reply still goes into the durable history — only what the user sees becomes fixed.

---

Turn leasing

A turn is token-leased, never a held database lock:

sequenceDiagram
    participant W as Worker
    participant DB as PostgreSQL

    W->>DB: claim lease (processing_at + token) — commit
    Note over W: LLM call runs with NO transaction open
    W->>DB: write result WHERE token matches
    alt token still ours
        DB-->>W: applied → commit reply
    else lease reclaimed meanwhile
        DB-->>W: not applied → roll back, drop the reply
    end

The uncommitted reply is never returned: whoever owns the lease redoes the turn. A lease goes stale after 60 seconds, and a heartbeat every few seconds during a long call both refreshes it and re-raises the typing indicator.

Concurrency is safe by construction: every user message enqueues a task, the lease serializes them, and a task that finds the lease held simply re-enqueues itself after a second — contention does not consume the retry budget.

---

Crash recovery

If a worker dies mid-turn, the message would otherwise sit unanswered until the broker's visibility timeout — potentially an hour. Instead a scheduler sweep runs every 60 seconds, finds sessions with pending inputs and no live lease, and re-kicks them. Recovery is bounded to roughly two minutes.

A session re-kicked five times without a single applied turn write in between is parked — skipped and error-logged — so a deterministic failure (corrupt state, revoked key, exhausted quota) cannot burn an LLM call every minute forever. Any applied write resets the counter.

---

Execution environment

Piece	Detail
Queue	A dedicated agents queue consumed by its own worker, so slow import/export work on the default queue never delays a chat reply
Worker loop	One persistent event loop per worker process, in its own thread — long-lived async clients (Redis, the model provider's HTTP client) bind to it, and task threads submit work onto it
Retries	Task-level automatic retry with backoff (four attempts). The drain is crash-idempotent: the lease, the token-gated write and the cursor mean a retried task simply redoes the uncommitted turn
Model	Provider registry with Gemini registered; the default model and the cheaper gate model are configuration, not code

---

Failure modes and what the user sees

Failure	Behavior
Scope classifier errors	Treated as off-topic (fail-closed) — the user gets the redirect, the main model never runs
Vision gate errors	Falls back to no image analysis; the turn still runs
Reply publish fails	Logged, non-fatal — the message is already committed and arrives on the next fetch
Lease reclaimed mid-call	Turn rolled back; the current owner redoes it
Worker crash	Recovered by the sweep within ~2 minutes
Repeated deterministic failure	Session parked after five recovery attempts

---

Related documentation

• Agent Sessions — kinds, state, memory, guardrails
• Chat — messages, visibility, real-time events
• Agent Job Queue — the other agent flow: work handed to an external bot