AI backends
Andon calls a model for attention triage, the assistant, recipe detection, Autopilot’s placement, Start work’s guesses, the finish classifier (Auto-Done) and voice. Every one of those calls goes through one client, built on the Vercel AI SDK, which can send it four ways. Which one is a setting, not a deploy.
The four backends
| Backend | Where the call runs | The key it needs |
|---|---|---|
anthropic |
Claude, through the SDK’s Anthropic provider. The default whenever there is an Anthropic key. Every feature but voice. | The org’s own Anthropic key (Settings → Connections → Advanced → Claude (Anthropic) key), else the server’s ANTHROPIC_API_KEY. |
gateway |
The Vercel AI Gateway. Models are named creator/model, such as google/gemini-3.6-flash. |
The org’s own AI Gateway key (Settings → Connections → Advanced), else the server’s AI_GATEWAY_API_KEY. |
direct |
Gemini itself, through the SDK’s Google provider. | The org’s own Gemini key, else the server’s GEMINI_API_KEY (and GEMINI_REFERER for a referrer-restricted key). |
machine |
One of the org’s machines, which runs the same call with its own key. | On the machine: andon policy llm-serve on, and its own key (see Serving model calls). The app needs no key. |
The prompts, the answers’ shape and the retries are the same on every backend. Features that ask for a JSON answer get one in the same shape, and the assistant’s tool calls work the same way.
Which backend runs a call
The first of these that is set wins:
- A backend the call itself asks for (a few internal calls do).
- The org’s choice: Settings → Connections → Advanced → AI backend. Admins change it; “Deployment default” leaves it to the server.
- The server’s
LLM_PROVIDER(anthropic,gateway,directormachine). - The keys the server and the org hold:
anthropicwith an Anthropic key, elsedirectwith a Gemini key, elsegatewaywith a gateway key, elsemachine.
When the chosen backend can’t serve the call, the next one in line does:
- the chosen backend first;
- then the other app-side backends that hold a key, in this order: Claude, Gemini directly, the gateway;
- a machine last.
A machine choice tries a machine first. When no machine is online and serving, or the machine doesn’t answer within
LLM_MACHINE_TIMEOUT_MS (45 seconds by default), the app’s own keys answer instead. The server logs which backend it
picked and why, for example llm recipe: machine wanted (org setting); no machine online → direct.
A machine’s wait comes on top of each feature’s own time limit, so the app’s key still has that limit to answer when the machine doesn’t. Start work’s guesses while you type are the exception: they keep their 10-second limit, so a slow machine there means no guess rather than one from the app’s key.
An error from the provider itself (a quota, a refused key) is not a reason to switch: it is reported as before.
Attention triage and recipe detection start from the database, which can’t read LLM_PROVIDER. Triage starts when
the deployment has a key (pnpm bootstrap turns it on), when the org holds its own Anthropic, Gemini or AI Gateway key, or when
the org chose On a machine and one of its machines serves model calls. A deployment with no key and only
LLM_PROVIDER=machine gets no triage; choose the machine backend in the org’s settings instead. Recipe detection
starts with a key, the deployment’s or the org’s; without one a machine’s own Claude picks the recipe, as before.
Voice
Voice is a live audio session between the browser and the model, so it always needs a Gemini or AI Gateway key the app holds. It never runs on Claude or on a machine.
- With
direct, voice is the Gemini Live API, withGEMINI_VOICE_MODEL(defaultgemini-3.8-live). - With
gateway, voice goes through the gateway’s realtime modality, with the org’s voice model (next to the AI backend setting), elseLLM_VOICE_MODEL, elsegoogle/gemini-3.8-live. The gateway names the model differently from Gemini, which is why it has its own setting.AI_GATEWAY_VOICE_MODELis the older name and still works. - With
anthropicormachine, voice quietly uses the voice backend that holds a key: Gemini directly first, then the gateway. With an Anthropic key and neither of those, there is no voice.
VOICE_PROVIDER (gemini or vercel) still overrides all of this for voice alone.
Voice providers has the details.
Models
Each feature’s model is set in one place, with a Claude name for anthropic, a Gemini name for direct and a gateway
name for gateway:
| Feature | Claude (anthropic) |
Gemini (direct) |
Gateway |
|---|---|---|---|
| Attention triage, drive answers | ANTHROPIC_TRIAGE_MODEL |
GEMINI_TRIAGE_MODEL |
AI_GATEWAY_TRIAGE_MODEL |
| The assistant | ANTHROPIC_ASSISTANT_MODEL (else the triage model) |
GEMINI_ASSISTANT_MODEL (else the triage model) |
AI_GATEWAY_ASSISTANT_MODEL |
| Auto-Done’s finish classifier | ANTHROPIC_FINISH_MODEL |
GEMINI_FINISH_MODEL |
AI_GATEWAY_FINISH_MODEL |
| Recipe detection | ANTHROPIC_RECIPE_MODEL |
GEMINI_RECIPE_MODEL |
AI_GATEWAY_RECIPE_MODEL |
| Autopilot’s placement, Start work’s breakdown | ANTHROPIC_PLAN_MODEL |
GEMINI_PLAN_MODEL |
AI_GATEWAY_PLAN_MODEL |
| Start work’s team guess and title | ANTHROPIC_ROUTE_MODEL |
GEMINI_ROUTE_MODEL |
AI_GATEWAY_ROUTE_MODEL |
Every Claude model defaults to ANTHROPIC_MODEL, else claude-opus-5-5. Start work’s guesses run while you type and
must answer within 10 seconds, so a faster model such as claude-haiku-5-5 for ANTHROPIC_ROUTE_MODEL is worth a
try. Every Gemini model defaults to gemini-3.6-flash. Every gateway model defaults to google/ and the feature’s
Gemini model. A machine runs the model the app names, under the name for its own backend.
On Claude, settings that are Gemini’s own (its thinking budget, sampling temperature) are left out: current Claude models take none. A feature that asks for a JSON answer gets it through Claude’s structured outputs.
What travels to a machine
The app hands the call to a machine over the machine’s command channel, as an llm_generate command: the prompt, the
model’s names and the options. The machine runs it on its own key and sends back the answer. Keys never travel either
way: the app’s keys stay on the server, and the machine’s stay on the machine.
A machine only takes these commands from Andon itself, never from a person’s command, and not when its policy requires signed commands.
What isn’t backend-agnostic
- Voice can’t run on a machine (above).
- A machine serves these calls with an Anthropic, Gemini or AI Gateway key, not with its Claude login. The older paths that ask a machine’s own Claude (recipe detection when nobody holds a key, answers while Andon drives) are unchanged.
