# Testing

Andon has nine Vitest projects and a Playwright suite. Most of them need nothing running; the rest need only this
worktree's own Postgres, which they start themselves. Run the cheap ones constantly and the heavy ones on purpose.

## Which suite needs which stack

| Suite | What it covers | Command | Needs |
|---|---|---|---|
| `@andon/shared` | the pure core: verdicts, templates, schemas, the `.andon/` format | `pnpm exec vitest run --project @andon/shared` | nothing |
| `board` | the board's logic: reducers, coalescing, hooks, a few components | `pnpm exec vitest run --project board` | `@andon/shared` and `@andon/client` built |
| `agent` | the agent's decisions, against a fake `exec` | `pnpm exec vitest run --project agent` | the same |
| `@andon/client` | the supabase-js shapes and the SSE client | `pnpm exec vitest run --project @andon/client` | nothing |
| `scripts` | the repository's own scripts: the docs lint | `pnpm exec vitest run --project scripts` | nothing |
| `docs` | the docs site's plain modules: reading `docs/`, link rewriting, the build's link check, code Copy text | `pnpm exec vitest run --project docs` | nothing. The site's build (`pnpm --filter @andon/docs build`) also fails on a broken on-site link |
| `desktop` | the desktop app's local pages | `pnpm exec vitest run --project desktop` | nothing; `cargo test` in `apps/desktop` covers the Rust side |
| `server` | the server: data API, auth, live updates, storage, functions, RLS, RPCs, triggers and the tenancy registry | `pnpm exec vitest run --project server` | this worktree's Postgres. Its global setup starts `pnpm db:local` and uses an `andon_test` database |
| `agent-int` | the agent end to end on the Andon server: a real tmux, git and ffmpeg, fake `orca` and `claude` | `pnpm exec vitest run --project agent-int` | this worktree's Postgres (its global setup starts `pnpm db:local`, makes a fresh `andon_agent_test` database and starts the server on a free port), tmux, ffmpeg. Build the agent first (`pnpm --filter @andon/agent build`) |
| e2e (Playwright) | the board in a browser at 1440, 1180 and 390 wide, and the installed PWA | `pnpm --filter @andon/board test:e2e` | nothing to start: it provisions its own database |

The two packages have no Vitest config of their own, so their projects take the package name: `--project shared`
matches nothing. On its own Vitest stops with "No projects were found", but beside another project
(`--project agent --project shared`) it runs the agent's tests and quietly skips all of the shared ones.

The unit projects fail whole files with `Failed to resolve entry for package "@andon/shared"` in a fresh clone. That
means the packages aren't built yet, not that a test failed:

```sh
pnpm --filter @andon/shared --filter @andon/client build
```

### The e2e suite

Playwright runs against the server. `scripts/e2e-db.ts` makes a fresh `andon_e2e` database in this
worktree's Postgres, and the config starts the server, the Vite board and a production preview (for the service worker)
on their own ports, then stops them. It is safe to run from any worktree. Two worktrees can run it at once with
different `E2E_PORT`, `E2E_PREVIEW_PORT` and `E2E_API_PORT`.

The projects are `desktop` (1440 × 900), `ipad` (1180 × 820), `iphone` (390 × 844) and `pwa`. A spec joins one with a
tag in its title (`@ipad`, `@iphone`, `@pwa`); untagged specs run on `desktop`. Browsers install once with
`pnpm --filter @andon/board exec playwright install chromium webkit`.

```sh
pnpm --filter @andon/board test:e2e -- --project=iphone
pnpm --filter @andon/board test:e2e -- e2e/board.spec.ts --project=desktop
```

## The everyday loop

Run the cheapest proof first, and stop when the change is covered:

```sh
pnpm exec vitest run --project board                         # about 30 s
pnpm exec vitest run --project agent --project @andon/shared  # about 30 s
pnpm exec vitest run --project agent src/pipeline.test.ts    # one file
pnpm exec vitest run --project board -t "beat the send"      # tests whose name matches
pnpm exec vitest --project board                             # watch
```

Before you push, run the gate:

```sh
pnpm typecheck
pnpm lint
pnpm test
```

`pnpm test` builds `shared`, `client` and `agent`, runs every Vitest project, then the board's bundle budget, then
Playwright. It needs nothing started first, and it takes minutes.

Also run the checks that match what you changed: `pnpm check:personal` always, `pnpm check:docs` for anything under
`docs/`, `pnpm db:check` for a schema change ([Schema changes](https://andon-docs.pages.dev/contributing/schema-changes.md)) and `pnpm --filter @andon/board budget`
for anything that adds to the board's first load ([Design system](https://andon-docs.pages.dev/contributing/design-system.md#the-bundle-budget)).

## What kind of test for which change

| Change | Test | Where |
|---|---|---|
| a pure rule: a verdict, an order, a template, a label | a unit test beside the module, table-driven, with the edge cases | `packages/shared/src/*.test.ts` |
| board logic: a reducer, columns, coalescing | a pure unit test, no rendering | `apps/board/src/lib/*.test.ts` |
| a hook that loads, patches or polls | a unit test of the gate or coalescer; Playwright when people see it | `apps/board/src/lib/`, `apps/board/e2e/` |
| a table, policy, RPC or view | a `server` test as the real caller, a tenancy registry row, and the other-org, non-member and `anon` cases | `apps/server/test/rls/` |
| a server endpoint, the query compiler, an auth flow | a `server` test | `apps/server/test/` |
| the agent: claiming, launching, hooks, commands | a unit test for the decision, `agent-int` for the loop | `apps/agent/src/*.test.ts`, `apps/agent/test/*.int.test.ts` |
| a parser for another tool's output | a unit test against a captured fixture | `fixtures/` and the parser's test |
| a flow someone clicks through | a Playwright spec with an axe check | `apps/board/e2e/*.spec.ts` |
| the desktop shell | `cargo test` and the TypeScript tests | `apps/desktop` |

Two habits that catch common gaps:

- **Test the wiring production uses.** Build the object the way the app does (`new Agent(config)`, `start()`, the
  exported factory), not a version the test assembles for itself. A test that injects the dependency production forgot
  passes while the feature is dead.
- **Assert as the caller the app is.** A policy test run as the table owner proves nothing; run it as a member, a
  member of another org, a machine and `anon`.

A new table, view, RPC, bucket, Realtime topic or server function isn't done until it is in the tenancy registry
(`apps/server/test/rls/tenancy-registry.ts`). The coverage test then fails on a relation without row-level security, a
policy, an org column or a registry entry, and on a function or topic the registry doesn't know. The isolation tests
seed a row of every registered table in one org, then check that nothing reaches another: no row, RPC answer, object,
topic (`tenancy-isolation.test.ts`) or function's answer (`test/functions/tenancy-fns.test.ts`).

An instance holds one org, so `create_org` refuses once the database has any. The suites make their many orgs with
`makeOrg` / `insertOrg` (both `helpers.ts`) or the e2e `makeOrg` (`backend().org`), which call
`andon_private.insert_org` as the database owner: the same rows, without the one-org check. A test of `create_org`'s
success path needs a database of its own, as `apps/server/test/api/setup.test.ts` has.

## Reading a result

- The exit status is the verdict. When you pipe a run through `tail` or `grep`, keep it: `cmd > log 2>&1; echo
  exit=$?`, or `set -o pipefail`.
- A run filtered by `-t` or a file path proves those tests, not the suite.
- A suite you skipped because its stack wasn't running is unverified. Say so in the pull request.
- If a test fails before you changed anything, say it was already failing. Don't weaken its assertion.

## Known quirks

- Many of the agent's tests build real git repositories in a temporary folder (`pipeline`, `andon-dir`,
  `git-workspaces`, `launch-files`, the `memory-*` files). On a busy machine they hit Vitest's 5 s timeout. Rerun the
  failing files alone or with `--testTimeout=60000`: a "Test timed out" that passes then isn't a regression.
- `apps/agent/src/executors/codex.test.ts` checks the exact environment a Codex launch gets, so `ANDON_*` variables in
  your own shell (set when you work inside an Andon run) make it fail. Unset them for the run.
- The `server` and `agent-int` suites each use one database and run their files one at a time. A test that leaves rows behind
  breaks the next file, not itself: look rows up by id, never by title. CI's `unit (N/4)` runs a quarter of the files,
  so a failure that only shows there reproduces with the same slice: `pnpm exec vitest run --project '!agent-int' --shard=N/4`.
- If the server suite can't start Postgres because a stale one holds the port, run `pnpm db:local stop` and try again.
- The `pwa` project serves a production build, so a service worker change needs it: the dev server doesn't register
  the worker.

## CI

`.github/workflows/ci.yml` runs on every pull request into `main` and by hand (Actions → CI → Run workflow, or
`gh workflow run CI --ref <branch>`). Pushes to `main` don't run it. Every ready pull request gets these checks:

| Check | What it runs | Needs |
|---|---|---|
| `lint` | `pnpm build`, `pnpm lint`, `check:personal`, `check:team-names`, `check:docs` | nothing |
| `typecheck` | `pnpm typecheck` | nothing |
| `unit` | every Vitest project but `agent-int` (the server suite brings its own Postgres), split by `--shard` across four runners, `unit (1/4)` to `unit (4/4)`; `unit` passes when all four do | tmux, ffmpeg |
| `secrets` | gitleaks over the whole history, with `.gitleaks.toml` and `.gitleaksignore` | nothing |
| `db-check` | `pnpm db:check`, when the pull request touches `db/`, `scripts/db-*` or `apps/server/src/db/` | the runner's own Postgres |

These run by hand, or on a pull request with the `full-ci` label:

| Check | What it runs | Needs |
|---|---|---|
| `andon-dir` | `check:andon-dir`, the `.andon/` memory lint | nothing |
| `integration` | the board's budget and Playwright e2e | the server and its own Postgres |
| `agent-linux` | the agent's end-to-end suite (`agent-int`) | Postgres, tmux |
| `compose` | `scripts/compose-smoke.sh` | Docker |
| `linux-install`, `agent-macos` | the installer in containers; the agent on macOS | Docker; a macOS runner |

`andon-dir`, `integration`, `agent-linux` and `compose` still fail on `main`. Andon's merge pipeline waits on every
check a pull request shows, required or not, so a job that fails everywhere would send every card's pull request to a CI
fix run; each moves to every pull request once it passes on `main`.

A draft pull request runs nothing until it is marked ready. A change to nothing but Markdown and `docs/` (outside
`docs/reference/`, which the `unit` suites check against the code) runs only `lint` and `secrets`; the other jobs
report "skipped", which counts as passing. The checks still don't cover e2e, so run
`pnpm typecheck`, `pnpm lint` and `pnpm test` before you push.

The workflow uses no secrets and triggers on `pull_request`, never `pull_request_target`, so a pull request from a fork
runs with a read-only token and none of the repository's secrets. The desktop signing secrets live in `desktop.yml`'s
environments, which only `main` and release tags may use.

A new gitleaks finding that is not a secret goes in `.gitleaks.toml` when a pattern describes it, or as its fingerprint
in `.gitleaksignore` when it is a one-off. Never allowlist a real key: rotate it.

Some things need a real device, a live Orca or a hosted deployment. They are listed in
[Manual checks](https://github.com/psci-labs/andon/blob/main/docs/internal/manual-checks.md).
