lila/CLAUDE.md
lila da9cdbfa1b updating docs to match the implemented phase 3 pipeline
The pipeline docs still described pipeline.ts as pseudocode and the
validation module as unwritten. Both have been implemented and run.

- CLAUDE.md: replace the "no executable pipeline yet" description with
  the actual module flow, plus the two invariants worth preserving
  (resumability via headword diffing, raw responses saved before parsing)
- DATA_PIPELINE.md: mark the seven implemented modules, add a module
  responsibility map and the CLI flag table, drop the resolved warning
  about hardcoded prompt values
- roadmap.md: check off phase 3 tasks 3.2/3.3/3.4, record where the
  build diverged from the plan, note that validate.ts is stricter than
  its own spec
- STATUS.md: phase 3 is data work now, not code work

Also records two open issues: the systemic difficulty-ordering rejection
cause, and the hard-tier shortfall pending a full run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 12:10:48 +02:00

6.3 KiB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

pnpm install                       # pnpm workspaces; pnpm only
docker compose up -d               # postgres (5432), pipeline postgres (5433), valkey (6379)

pnpm dev                           # api (:3000) + web (:5173) concurrently
pnpm build                         # shared → db → api, in dependency order
pnpm typecheck                     # tsc --build --noEmit across all projects
pnpm lint                          # eslint .
pnpm format                        # prettier --write .

pnpm test                          # vitest watch, all workspace projects
pnpm test:run                      # single run (also what the pre-commit hook runs)
pnpm vitest run apps/api/src/services/gameService.test.ts   # single file
pnpm vitest run -t "returns 400"                            # single test by name

pnpm --filter @lila/db generate    # drizzle-kit generate (after editing schema.ts)
pnpm --filter @lila/db migrate     # apply migrations (uses DATABASE_URL_LOCAL)
pnpm --filter @lila/pipeline pipeline:run

packages/shared and packages/db are consumed as built dist/ output — rebuild them (pnpm --filter @lila/shared build) after changing them, or downstream packages will typecheck against stale types. pnpm --filter @lila/api dev does this automatically.

Husky pre-commit runs lint-staged (prettier + eslint --fix) then the full test suite.

Architecture

Monorepo: apps/api (Express + ws), apps/web (React 19, Vite, TanStack Router, Tailwind 4), packages/shared (Zod contract), packages/db (Drizzle), data-pipeline (vocabulary ETL). Docs live in documentation/ — ARCHITECTURE.md, DECISIONS.md, DATA_PIPELINE.md, STATUS.md, BACKLOG.md.

Strict layering in apps/api: router → controller → service → model (packages/db) → PostgreSQL. Each layer talks only to the one below it. Controllers do HTTP only (Zod safeParse, then next(error)); services hold business logic and never read req; models hold queries and no domain semantics. apps/api must never import drizzle-orm — all queries live in packages/db/src/models/.

Errors: AppError subclasses carry their own statusCode; a single errorHandler middleware maps them. ValidationError is thrown in controllers, NotFoundError in services.

packages/shared is the single source of truth for every shape crossing the API boundary (schemas/game.ts, schemas/lobby.ts, schemas/auth.ts, constants.ts). Changing a schema breaks compilation in both api and web simultaneously — that's intended. Supported languages (en/it/de/es/fr) and POS values are constants here and are CHECK-constrained in the DB schema.

WebSocket: upgrades on /ws on the same HTTP server. Better Auth cookie is validated at upgrade (ws/auth.ts), then ws/router.ts dispatches on the type field of a Zod discriminated union to ws/handlers/. Lobby membership is persisted in PostgreSQL; live game/room state lives in InMemoryGameSessionStore / InMemoryLobbyGameStore behind interfaces, so it is lost on restart (Valkey swap is planned — keep the interface boundary intact).

Answer evaluation is server-side. The correct answer is never included in what is sent to the client.

Data model: two vocabulary schemas coexist. The app reads vocabulary_entries + entry_translations (one row per word sense); the new pipeline targets words → senses → translations, which is migrated but empty. packages/db/src/models/termModel.ts is still on the live pair. Adding a language is rows, not schema changes. Auth tables (user, session, account, verification) are owned by Better Auth.

Data pipeline

data-pipeline/ was rewritten on this branch (refactor/gemini-only-pipeline): the old local-LLM/adapter architecture is gone, replaced by a Gemini-only pipeline that is implemented and running (Phase 3). Flow: sourceLists.ts (discover + dedup source-data/{lang}/{pos}) → promptTemplate.ts (render prompt placeholders) → gemini.ts (structured output with a per-batch responseSchema, retry/backoff) → validate.ts (per-entry, three-way valid/empty/invalid) → staging.ts (SQLite, one transaction per word), orchestrated by pipeline.ts. A dedicated pipeline PostgreSQL runs on port 5433; the SQLite → PostgreSQL import script is Phase 4 and does not exist yet.

Two invariants to preserve when touching it: runs are resumable because already-staged headwords are diffed out before batching, and raw responses are written to responses/ before parsing, so validation changes can be replayed without re-spending API quota. Rejected entries go to rejections/{lang}-{pos}.jsonl; "senses": [] means "not a valid word of this POS" and is skipped, not rejected.

Read documentation/DATA_PIPELINE.md for orientation and current phase status, documentation/pipeline/design-doc.md for the schema/difficulty model/Gemini JSON contract, and documentation/pipeline/roadmap.md for the phase plan. Everything in documentation/archive/ is superseded — data-pipeline-local-llm.md, llm-setup-local.md, and model-strategy-cefr-voters.md describe the removed local-LLM / CEFR-voter pipeline and are historical only.

The sense-based words/senses/translations schema is what the pipeline targets; the app is not migrated to it yet and still queries vocabulary_entries/entry_translations.

Conventions

  • TypeScript is maximally strict (tsconfig.base.json): exactOptionalPropertyTypes, noUncheckedIndexedAccess, noPropertyAccessFromIndexSignature, verbatimModuleSyntax, erasableSyntaxOnly. Index access yields T | undefined — handle it rather than loosening the config. Env vars are read as process.env["KEY"].
  • The base tsconfig deliberately omits lib/module/moduleResolution; each package sets its own (api NodeNext, web ESNext/bundler).
  • Tests are co-located (gameService.test.ts beside gameService.ts), use vitest globals, and mock @lila/db with vi.mock — no test database. Endpoint tests use supertest against the createApp() factory without starting a server.
  • All env config lives in the single root .env (see .env.example); packages/db and the pipeline both resolve it from the repo root.