# Decisions Log A record of non-obvious technical decisions made during development, with reasoning. Intended to preserve context across sessions. Grouped by topic area. --- ## Tooling ### Monorepo: pnpm workspaces (not Turborepo) Turborepo adds parallel task running and build caching on top of pnpm workspaces. For a two-app monorepo of this size, plain pnpm workspace commands are sufficient and there is one less tool to configure and maintain. ### TypeScript runner: `tsx` (not `ts-node`) `tsx` is faster, requires no configuration, and uses esbuild under the hood. `ts-node` is older and more complex to configure. `tsx` does not do type checking — that is handled separately by `tsc` and the editor. Installed as a dev dependency in `apps/api` only. ### ORM: Drizzle (not Prisma) Drizzle is lighter — no binary, no engine. Queries map closely to SQL. Migrations are plain SQL files. Works naturally with Zod for type inference. Prisma would add Docker complexity (engine binary in containers) and abstraction that is not needed for this schema. ### WebSocket: `ws` library (not Socket.io) For rooms of 2–4 players, Socket.io's room management, transport fallbacks, and reconnection abstractions are unnecessary overhead. The WS protocol is defined explicitly as a Zod discriminated union in `packages/shared`, giving the same type safety guarantees. Reconnection logic is deferred to Phase 7. ### Auth: Better Auth (not OpenAuth or Keycloak) Better Auth embeds as middleware in the Express API — no separate auth service or Docker container. It connects to the existing PostgreSQL via the Drizzle adapter and manages its own tables (user, session, account, verification). Social providers (Google, GitHub) are configured in a single config object. Session validation is a function call within the same process, not a network request. OpenAuth was considered but requires a standalone service and leaves user management to you. Keycloak is too heavy for a single-app project. --- ## Docker ### Multi-stage builds for monorepo context Both `apps/web` and `apps/api` use multi-stage Dockerfiles (`deps`, `dev`, `builder`, `runner`) because the monorepo structure requires copying `pnpm-workspace.yaml`, root `package.json`, and cross-dependencies before installing. Stages allow caching `pnpm install` separately from source code changes. ### Vite as dev server (not Nginx) In development, `apps/web` uses `vite dev` directly, not Nginx. HMR requires Vite's WebSocket dev server. Production will use Nginx to serve static Vite build output. --- ## Architecture ### Express app structure: factory function pattern `app.ts` exports a `createApp()` factory function. `server.ts` imports it and calls `.listen()`. This allows tests to import the app directly without starting a server (used by supertest). ### Zod schemas belong in `packages/shared` Both the API and frontend import from the same schemas. If the shape changes, TypeScript compilation fails in both places simultaneously — silent drift is impossible. ### Server-side answer evaluation The correct answer is never sent to the frontend in `GameQuestion`. It is only revealed in `AnswerResult` after the client submits. Prevents cheating and keeps game logic authoritative on the server. ### `safeParse` over `parse` in controllers `parse` throws a raw Zod error → ugly 500 response. `safeParse` returns a result object → clean 400 with early return via the error handler. ### POST not GET for game start `GET` requests have no body. Game configuration is submitted as a JSON body → `POST` is semantically correct. ### Model parameters use shared types, not `GameRequestType` The model layer should not know about `GameRequestType` — that's an HTTP boundary concern. Parameters are typed using the derived constant types (`SupportedLanguageCode`, `SupportedPos`, `DifficultyLevel`) exported from `packages/shared`. ### Model returns neutral field names, not quiz semantics `getGameTerms` returns `sourceText` / `targetText` / `sourceGloss` rather than `prompt` / `answer` / `gloss`. Quiz semantics are applied in the service layer. Keeps the model reusable for non-quiz features. ### Asymmetric difficulty filter Difficulty is filtered on the target (answer) side only. A word can be A2 in Italian but B1 in English, and what matters is the difficulty of the word being learned. ### optionId as integer 0-3, not UUID Options only need uniqueness within a single question; cheating prevented by shuffling, not opaque IDs. ### questionId and sessionId as UUIDs Globally unique, opaque, natural Valkey keys when storage moves later. ### gloss is `string | null` rather than optional Predictable shape on the frontend — always present, sometimes null. ### GameSessionStore stores only the answer key Minimal payload (`questionId → correctOptionId`) for easy Valkey migration. All methods are async even for the in-memory implementation, so the service layer is already written for Valkey. ### Distractors fetched per-question (N+1 queries) Correct shape for the problem; 10 queries on local Postgres is negligible latency. ### No fallback logic for insufficient distractors Data volumes are sufficient; strict query throws if something is genuinely broken. ### Distractor query excludes both term ID and answer text Prevents duplicate options from different terms with the same translation. ### Submit-before-send flow on frontend User selects, then confirms. Prevents misclicks. ### Multiplayer mechanic: simultaneous answers (not buzz-first) All players see the same question at the same time and submit independently. The server waits for all answers or a 15-second timeout, then broadcasts the result. Keeps the experience symmetric. ### Room model: room codes (not matchmaking queue) Players create rooms and share a human-readable code (e.g. `WOLF-42`). Auto-matchmaking deferred. --- ## Error Handling ### `AppError` base class over error code maps A `statusCode` on the error itself means the middleware doesn't need a lookup table. New error types are self-contained — one class, one status code. `ValidationError` (400) and `NotFoundError` (404) extend `AppError`. ### `next(error)` over `res.status().json()` in controllers Express requires explicit `next(error)` for async handlers — it does not catch async errors automatically. Centralises all error formatting in one middleware. Controllers stay clean: validate, call service, send response. ### Zod `.message` over `.issues[0]?.message` Returns all validation failures at once, not just the first. Output is verbose (raw JSON string) — revisit formatting post-MVP if the frontend needs structured `{ field, message }[]` error objects. ### Where errors are thrown `ValidationError` is thrown in the controller (the layer that runs `safeParse`). `NotFoundError` is thrown in the service (the layer that knows whether a session or question exists). The service doesn't know about HTTP — it throws a typed error, and the middleware maps it to a status code. --- ## Testing ### Mocked DB for unit tests (not test database) Unit tests mock `@glossa/db` via `vi.mock` — the real database is never touched. Tests run in milliseconds with no infrastructure dependency. Integration tests with a real test DB are deferred post-MVP. ### Co-located test files `gameService.test.ts` lives next to `gameService.ts`, not in a separate `__tests__/` directory. Convention matches the `vitest` default and keeps related files together. ### supertest for endpoint tests Uses `createApp()` factory directly — no server started. Tests the full HTTP layer (routing, middleware, error handler) with real request/response assertions. --- ## TypeScript Configuration ### Base config: no `lib`, `module`, or `moduleResolution` Intentionally omitted from `tsconfig.base.json` because different packages need different values — `apps/api` uses `NodeNext`, `apps/web` uses `ESNext`/`bundler` (Vite). Each package declares its own. ### `outDir: "./dist"` per package The base config originally had `outDir: "dist"` which resolved relative to the base file location, pointing to the root `dist` folder. Overridden in each package with `"./dist"`. ### `apps/web` tsconfig: deferred to Vite scaffold Filled in after `pnpm create vite` generated tsconfig files. The generated files were trimmed to remove options already covered by the base. ### `rootDir: "."` on `apps/api` Set explicitly to allow `vitest.config.ts` (outside `src/`) to be included in the TypeScript program. ### Type naming: PascalCase `supportedLanguageCode` → `SupportedLanguageCode`. TypeScript convention. ### Primitive types: always lowercase `number` not `Number`, `string` not `String`. The uppercase versions are object wrappers and not assignable to Drizzle's expected primitive types. ### `globals: true` with `"types": ["vitest/globals"]` Using Vitest globals requires `"types": ["vitest/globals"]` in each package's tsconfig. Added to `apps/api`, `packages/shared`, `packages/db`, and `apps/web/tsconfig.app.json`. --- ## ESLint ### Two-config approach for `apps/web` Root `eslint.config.mjs` handles TypeScript linting across all packages. `apps/web/eslint.config.js` adds React-specific plugins only. ESLint flat config merges them by directory proximity. ### Coverage config at root only Vitest coverage configuration lives in the root `vitest.config.ts` only. Produces a single aggregated report. --- ## Data Model ### Users: Better Auth manages the user table Better Auth creates and owns the user table (plus session, account, verification). The account table links social provider identities to users — one user can have both Google and GitHub linked. Other tables (rooms, stats) reference user.id via FK. No need to design a custom user schema or handle provider-specific claims manually. ### Rooms: `updated_at` for stale recovery only Most tables omit `updated_at`. `rooms.updated_at` is kept specifically for identifying rooms stuck in `in_progress` status after server crashes. ### Translations: UNIQUE (term_id, language_code, text) Allows multiple synonyms per language per term (e.g. "dog", "hound" for same synset). Prevents exact duplicate rows. ### One gloss per term per language The unique constraint on `term_glosses` was tightened from `(term_id, language_code, text)` to `(term_id, language_code)` to prevent left joins from multiplying question rows. Revisit if multiple glosses per language are ever needed. ### Decks: `source_language` + `validated_languages` (not `pair_id`) One deck can serve multiple target languages as long as translations exist for all its terms. `source_language` is the language the wordlist was curated from. `validated_languages` is recalculated on every generation script run. Enforced via CHECK: `source_language` is never in `validated_languages`. ### Decks: wordlist tiers as scope (not POS-split decks) One deck per frequency tier per source language (e.g. `en-core-1000`). POS, difficulty, and category are query filters applied inside that boundary. Decks must not overlap — each term appears in exactly one tier. ### Decks: SUBTLEX as wordlist source (not manual curation) The most common 1000 nouns in English are not the same 1000 nouns that are most common in Italian. SUBTLEX exists in per-language editions derived from subtitle corpora using the same methodology — making them comparable. `en-core-1000` built from SUBTLEX-EN, `it-core-1000` from SUBTLEX-IT. ### `language_pairs` table: dropped Valid pairs are implicitly defined by `decks.source_language` + `decks.validated_languages`. The table was redundant. ### Terms: `synset_id` nullable (not NOT NULL) Non-WordNet terms won't have a synset ID. Postgres `UNIQUE` on a nullable column allows multiple NULL values. ### Terms: `source` + `source_id` columns Once multiple import pipelines exist (OMW, Wiktionary), `synset_id` alone is insufficient as an idempotency key. Unique constraint on the pair. Postgres allows multiple NULL pairs. `synset_id` remains for now — deprecate during a future pipeline refactor. ### `cefr_level` on `translations` (not `terms`) CEFR difficulty is language-relative, not concept-relative. "House" in English is A1, "domicile" is also English but B2 — same concept, different words, different difficulty. Added as nullable `varchar(2)` with CHECK. ### Categories + term_categories: empty for MVP Schema exists. Grammar maps to POS (already on `terms`), Media maps to deck membership. Thematic categories require a metadata source still under research. ### CHECK over pgEnum for extensible value sets `ALTER TYPE enum_name ADD VALUE` in Postgres is non-transactional — cannot be rolled back if a migration fails. CHECK constraints are fully transactional. Rule: pgEnum for truly static sets, CHECK for any set tied to a growing constant. ### `language_code` always CHECK-constrained Unlike `source` (only written by import scripts), `language_code` is a query-critical filter column. A typo would silently produce missing data. Rule: any column game queries filter on should be CHECK-constrained. ### Unique constraints make explicit FK indexes redundant Postgres automatically creates an index to enforce a unique constraint. A separate index on the leading column of an existing unique constraint adds no value. --- ## Data Pipeline ### Seeding v1: batch, truncate-based For dev/first-time setup. Read JSON, batch inserts in groups of 500, truncate tables before each run. Simple and fast. Key pitfalls encountered: - Duplicate key on re-run: truncate before seeding - `onConflictDoNothing` breaks FK references: when it skips a `terms` insert, the in-memory UUID is never written, causing FK violations on `translations` - `forEach` doesn't await: use `for...of` - Final batch not flushed: guard with `if (termsArray.length > 0)` after loop ### Seeding v2: incremental upsert, multi-file For production / adding languages. Extends the database without truncating. Each synset processed individually (no batching — need real `term.id` from DB before inserting translations). Filename convention: `sourcelang-targetlang-pos.json`. ### CEFR enrichment pipeline Staged ETL: `extract-*.py` → `compare-*.py` (quality gate) → `merge-*.py` (resolve conflicts) → `enrich.ts` (write to DB). Source priority: English `en_m3 > cefrj > octanove > random`, Italian `it_m3 > italian`. Enrichment results: English 42,527/171,394 (~25%), Italian 23,061/54,603 (~42%). Both sufficient for MVP. Italian C2 has only 242 terms — noted as constraint for distractor algorithm. ### Term glosses: Italian coverage is sparse OMW gloss data is primarily English. English glosses: 95,882 (~100%), Italian: 1,964 (~2%). UI falls back to English gloss when no gloss exists for the user's language. ### Glosses can leak answers Some WordNet glosses contain the target-language word in the definition text (e.g. "Padre" in the English gloss for "father"). Address during post-MVP data enrichment — clean glosses, replace with custom definitions, or filter at service layer. ### `packages/db` exports fix The `exports` field must be an object, not an array: ```json "exports": { ".": "./src/index.ts", "./schema": "./src/db/schema.ts" } ``` --- ## API Development: Problems & Solutions 1. **Messy API structure.** Responsibilities bleeding across layers. Fixed with strict layered architecture. 2. **No shared contract.** API could return different shapes silently. Fixed with Zod schemas in `packages/shared`. 3. **Type safety gaps.** `any` types, `Number` vs `number`. Fixed with derived types from constants. 4. **`getGameTerms` in wrong package.** Model queries in `apps/api` meant direct `drizzle-orm` dependency. Moved to `packages/db/src/models/`. 5. **Deck generation complexity.** 12 decks assumed, only 2 needed. Then skipped entirely for MVP — query terms table directly. 6. **GAME_ROUNDS type conflict.** `z.enum()` only accepts strings. Keep as strings, convert to number in service. 7. **Gloss join multiplied rows.** Multiple glosses per term per language. Fixed by tightening unique constraint. 8. **Model leaked quiz semantics.** Return fields named `prompt`/`answer`. Renamed to neutral `sourceText`/`targetText`. 9. **AnswerResult wasn't self-contained.** Frontend needed `selectedOptionId` but schema didn't include it. Added. 10. **Distractor could duplicate correct answer.** Different terms with same translation. Fixed with `ne(translations.text, excludeText)`. 11. **TypeScript strict mode flagged Fisher-Yates shuffle.** `noUncheckedIndexedAccess` treats `result[i]` as `T | undefined`. Fixed with non-null assertion + temp variable. --- ## Known Issues / Dev Notes ### glossa-web has no healthcheck Vite's dev server has no built-in health endpoint. `depends_on` uses API healthcheck as proxy. For production (Nginx), add a health endpoint or TCP port check. ### Valkey memory overcommit warning Harmless in dev. Fix before production: add `vm.overcommit_memory = 1` to host `/etc/sysctl.conf`. --- ## Open Research ### Semantic category metadata source Categories (`animals`, `kitchen`, etc.) are in the schema but empty. Options researched: 1. **WordNet domain labels** — already in OMW, coarse and patchy 2. **Princeton WordNet Domains** — ~200 hierarchical domains, freely available, meaningfully better 3. **Kelly Project** — CEFR levels AND semantic fields, designed for language learning. Could solve frequency tiers and categories in one shot 4. **BabelNet / WikiData** — rich but complex integration, licensing issues 5. **LLM-assisted categorization** — fast and cheap at current term counts, not reproducible without saving output 6. **Hybrid (WordNet Domains + LLM gap-fill)** — likely most practical 7. **Manual curation** — full control, too expensive at scale **Current recommendation:** research Kelly Project first. If coverage is insufficient, go with Option 6. ### SUBTLEX → `cefr_level` mapping strategy Raw frequency ranks need mapping to A1–C2 bands before tiered decks are meaningful. Decision pending. ### Future extensions: morphology and pronunciation All deferred post-MVP, purely additive (new tables referencing existing `terms`): - `noun_forms` — gender, singular, plural, articles per language (source: Wiktionary) - `verb_forms` — conjugation tables per language (source: Wiktionary) - `term_pronunciations` — IPA and audio URLs per language (source: Wiktionary / Forvo)