updating documentation, prettier format
This commit is contained in:
lila
parent
039ed50567
commit
e534b98bc5
16 changed files with 1271 additions and 1070 deletions
|
|
@ -1,136 +1,123 @@
|
|||
# 02 — Data Model
|
||||
|
||||
> **Purpose:** Database schema reference for LLMs working on features that query or modify data. Concatenate with 00-project-overview.md and 99-current-task.md.
|
||||
> **Last updated:** 2026-05-15
|
||||
> **Last updated:** 2026-08-01
|
||||
> **Depends on:** 00-project-overview.md
|
||||
> **Source of truth:** `packages/db/src/db/schema.ts`. If this file and the schema disagree, the schema wins.
|
||||
|
||||
---
|
||||
|
||||
## Core Tables
|
||||
## Two vocabulary schemas exist right now
|
||||
|
||||
### `terms` — Language-neutral concepts
|
||||
The database is mid-migration and contains **both** vocabulary schemas. This is the most important thing to know before writing a query.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ------------ | --------- | -------------------------------------------- | ------------------------------------------------------ |
|
||||
| `id` | uuid | PK | |
|
||||
| `pos` | varchar | CHECK: `noun`, `verb`, `adjective`, `adverb` | Part of speech |
|
||||
| `source` | varchar | | Pipeline that created this term (e.g. `kaikki`, `omw`) |
|
||||
| `source_id` | varchar | UNIQUE(`source`, `source_id`) | Idempotency key for imports |
|
||||
| `synset_id` | varchar | nullable | WordNet synset ID. Nullable for non-WordNet terms. |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
| Schema | Status |
|
||||
| ------------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| `vocabulary_entries` + `entry_translations` | **Live.** What the app queries today (`packages/db/src/models/termModel.ts`). |
|
||||
| `words` → `senses` → `translations` | **Target.** Migrated and empty. The new pipeline writes here; no app code reads it yet. |
|
||||
|
||||
**Rule:** One row per concept. The word "cat" (animal) and "cat" (nautical) are separate rows because they have different `source_id` values.
|
||||
The `terms` / `term_glosses` / `decks` / `deck_terms` tables described in earlier versions of this doc **no longer exist**.
|
||||
|
||||
Migration path: the pipeline fills `words`/`senses`/`translations`, then `termModel.ts` is rewritten against it (roadmap Phase 5), then the `vocabulary_entries` tables are dropped.
|
||||
|
||||
---
|
||||
|
||||
### `translations` — Per-language words
|
||||
## Live schema (what the app queries)
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| --------------- | ---------- | ----------------------------------- | ---------------------------------------- |
|
||||
| `id` | uuid | PK | |
|
||||
| `term_id` | uuid | FK → terms.id | |
|
||||
| `language_code` | varchar(2) | CHECK: `en`, `it`, `de`, `es`, `fr` | |
|
||||
| `text` | varchar | | The actual word |
|
||||
| `cefr_level` | varchar(2) | nullable, CHECK: `A1`–`C2` | Difficulty of THIS word in THIS language |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
### `vocabulary_entries` — one row per word sense
|
||||
|
||||
**Unique constraint:** (`term_id`, `language_code`, `text`) — allows synonyms (e.g. "dog" and "hound" for same term), prevents exact duplicates.
|
||||
| Column | Type | Constraints | Notes |
|
||||
| --------------- | ----------- | ----------------------------------- | ----------------------------------------- |
|
||||
| `id` | uuid | PK, default random | |
|
||||
| `headword` | text | NOT NULL | The word itself |
|
||||
| `language_code` | varchar(10) | CHECK `SUPPORTED_LANGUAGE_CODES` | |
|
||||
| `pos` | varchar(20) | CHECK `SUPPORTED_POS` | |
|
||||
| `sense_index` | smallint | NOT NULL, default 0 | Distinguishes senses of the same headword |
|
||||
| `gloss` | text | nullable | Definition |
|
||||
| `examples` | text[] | NOT NULL, default `[]` | |
|
||||
| `cefr_level` | varchar(2) | nullable, CHECK `A1`–`C2` | |
|
||||
| `difficulty` | varchar(20) | nullable, CHECK `DIFFICULTY_LEVELS` | |
|
||||
| `source` | varchar(50) | NOT NULL, default `"kaikki"` | |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() | |
|
||||
|
||||
**Key design:** `cefr_level` is on `translations`, not `terms`. "House" in English is A1; "domicile" is also English but B2 — same concept, different words, different difficulty.
|
||||
UNIQUE (`headword`, `language_code`, `pos`, `sense_index`) · INDEX (`language_code`, `pos`, `difficulty`)
|
||||
|
||||
### `entry_translations`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
| ---------------------- | ----------- | ----------------------------------------------- |
|
||||
| `id` | uuid | PK |
|
||||
| `entry_id` | uuid | FK → `vocabulary_entries.id`, ON DELETE CASCADE |
|
||||
| `target_language_code` | varchar(10) | CHECK `SUPPORTED_LANGUAGE_CODES` |
|
||||
| `translation` | text | NOT NULL |
|
||||
| `sense_hint` | text | nullable |
|
||||
| `cefr_level` | varchar(2) | nullable, CHECK `A1`–`C2` |
|
||||
| `difficulty` | varchar(20) | nullable, CHECK `DIFFICULTY_LEVELS` |
|
||||
| `source` | varchar(50) | NOT NULL, default `"kaikki"` |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
UNIQUE (`entry_id`, `target_language_code`, `translation`) · INDEX (`target_language_code`, `difficulty`, `entry_id`)
|
||||
|
||||
---
|
||||
|
||||
### `term_glosses` — Definitions per language
|
||||
## Target schema (what the new pipeline writes)
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| --------------- | ---------- | ----------------------------------- | ---------------------- |
|
||||
| `id` | uuid | PK | |
|
||||
| `term_id` | uuid | FK → terms.id | |
|
||||
| `language_code` | varchar(2) | CHECK: `en`, `it`, `de`, `es`, `fr` | |
|
||||
| `text` | text | | Definition/explanation |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
Three levels: a **word** has **senses**, and translations hang off a **sense**, not off the word. That is the point of the redesign — a quiz question is tied to one specific meaning.
|
||||
|
||||
**Unique constraint:** (`term_id`, `language_code`) — one gloss per term per language. Prevents left joins from multiplying question rows.
|
||||
### `words`
|
||||
|
||||
**Note:** Italian gloss coverage is sparse (~2% of terms have Italian glosses). UI falls back to English gloss when no gloss exists for the user's language.
|
||||
| Column | Type | Constraints |
|
||||
| --------------- | ----------- | -------------------------------- |
|
||||
| `id` | uuid | PK |
|
||||
| `headword` | text | NOT NULL |
|
||||
| `language_code` | varchar(10) | CHECK `SUPPORTED_LANGUAGE_CODES` |
|
||||
| `pos` | varchar(20) | CHECK `SUPPORTED_POS` |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
---
|
||||
UNIQUE `unique_word_per_language_and_pos` (`headword`, `language_code`, `pos`) · INDEX `idx_language_code_pos`
|
||||
|
||||
### `decks` — Curated wordlists
|
||||
### `senses`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| --------------------- | ------------ | ------------------------------------------------- | ------------------------------------------------------- |
|
||||
| `id` | uuid | PK | |
|
||||
| `name` | varchar | | e.g. `en-core-1000` |
|
||||
| `source_language` | varchar(2) | CHECK | Language the wordlist was built from |
|
||||
| `validated_languages` | varchar(2)[] | CHECK: source_language NOT IN validated_languages | Languages with complete translations for all deck terms |
|
||||
| `description` | text | nullable | |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
| Column | Type | Constraints |
|
||||
| ------------- | ----------- | ----------------------------------- |
|
||||
| `id` | uuid | PK |
|
||||
| `word_id` | uuid | FK → `words.id`, ON DELETE CASCADE |
|
||||
| `sense_index` | smallint | NOT NULL, default 0 |
|
||||
| `difficulty` | varchar(20) | NOT NULL, CHECK `DIFFICULTY_LEVELS` |
|
||||
| `definitions` | text[] | NOT NULL, default `[]` |
|
||||
| `examples` | text[] | NOT NULL, default `[]` |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
**Design:** One deck per frequency tier per source language. POS, difficulty, and category are query filters, not separate decks. Decks must not overlap — each term appears in exactly one tier.
|
||||
UNIQUE `unique_sense_per_word` (`word_id`, `sense_index`) · INDEX `idx_word_sense_difficulty`
|
||||
|
||||
**Source:** SUBTLEX frequency lists (per-language editions, same methodology).
|
||||
### `translations`
|
||||
|
||||
---
|
||||
| Column | Type | Constraints |
|
||||
| ---------------------- | ----------- | -------------------------------------- |
|
||||
| `id` | uuid | PK |
|
||||
| `sense_id` | uuid | FK → `senses.id`, ON DELETE CASCADE |
|
||||
| `target_language_code` | varchar(10) | CHECK `SUPPORTED_LANGUAGE_CODES` |
|
||||
| `translation` | text | NOT NULL |
|
||||
| `gender` | varchar(20) | nullable, CHECK NULL or `NOUN_GENDERS` |
|
||||
| `difficulty` | varchar(20) | NOT NULL, CHECK `DIFFICULTY_LEVELS` |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
### `deck_terms` — Junction table
|
||||
UNIQUE `unique_translation_per_sense` (`sense_id`, `target_language_code`, `translation`) · INDEX `idx_translations_sense_language_difficulty`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ------------ | --------- | ------------- | ----- |
|
||||
| `deck_id` | uuid | FK → decks.id | |
|
||||
| `term_id` | uuid | FK → terms.id | |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
**Two difficulty columns, two meanings.** `senses.difficulty` = is this _meaning_ appropriate for the level. `translations.difficulty` = is this _word_ an acceptable answer. Queries use sense difficulty as a ceiling and translation difficulty as the target; a translation's difficulty is never lower than its sense's. See `documentation/pipeline/design-doc.md` §4.
|
||||
|
||||
**PK:** (`deck_id`, `term_id`)
|
||||
**Note:** `DIFFICULTY_LEVELS` is `easy | medium | hard`. `"intermediate"` was renamed to `"medium"` and no longer exists anywhere.
|
||||
|
||||
---
|
||||
|
||||
## Auth Tables (managed by Better Auth)
|
||||
|
||||
Better Auth creates and owns these tables. Do not modify directly.
|
||||
Better Auth creates and owns `user`, `session`, `account`, and `verification`. Do not modify them directly — changes come from Better Auth config. `user.id` is `text`, not uuid, so foreign keys to it must also be `text`.
|
||||
|
||||
### `user`
|
||||
|
||||
| Column | Type | Notes |
|
||||
| ---------------- | --------- | -------------------- |
|
||||
| `id` | varchar | PK |
|
||||
| `name` | varchar | Display name |
|
||||
| `email` | varchar | |
|
||||
| `email_verified` | boolean | |
|
||||
| `image` | varchar | nullable, avatar URL |
|
||||
| `created_at` | timestamp | |
|
||||
| `updated_at` | timestamp | |
|
||||
|
||||
### `session`
|
||||
|
||||
| Column | Type | Notes |
|
||||
| ------------ | --------- | ------------- |
|
||||
| `id` | varchar | PK |
|
||||
| `user_id` | varchar | FK → user.id |
|
||||
| `token` | varchar | Session token |
|
||||
| `expires_at` | timestamp | |
|
||||
| `ip_address` | varchar | nullable |
|
||||
| `user_agent` | text | nullable |
|
||||
| `created_at` | timestamp | |
|
||||
|
||||
### `account` — Social provider links
|
||||
|
||||
| Column | Type | Notes |
|
||||
| --------------- | --------- | -------------------- |
|
||||
| `id` | varchar | PK |
|
||||
| `user_id` | varchar | FK → user.id |
|
||||
| `account_id` | varchar | Provider's user ID |
|
||||
| `provider_id` | varchar | `google` or `github` |
|
||||
| `access_token` | text | nullable |
|
||||
| `refresh_token` | text | nullable |
|
||||
| `id_token` | text | nullable |
|
||||
| `expires_at` | timestamp | nullable |
|
||||
|
||||
**Note:** One user can have multiple accounts (Google + GitHub linked to same user).
|
||||
|
||||
### `verification`
|
||||
|
||||
Email verification tokens. Unused for social-only auth but managed by Better Auth.
|
||||
- `user` — `id`, `name`, `email` (unique), `email_verified`, `image`, timestamps
|
||||
- `session` — `id`, `user_id`, `token`, `expires_at`, `ip_address`, `user_agent`
|
||||
- `account` — social provider links; one user can have both Google and GitHub
|
||||
- `verification` — email verification tokens; managed but unused for social-only auth
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -138,84 +125,61 @@ Email verification tokens. Unused for social-only auth but managed by Better Aut
|
|||
|
||||
### `lobbies`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ------------- | --------- | ------------------------------------------- | -------------------------------------------- |
|
||||
| `id` | uuid | PK | |
|
||||
| `code` | varchar | UNIQUE | Human-readable room code (e.g. `WOLF-42`) |
|
||||
| `host_id` | varchar | FK → user.id | |
|
||||
| `status` | varchar | CHECK: `waiting`, `in_progress`, `finished` | |
|
||||
| `max_players` | integer | default 4 | |
|
||||
| `settings` | jsonb | nullable | Game mode, round count, timer duration, etc. |
|
||||
| `created_at` | timestamp | default now() | |
|
||||
| `updated_at` | timestamp | default now() | Used for stale recovery |
|
||||
| Column | Type | Constraints |
|
||||
| -------------- | ----------- | --------------------------------------------------- |
|
||||
| `id` | uuid | PK |
|
||||
| `code` | varchar(10) | NOT NULL, UNIQUE — room code |
|
||||
| `host_user_id` | text | FK → `user.id`, ON DELETE CASCADE |
|
||||
| `status` | varchar(20) | NOT NULL, default `waiting`, CHECK `LOBBY_STATUSES` |
|
||||
| `created_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
### `lobby_players`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| -------------- | --------- | --------------- | ---------------------------- |
|
||||
| `id` | uuid | PK | |
|
||||
| `lobby_id` | uuid | FK → lobbies.id | |
|
||||
| `user_id` | varchar | FK → user.id | |
|
||||
| `display_name` | varchar | | Player's shown name in lobby |
|
||||
| `is_host` | boolean | default false | |
|
||||
| `joined_at` | timestamp | default now() | |
|
||||
| Column | Type | Constraints |
|
||||
| ----------- | ----------- | ------------------------------------ |
|
||||
| `lobby_id` | uuid | FK → `lobbies.id`, ON DELETE CASCADE |
|
||||
| `user_id` | text | FK → `user.id`, ON DELETE CASCADE |
|
||||
| `score` | integer | NOT NULL, default 0 |
|
||||
| `joined_at` | timestamptz | NOT NULL, default now() |
|
||||
|
||||
**Unique constraint:** (`lobby_id`, `user_id`) — one entry per player per lobby.
|
||||
**Composite PK:** (`lobby_id`, `user_id`) — no surrogate `id` column, one row per player per lobby.
|
||||
|
||||
Only lobby _membership_ is persisted. Live game state (questions, timers, per-round answers) lives in the in-memory stores in `apps/api`, not in these tables. Max players is the `MAX_LOBBY_PLAYERS` constant in `packages/shared`, not a column.
|
||||
|
||||
---
|
||||
|
||||
## Key Relationships
|
||||
|
||||
```
|
||||
terms (1) ←──→ (N) translations
|
||||
terms (1) ←──→ (N) term_glosses
|
||||
terms (N) ←──→ (N) decks via deck_terms
|
||||
user (1) ←──→ (N) sessions
|
||||
user (1) ←──→ (N) accounts
|
||||
vocabulary_entries (1) ←──→ (N) entry_translations ← live
|
||||
words (1) ←──→ (N) senses (1) ←──→ (N) translations ← target
|
||||
user (1) ←──→ (N) session
|
||||
user (1) ←──→ (N) account
|
||||
user (1) ←──→ (N) lobbies (as host)
|
||||
user (1) ←──→ (N) lobby_players
|
||||
lobbies (1) ←──→ (N) lobby_players
|
||||
lobbies (1) ←──→ (N) lobby_players (N) ←──→ (1) user
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Query Patterns
|
||||
|
||||
### Get quiz terms (singleplayer)
|
||||
All queries live in `packages/db/src/models/` — `apps/api` never imports `drizzle-orm`.
|
||||
|
||||
```sql
|
||||
SELECT t.id, t.pos, src.text AS source_text, tgt.text AS target_text, g.text AS gloss
|
||||
FROM terms t
|
||||
JOIN translations src ON src.term_id = t.id AND src.language_code = ?
|
||||
JOIN translations tgt ON tgt.term_id = t.id AND tgt.language_code = ?
|
||||
LEFT JOIN term_glosses g ON g.term_id = t.id AND g.language_code = ?
|
||||
WHERE t.pos = ? AND tgt.cefr_level IN (?)
|
||||
LIMIT ?
|
||||
```
|
||||
`termModel.ts` queries the live schema: it self-joins `vocabulary_entries` (aliased source and target), joins `entry_translations` for the answer, and fetches distractors with a separate query per question that excludes both the current entry id and the correct answer text — different entries can share a translation string.
|
||||
|
||||
### Get distractors
|
||||
The distractor query is N+1, one round trip per question. Batching is a known BACKLOG item.
|
||||
|
||||
```sql
|
||||
SELECT text FROM translations
|
||||
WHERE language_code = ? AND pos = ? AND cefr_level IN (?)
|
||||
AND term_id != ? AND text != ?
|
||||
ORDER BY RANDOM()
|
||||
LIMIT 3
|
||||
```
|
||||
|
||||
**Note:** This is the N+1 query mentioned in BACKLOG.md. Each question fetches 3 distractors separately. Batching is planned.
|
||||
Phase 5 rewrites these against `words`/`senses`/`translations`, which changes the shape: filter on `senses.difficulty` as a ceiling, then select `translations` at the requested difficulty. Target queries are sketched in `documentation/pipeline/design-doc.md` §5.
|
||||
|
||||
---
|
||||
|
||||
## Deferred Schema Extensions (Not Yet Implemented)
|
||||
|
||||
These tables are planned but do not exist yet. All are additive — they reference existing `terms` rows via FK.
|
||||
Planned, additive, and keyed off the **target** schema:
|
||||
|
||||
| Table | Purpose | Trigger |
|
||||
| --------------------- | ----------------------------------------------- | ----------------------- |
|
||||
| `noun_forms` | Gender, singular, plural, articles per language | Grammar quiz mode |
|
||||
| `verb_forms` | Conjugation tables per language | Grammar quiz mode |
|
||||
| `term_pronunciations` | IPA + audio URLs per language | Pronunciation quiz mode |
|
||||
| `user_decks` | Which decks a user studies | User customization |
|
||||
| `user_term_progress` | Spaced repetition state per user/term/language | SRS review queue |
|
||||
| `quiz_answers` | Answer history for stats/analytics | User stats dashboard |
|
||||
| Table | Purpose | Trigger |
|
||||
| -------------------- | ---------------------------------------------- | ----------------------- |
|
||||
| `inflection_forms` | Gender, plural, conjugation/declension tables | Grammar quiz mode |
|
||||
| `pronunciations` | IPA + audio URLs per language | Pronunciation quiz mode |
|
||||
| `user_word_progress` | Spaced repetition state per user/word/language | SRS review queue |
|
||||
| `quiz_answers` | Answer history for stats/analytics | User stats dashboard |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue