forgejo-lila/lila
1
0
Fork 0
Code Issues 50 Pull requests Projects 1 Releases Packages Wiki Activity Actions

updating documentation

Browse source
This commit is contained in:
lila 2026-07-23 16:57:07 +02:00
parent de3ebf66a9
commit e0b1f7f92f
2 changed files with 139 additions and 143 deletions
Download patch file Download diff file Expand all files Collapse all files

47
documentation/pipeline/design-doc.md
View file

@ -3,7 +3,7 @@
> **Project:** PERN-stack vocabulary trainer with Gemini-powered data pipeline
> **Author:** [Your Name]
> **Date:** July 2026
> **Status:** Approved — ready for implementation
> **Status:** Approved — schema implemented
---
@ -83,9 +83,8 @@ staging).
│ headword │ └───>│ word_id(FK) │ └───>│ sense_id (FK) │
│ language_code│ │ sense_index │ │ target_lang_code │
│ pos │ │ difficulty │ │ translation │
│ │ │ cefr_level │ │ gender │
│ │ │ definitions │ │ difficulty │
│ │ │ examples │ │ │
│ │ │ definitions │ │ gender │
│ │ │ examples │ │ difficulty │
└─────────────┘ └─────────────┘ └──────────────────┘
Future (not yet implemented):
@ -119,7 +118,7 @@ One row per unique word in a specific language.
**Index:**
- `idx_words_lang_pos ON (language_code, pos)` — accelerates the
- `idx_language_code_pos ON (language_code, pos)` — accelerates the
primary game query filter.
**Design note:** Each language gets its own headword entries. "Haus"
@ -139,7 +138,6 @@ two senses of one word.
| word_id | UUID | FK → words.id, ON DELETE CASCADE | |
| sense_index | SMALLINT | NOT NULL, default 0 | 0 = primary meaning, 1 = secondary, etc. |
| difficulty | VARCHAR(20) | NOT NULL, CHECK in allowed list | "easy", "medium", "hard" |
| cefr_level | VARCHAR(2) | nullable, CHECK in allowed list | "A1","A2","B1","B2","C1","C2" |
| definitions | TEXT[] | NOT NULL, default '{}' | 1–3 definitions in the word's language |
| examples | TEXT[] | NOT NULL, default '{}' | 1–3 example sentences in the word's language |
| created_at | TIMESTAMPTZ | NOT NULL, default now() | |
@ -148,18 +146,11 @@ two senses of one word.
- `UNIQUE (word_id, sense_index)` — one sense per index per word.
- `CHECK (difficulty IN ('easy','medium','hard'))`
- `CHECK (cefr_level IS NULL OR cefr_level IN ('A1','A2','B1','B2','C1','C2'))`
**CEFR → difficulty mapping:**
- A1, A2 → easy
- B1, B2 → medium
- C1, C2 → hard
**Index:**
- `idx_senses_word_diff ON (word_id, difficulty)` — accelerates the
join from words and the difficulty filter.
- `idx_word_sense_difficulty ON (word_id, difficulty)` — accelerates
the join from words and the difficulty filter.
**Design note — definitions and examples as arrays:**
Definitions and examples are stored as `TEXT[]` arrays on the sense
@ -201,7 +192,7 @@ different difficulty levels (e.g., "Bank" easy, "Geldinstitut" medium).
**Index:**
- `idx_translations_sense_lang_diff ON (sense_id, target_language_code, difficulty)`
- `idx_translations_sense_language_difficulty ON (sense_id, target_language_code, difficulty)`
— accelerates the join from senses and the language/difficulty filter.
**Design note — gender as a real column:**
@ -409,7 +400,6 @@ The API is prompted to return an array of objects. Expected shape:
"senses": [
{
"sense_index": 0,
"cefr_level": "A1",
"difficulty": "easy",
"definitions": ["Ein Gebäude zum Wohnen."],
"examples": ["Sie kauften ein Haus in der Stadt."],
@ -442,7 +432,6 @@ The API is prompted to return an array of objects. Expected shape:
},
{
"sense_index": 1,
"cefr_level": "C1",
"difficulty": "hard",
"definitions": ["Ein Adelsgeschlecht, eine Dynastie."],
"examples": ["Das Haus der Merowinger herrschte über Franken."],
@ -474,8 +463,6 @@ Before writing to SQLite, every entry is checked:
- At least one sense per word.
- Each sense has at least one definition and one example.
- `difficulty` is one of: `easy`, `medium`, `hard`.
- `cefr_level` is one of: `A1`, `A2`, `B1`, `B2`, `C1`, `C2`.
- CEFR → difficulty mapping is consistent.
- `gender` is valid for the target language:
- German: masculine, feminine, neuter
- French, Spanish, Italian: masculine, feminine
@ -502,13 +489,13 @@ used only as a pipeline staging file.
Three indexes cover the game and distractor queries:
```sql
CREATE INDEX idx_words_lang_pos
CREATE INDEX idx_language_code_pos
ON words (language_code, pos);
CREATE INDEX idx_senses_word_diff
CREATE INDEX idx_word_sense_difficulty
ON senses (word_id, difficulty);
CREATE INDEX idx_translations_sense_lang_diff
CREATE INDEX idx_translations_sense_language_difficulty
ON translations (sense_id, target_language_code, difficulty);
```
@ -563,13 +550,13 @@ one-time or occasional operation.
## 9. Implementation Plan
```
1. ✅ Design doc (this document)
2. Acquire word frequency lists (5 languages, nouns first)
3. Build + test Gemini prompt (5 words → 20 words → full batch)
4. Validation script (Gemini output → clean JSON)
5. SQLite staging schema + pipeline write
6. Import script (SQLite → local Postgres)
7. Drizzle schema: words, senses, translations + indexes
1. ✅ Schema design (this document)
2. ✅ Drizzle schema: words, senses, translations, relations, indexes
3. Acquire word frequency lists (5 languages, nouns first)
4. Build + test Gemini prompt (5 words → 20 words → full batch)
5. Validation script (Gemini output → clean JSON)
6. SQLite staging schema + pipeline write
7. Import script (SQLite → local Postgres)
8. Drizzle migration on local Postgres
9. Dev branch: new queries (game + distractor), full game flow test
10. Drizzle migration on prod Postgres + data import + verify

231
documentation/pipeline/roadmap.md
View file

@ -14,42 +14,48 @@
This roadmap has three zoom levels:
- **Part 1 — Overview:** The phases at a glance. Read this to understand the full scope in 30 seconds.
- **Part 2 — Phase Breakdown:** Goals, tasks, dependencies, and acceptance criteria per phase. Read this to plan your week.
- **Part 3 — Detailed Tasks:** Step-by-step instructions within each phase. Read this when you sit down to code.
- **Part 1 — Overview:** The phases at a glance. Read this to
understand the full scope in 30 seconds.
- **Part 2 — Phase Breakdown:** Goals, tasks, dependencies, and
acceptance criteria per phase. Read this to plan your week.
- **Part 3 — Detailed Tasks:** Step-by-step instructions within each
phase. Read this when you sit down to code.
---
# Part 1 — High-Level Overview
```
Phase 0 Preparation
Phase 1 Schema ✅
New Drizzle schema (words, senses, translations) with
relations, constraints, and indexes. Committed.
Phase 2 Preparation
Get wordlists, set up tooling, finalize the Gemini prompt.
Phase 1 Data Pipeline
Phase 3 Data Pipeline
Build the Gemini → validate → SQLite pipeline.
Produce a clean dataset of ~1000 nouns × 5 languages.
Phase 2 Schema & Migration
Create the new Drizzle schema (words, senses, translations).
Phase 4 Migration & Import
Generate and apply the Drizzle migration.
Write the SQLite → Postgres import script.
Migrate the local dev database.
Phase 3 App Integration
Phase 5 App Integration
Rewrite the game and distractor queries against the new schema.
Update the exercise-generation logic.
Test the full game flow in dev.
Phase 4 Production Deploy
Phase 6 Production Deploy
Run the Drizzle migration on prod.
Import the dataset.
Verify the live app works end-to-end.
Phase 5 Extend POS
Phase 7 Extend POS
Run the pipeline for verbs, adjectives, adverbs.
No schema changes needed — new wordlists + adjusted prompts.
Phase 6 Future Features (out of scope for now)
Phase 8 Future Features (out of scope for now)
Inflection tables, conjugation/declension exercises,
gender exercises, spaced-repetition scheduling.
```
@ -57,22 +63,41 @@ Phase 6 Future Features (out of scope for now)
**Dependency chain:**
```
Phase 0 → Phase 1 → Phase 2 → Phase 3 → Phase 4 → Phase 5
Phase 1 ✅ → Phase 2 → Phase 3 → Phase 4 → Phase 5 → Phase 6 → Phase 7
│
▼
Phase 6
Phase 8
```
Each phase depends on the previous one being complete. Do not skip
ahead.
---
# Part 2 — Phase Breakdown
---
## Phase 0: Preparation
## Phase 1: Schema ✅ COMPLETE
**Goal:** New normalized schema exists in Drizzle with all tables,
constraints, indexes, and relations.
**Completed tasks:**
- [x] `words` table: headword, language_code, pos, UNIQUE, CHECKs, index
- [x] `senses` table: word_id FK (cascade), sense_index, difficulty,
definitions TEXT[], examples TEXT[], UNIQUE, CHECK, index
- [x] `translations` table: sense_id FK (cascade), target_language_code,
translation, gender (nullable), difficulty, UNIQUE, CHECKs, index
- [x] Relations: words→senses (many), senses→word (one) + translations
(many), translations→sense (one)
- [x] `NOUN_GENDERS` constant added to `@lila/shared`
- [x] `DIFFICULTY_LEVELS` updated: "intermediate" → "medium"
- [x] Old tables (`vocabulary_entries`, `entry_translations`) untouched
- [x] Auth and lobby tables untouched
- [x] Build passes, committed
---
## Phase 2: Preparation
**Goal:** Have everything you need before writing pipeline code.
@ -86,11 +111,12 @@ ahead.
- [ ] Refine the prompt until the JSON output matches the contract
defined in `docs/schema-design.md` §6.3
**Dependencies:** None.
**Dependencies:** Phase 1 complete.
**Acceptance criteria:**
- You have 5 wordlist files in `data/wordlists/` (one per language).
- You have 5 wordlist files in `data-pipeline/source-data/` (one per
language).
- A Gemini call with 5 German nouns returns valid JSON matching the
contract, including definitions, examples, translations with gender,
and difficulty levels.
@ -98,7 +124,7 @@ ahead.
---
## Phase 1: Data Pipeline
## Phase 3: Data Pipeline
**Goal:** A repeatable script that takes a wordlist, calls Gemini in
batches of 20, validates the output, and writes clean rows to SQLite.
@ -112,7 +138,7 @@ batches of 20, validates the output, and writes clean rows to SQLite.
- [ ] Review the rejection log, fix prompt issues, re-run failed batches
- [ ] Spot-check 50 random entries for correctness
**Dependencies:** Phase 0 complete.
**Dependencies:** Phase 2 complete.
**Acceptance criteria:**
@ -125,16 +151,13 @@ batches of 20, validates the output, and writes clean rows to SQLite.
---
## Phase 2: Schema & Migration
## Phase 4: Migration & Import
**Goal:** The new normalized schema exists in Drizzle, the local
Postgres database is migrated, and the SQLite data is imported.
**Goal:** The new schema exists in Postgres and the SQLite data is
imported.
**Tasks:**
- [ ] Write the new Drizzle schema file: - `words` table (headword, language_code, pos) - `senses` table (word_id FK, sense_index, difficulty,
cefr_level, definitions TEXT[], examples TEXT[]) - `translations` table (sense_id FK, target_language_code,
translation, gender, difficulty) - All CHECK constraints, UNIQUE constraints, indexes
- [ ] Generate the Drizzle migration (`npx drizzle-kit generate`)
- [ ] Inspect the generated SQL file — verify it creates the right
tables, constraints, and indexes
@ -146,7 +169,7 @@ Postgres database is migrated, and the SQLite data is imported.
- [ ] Run 3–5 manual SQL queries against Postgres to sanity-check
the data
**Dependencies:** Phase 1 complete (SQLite has data).
**Dependencies:** Phase 3 complete (SQLite has data).
**Acceptance criteria:**
@ -158,7 +181,7 @@ Postgres database is migrated, and the SQLite data is imported.
---
## Phase 3: App Integration
## Phase 5: App Integration
**Goal:** The running app uses the new schema. Game rounds and
distractors work correctly for all language pairs.
@ -177,7 +200,7 @@ distractors work correctly for all language pairs.
- [ ] Test edge cases: - A difficulty/pos/language combo with very few words
(does the app handle < 4 available words gracefully?) - A word with multiple senses (does the correct sense appear?)
**Dependencies:** Phase 2 complete (Postgres has data, schema exists).
**Dependencies:** Phase 4 complete (Postgres has data, schema exists).
**Acceptance criteria:**
@ -189,7 +212,7 @@ distractors work correctly for all language pairs.
---
## Phase 4: Production Deploy
## Phase 6: Production Deploy
**Goal:** The live app runs on the new schema with the new data.
@ -203,7 +226,7 @@ distractors work correctly for all language pairs.
- [ ] Monitor for errors (server logs, browser console) for 24h
- [ ] Remove old tables from prod (after confirming everything works)
**Dependencies:** Phase 3 complete (dev is fully working).
**Dependencies:** Phase 5 complete (dev is fully working).
**Acceptance criteria:**
@ -214,7 +237,7 @@ distractors work correctly for all language pairs.
---
## Phase 5: Extend POS
## Phase 7: Extend POS
**Goal:** Verbs, adjectives, and adverbs are in the database and
usable in the app.
@ -229,7 +252,7 @@ usable in the app.
- [ ] Test game flow with verbs, adjectives, adverbs
- [ ] Verify the POS filter in the app UI works for all types
**Dependencies:** Phase 4 complete.
**Dependencies:** Phase 6 complete.
**Acceptance criteria:**
@ -238,7 +261,7 @@ usable in the app.
---
## Phase 6: Future Features (out of scope)
## Phase 8: Future Features (out of scope)
Listed here for visibility. Not planned, not estimated.
@ -255,9 +278,9 @@ Listed here for visibility. Not planned, not estimated.
---
## Phase 0: Preparation — Detailed
## Phase 2: Preparation — Detailed
### 0.1 Acquire wordlists
### 2.1 Acquire wordlists
- Search for frequency lists. Good starting points:
- "Leipzig Corpora Collection" (academic, per-language)
@ -267,11 +290,11 @@ Listed here for visibility. Not planned, not estimated.
- Tatoeba sentence counts as a proxy for word frequency
- Target: ~1000 nouns per language, sorted by frequency.
- Format: plain text, one word per line, UTF-8, no headers.
- Save to `data/wordlists/nouns_de.txt`, `nouns_es.txt`, etc.
- Save to `data-pipeline/source-data/{language}/nouns/`.
- Clean the lists: remove duplicates, remove words with spaces
(multi-word expressions), remove proper nouns if desired.
### 0.2 Set up local Postgres
### 2.2 Set up local Postgres
- Option A (Docker):
```
@ -288,7 +311,7 @@ Listed here for visibility. Not planned, not estimated.
- Verify: connect with `psql` or a GUI client (TablePlus, DBeaver,
pgAdmin). Run `SELECT 1;`.
### 0.3 Write and test the Gemini prompt
### 2.3 Write and test the Gemini prompt
- Start with 5 German nouns.
- The prompt should specify:
@ -297,7 +320,6 @@ Listed here for visibility. Not planned, not estimated.
- That translations are needed for all 4 other supported languages
- That gender must be provided for de/fr/es/it, null for en
- That difficulty must be one of: easy, medium, hard
- That CEFR level must be one of: A1, A2, B1, B2, C1, C2
- That multiple senses should be included for polysemous words
- Call the API. Inspect the JSON. Common issues to fix:
- Gemini wraps the JSON in markdown code fences → strip them
@ -308,12 +330,12 @@ Listed here for visibility. Not planned, not estimated.
---
## Phase 1: Data Pipeline — Detailed
## Phase 3: Data Pipeline — Detailed
### 1.1 Create the SQLite schema
### 3.1 Create the SQLite schema
- Create a file `pipeline/schema.sql` or define it in your pipeline
script.
- Create a file `data-pipeline/schema.sql` or define it in your
pipeline script.
- Tables mirror the Postgres schema:
```sql
@ -331,7 +353,6 @@ Listed here for visibility. Not planned, not estimated.
word_id TEXT NOT NULL REFERENCES words(id),
sense_index INTEGER NOT NULL DEFAULT 0,
difficulty TEXT NOT NULL,
cefr_level TEXT,
definitions TEXT NOT NULL DEFAULT '[]', -- JSON array as text
examples TEXT NOT NULL DEFAULT '[]', -- JSON array as text
created_at TEXT DEFAULT (datetime('now')),
@ -353,9 +374,9 @@ Listed here for visibility. Not planned, not estimated.
- Note: SQLite has no native TEXT[]. Store arrays as JSON text.
The import script will parse them into Postgres TEXT[] on import.
### 1.2 Write the validation module
### 3.2 Write the validation module
- Create `pipeline/validate.ts` (or .js).
- Create `data-pipeline/validate.ts`.
- Input: one parsed Gemini entry (the JSON object for one word).
- Checks (return a list of errors, empty = valid):
- `headword` is a non-empty string
@ -365,8 +386,6 @@ Listed here for visibility. Not planned, not estimated.
- Each sense has:
- `sense_index` is a non-negative integer
- `difficulty` is in ['easy','medium','hard']
- `cefr_level` is in ['A1','A2','B1','B2','C1','C2'] or null
- CEFR ↔ difficulty mapping is consistent
- `definitions` is a non-empty array of non-empty strings
- `examples` is a non-empty array of non-empty strings
- `translations` is a non-empty array
@ -381,9 +400,9 @@ Listed here for visibility. Not planned, not estimated.
- `difficulty` is in ['easy','medium','hard']
- Output: `{ valid: boolean, errors: string[] }`
### 1.3 Write the pipeline script
### 3.3 Write the pipeline script
- Create `pipeline/run.ts`.
- Create `data-pipeline/run.ts`.
- Pseudocode:
```
for each language in [de, en, es, fr, it]:
@ -398,7 +417,7 @@ Listed here for visibility. Not planned, not estimated.
generate UUIDs for word, senses, translations
INSERT into SQLite (words, senses, translations)
else:
append to rejection log (data/rejections/{lang}_{pos}.jsonl)
append to rejection log
log progress: "Batch 12/50 done. 238 words imported, 2 rejected."
sleep 1s (rate limiting)
```
@ -407,7 +426,7 @@ Listed here for visibility. Not planned, not estimated.
- Store definitions/examples as JSON strings in SQLite
(`JSON.stringify(arr)`).
### 1.4 Run and review
### 3.4 Run and review
- Run the pipeline for all 5 languages.
- Check the rejection log. If rejection rate > 10%, fix the prompt
@ -427,31 +446,21 @@ Listed here for visibility. Not planned, not estimated.
---
## Phase 2: Schema & Migration — Detailed
## Phase 4: Migration & Import — Detailed
### 2.1 Write the Drizzle schema
- Create or update `src/db/schema.ts`.
- Define the three tables using Drizzle's `pgTable` builder.
- Include all columns, types, constraints, and indexes as specified
in `docs/schema-design.md` §3.
- Keep the old tables in the file for now (commented out or in a
separate file) so the app doesn't break during development.
### 2.2 Generate and inspect the migration
### 4.1 Generate and inspect the migration
- Run: `npx drizzle-kit generate`
- Open the generated SQL file in `drizzle/` (or wherever your
config puts it).
- Open the generated SQL file in `packages/db/drizzle/`.
- Read it. Verify:
- Three CREATE TABLE statements
- Three CREATE TABLE statements (words, senses, translations)
- CHECK constraints match your schema
- UNIQUE constraints are present
- Three CREATE INDEX statements
- Foreign keys reference the correct tables with ON DELETE CASCADE
- If something looks wrong, fix the schema file and regenerate.
### 2.3 Apply the migration locally
### 4.2 Apply the migration locally
- Run: `npx drizzle-kit migrate`
- Connect to local Postgres and verify:
@ -462,9 +471,9 @@ Listed here for visibility. Not planned, not estimated.
\d translations -- describe translations table
```
### 2.4 Write the import script
### 4.3 Write the import script
- Create `scripts/import-to-postgres.ts`.
- Create `data-pipeline/import-to-postgres.ts`.
- Pseudocode:
```
@ -492,7 +501,7 @@ Listed here for visibility. Not planned, not estimated.
into actual arrays (Postgres TEXT[]).
- Use `ON CONFLICT DO NOTHING` to handle re-runs gracefully.
### 2.5 Run and verify
### 4.4 Run and verify
- Run the import script.
- Compare counts:
@ -529,11 +538,11 @@ Listed here for visibility. Not planned, not estimated.
---
## Phase 3: App Integration — Detailed
## Phase 5: App Integration — Detailed
### 3.1 Rewrite getGameTerms
### 5.1 Rewrite getGameTerms
- Open the file containing `getGameTerms`.
- Open `packages/db/src/models/termModel.ts`.
- Replace the old query (2-table join on `vocabulary_entries` +
`entry_translations`) with the new 3-table join
(words → senses → translations).
@ -545,7 +554,7 @@ Listed here for visibility. Not planned, not estimated.
- Return: word_id, headword, sense_id, definitions, examples,
translation, gender.
### 3.2 Rewrite getDistractors
### 5.2 Rewrite getDistractors
- Same 3-table join.
- Additional filters:
@ -555,7 +564,7 @@ Listed here for visibility. Not planned, not estimated.
- If fewer than 3 distractors are found (small data pool), handle
gracefully: reduce the number of options or log a warning.
### 3.3 Update exercise generation
### 5.3 Update exercise generation
- In the function that assembles a game round:
- Pick one random definition:
@ -566,7 +575,7 @@ Listed here for visibility. Not planned, not estimated.
- Shuffle the 4 options (Fisher-Yates or similar).
- Attach gender to each option for display.
### 3.4 Test matrix
### 5.4 Test matrix
Run through this matrix manually in the dev app:
@ -584,18 +593,18 @@ appear as distractors, no duplicate options.
---
## Phase 4: Production Deploy — Detailed
## Phase 6: Production Deploy — Detailed
### 4.1 Pre-deploy checklist
### 6.1 Pre-deploy checklist
- [ ] All Phase 3 acceptance criteria pass
- [ ] All Phase 5 acceptance criteria pass
- [ ] `git status` is clean, all changes committed
- [ ] The Drizzle migration file is committed to the repo
- [ ] You know your prod database connection string
- [ ] You have a backup method for prod (pg_dump, hosting provider
snapshot, etc.)
### 4.2 Deploy
### 6.2 Deploy
- Back up prod:
```
@ -607,11 +616,11 @@ appear as distractors, no duplicate options.
```
- Run import script against prod:
```
DATABASE_URL=<prod-url> npx tsx scripts/import-to-postgres.ts
DATABASE_URL=<prod-url> npx tsx data-pipeline/import-to-postgres.ts
```
- Verify counts on prod.
### 4.3 Post-deploy verification
### 6.3 Post-deploy verification
- Open the live app. Play 2 full games with different settings.
- Check server logs for errors.
@ -623,24 +632,24 @@ appear as distractors, no duplicate options.
```
(Or keep them for another week if you want a safety net.)
### 4.4 Rollback plan
### 6.4 Rollback plan
If something goes wrong:
- Restore the backup:
```
psql -h <host> -U <user> -d <db> < backup_20260722.sql
psql -h <host> -U <user> -d <db> < backup_YYYYMMDD.sql
```
- Revert the code to the previous commit.
- Redeploy.
---
## Phase 5: Extend POS — Detailed
## Phase 7: Extend POS — Detailed
### 5.1 Per POS
### 7.1 Per POS
Repeat the Phase 1 pipeline for each new POS:
Repeat the Phase 3 pipeline for each new POS:
- Acquire wordlists (verbs, adjectives, adverbs) for all 5 languages.
- Adjust the Gemini prompt:
@ -653,7 +662,7 @@ Repeat the Phase 1 pipeline for each new POS:
- Run pipeline → validate → SQLite → import → Postgres.
- Test in the app with the POS filter set to the new type.
### 5.2 No schema changes
### 7.2 No schema changes
The `pos` column already supports all four types. The `gender`
column is nullable and simply won't be populated for adverbs or
@ -678,27 +687,27 @@ English words. No migration needed.
```
project/
├── docs/
│ ├── schema-design.md ← companion design doc
│ └── roadmap.md ← this document
├── data/
│ ├── wordlists/
│ │ ├── nouns_de.txt
│ │ ├── nouns_en.txt
│ │ ├── nouns_es.txt
│ │ ├── nouns_fr.txt
│ │ └── nouns_it.txt
├── data-pipeline/
│ ├── source-data/
│ │ ├── german/nouns/ ← wordlist files
│ │ ├── english/nouns/
│ │ ├── spanish/nouns/
│ │ ├── french/nouns/
│ │ └── italian/nouns/
│ ├── rejections/ ← invalid Gemini entries (for review)
│ └── staging.db ← SQLite staging database
├── pipeline/
│ ├── run.ts ← main pipeline script
│ ├── staging.db ← SQLite staging database
│ ├── schema.sql ← SQLite schema definition
│ ├── validate.ts ← validation module
│ ├── prompt.ts ← Gemini prompt template
│ └── schema.sql ← SQLite schema definition
├── scripts/
│ ├── run.ts ← main pipeline script
│ └── import-to-postgres.ts ← SQLite → Postgres import
├── src/
│ └── db/
│ └── schema.ts ← Drizzle schema (updated)
└── drizzle/ ← generated migration files
├── packages/
│ ├── db/
│ │ └── src/db/schema.ts ← Drizzle schema (updated ✅)
│ └── shared/
│ └── src/constants.ts ← NOUN_GENDERS added ✅, "medium" ✅
├── documentation/
│ └── pipeline/
│ ├── design-doc.md ← schema design doc (updated ✅)
│ └── roadmap.md ← this document (updated ✅)
└── ...
```