- Add extractors for Italian sources: it_m3.xls and italian.json
- Add comparison script (compare-italian.py) to report source overlaps and conflicts
- Add merge script (merge-italian-json.py) with priority order ['italian', 'it_m3']
- Output authoritative dataset to datafiles/italian-merged.json
- Update README to document both English and Italian pipelines
- add deck_terms to schema imports
- add addTermsToDeck — diffs source term IDs against existing deck_terms,
inserts only new ones, returns count of inserted terms
- add updateValidatedLanguages — recalculates and persists validated_languages
on every run so coverage stays accurate as translation data grows
- wire both functions into main with isNewDeck guard to avoid redundant
validated_languages update on deck creation
- add final summary report
- fix possible undefined on result[0] in createDeck
- tick off remaining roadmap items
Implements packages/db/src/seed.ts — reads all JSON files from
scripts/datafiles/, validates filenames against supported language
codes and POS, and upserts synsets into and
via onConflictDoNothing. Safe to re-run; produces 0 writes on
a duplicate run.
- terms, translations, term_glosses with cascade deletes and pos check constraint
- language_pairs with source/target language check constraints and no-self-pair guard
- users with openauth_sub as identity provider key
- decks and deck_terms with composite PK and position ordering
- indexes on all hot query paths (distractor generation, deck lookups, FK joins)
- SUPPORTED_POS and SUPPORTED_LANGUAGE_CODES as single source of truth in @glossa/shared
Database Schema:
- Add decks table for curated word lists (A1, Most Common, etc.)
- Add deck_terms join table with position ordering
- Link rooms to decks via rooms.deck_id FK
- Remove frequency_rank from terms (now deck-scoped)
- Change users.id to uuid, add openauth_sub for auth mapping
- Add room_players.left_at for disconnect tracking
- Add rooms.updated_at for stale room recovery
- Add CHECK constraints for data integrity (pos, status, etc.)
Extraction Script:
- Rewrite extract.py to mirror complete OMW dataset
- Extract all 25,204 bilingual noun synsets (en-it)
- Remove frequency filtering and block lists
- Output all lemmas per synset for full synonym support
- Seed data now uncurated; decks handle selection
Architecture:
- Separate concerns: raw OMW data in DB, curation in decks
- Enables user-created decks and multiple difficulty levels
- Rooms select vocabulary by choosing a deck
- Configure PostgreSQL 18 and Valkey 9.1 services
- Create multi-stage Dockerfiles for API and Web apps
- Set up pnpm workspace support in container builds
- Configure hot reload via volume mounts for both services
- Add healthchecks for service orchestration
- Support dev/production stage targets (tsx watch vs compiled)
