Commit graph

120 commits

Author SHA1 Message Date
lila
da9cdbfa1b updating docs to match the implemented phase 3 pipeline
The pipeline docs still described pipeline.ts as pseudocode and the
validation module as unwritten. Both have been implemented and run.

- CLAUDE.md: replace the "no executable pipeline yet" description with
  the actual module flow, plus the two invariants worth preserving
  (resumability via headword diffing, raw responses saved before parsing)
- DATA_PIPELINE.md: mark the seven implemented modules, add a module
  responsibility map and the CLI flag table, drop the resolved warning
  about hardcoded prompt values
- roadmap.md: check off phase 3 tasks 3.2/3.3/3.4, record where the
  build diverged from the plan, note that validate.ts is stricter than
  its own spec
- STATUS.md: phase 3 is data work now, not code work

Also records two open issues: the systemic difficulty-ordering rejection
cause, and the hard-tier shortfall pending a full run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 12:10:48 +02:00
lila
303bb9388c updating pipeline roadmap and status to match branch reality
Roadmap: phase 4 re-scoped to import only (migration already applied),
phase 3 gains prompt templating, wordlist dedup, structured output,
raw-response persistence, and validation tests; idempotency decision
documented (skip already-staged words, one transaction per word).
Stale paths, postgres setup, and file structure corrected.

STATUS.md: refreshed from stale 2026-05-15 Kaikki state to current
gemini-only pipeline work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 13:32:15 +02:00
lila
e534b98bc5 updating documentation, prettier format 2026-08-02 01:01:33 +02:00
lila
29844ca0a6 finishing phase 2 2026-07-24 13:53:33 +02:00
lila
061b05afb6 setting up postgres container for new schema 2026-07-23 23:57:11 +02:00
lila
e0b1f7f92f updating documentation 2026-07-23 16:57:07 +02:00
lila
61fa032d6c updating docs 2026-07-23 14:15:33 +02:00
lila
88b16a1ed7 removing not needed files 2026-07-21 17:45:10 +02:00
lila
55ddcd3180 updating documentation 2026-07-18 14:50:15 +02:00
lila
56f479da45 wip 2026-07-06 18:07:32 +02:00
lila
8e8484f875 wip 2026-07-06 15:35:22 +02:00
lila
2a6c56ed23 bugfixing #1 2026-07-06 14:27:47 +02:00
lila
afd28d934e wip 2026-07-06 13:09:30 +02:00
lila
cc89f0c75c wip 2026-06-03 00:02:19 +02:00
lila
37f6a55798 updating docs 2026-05-30 03:47:52 +02:00
lila
caa2f7d395 updating docs 2026-05-25 01:04:49 +02:00
lila
7e0311683f updating documentation 2026-05-16 01:59:43 +02:00
lila
1ba57c7e9d adding tasks 2026-05-15 23:12:55 +02:00
lila
97b0f302d0 refactoring documentation 2026-05-15 23:09:54 +02:00
lila
04a581efe1 WIP: checkpoint before stage-3 sub-stage rewrite 2026-05-12 22:13:14 +02:00
lila
73fb12ac35 feat: enrich script working, redesigning to sub-stage architecture
- Enrich script functional with timeout, progress tracking, rejection mechanism
- Identified ordering issue: CEFR voting needs validated translations first
- Redesign: round1_gloss → round1_example → round1_translations → round1_cefr
- Update data-pipeline.md with new sub-stage design and roadmap
- Qwen3.5-4B confirmed working with thinking disabled
2026-05-07 13:09:43 +02:00
lila
7f10c35e03 docs: update roadmap — stage 3 enrich script written, llama.cpp next 2026-05-05 19:30:18 +02:00
lila
1c44ef989b feat: update pipeline orchestrator for Kaikki — wire up stages 1 and 2
- Replace checkOmwExists with checkExtractedFilesExist
- Wire up importKaikki and reverseLink as real stage implementations
- Track reverse link completion via sentinel row in run_status
- Update report to use resolved_entry_cefr and entry counts
- Stages 3 onwards remain as stubs
2026-05-05 19:04:28 +02:00
lila
b5a76ee178 docs: update roadmap — stage 1 in progress, sample extraction complete 2026-05-05 18:52:10 +02:00
lila
38d8b85228 docs: rewrite data-pipeline.md for Kaikki migration 2026-05-05 17:14:48 +02:00
lila
f59399be02 feat: add db import script, fix duplicate translations in extract, add annotate script 2026-05-03 22:05:10 +02:00
lila
4fa3073412 feat: add db schema, init, and vitest config 2026-05-03 17:56:29 +02:00
lila
74cfc82bdd docs: finalise data-pipeline.md with tiebreak, pipeline.db, reports, sync 2026-05-03 17:21:02 +02:00
lila
6007fe1e38 docs: update data-pipeline.md and llm-setup.md to reflect sqlite architecture 2026-05-02 20:13:05 +02:00
lila
57d2190549 adding task to prompts 2026-04-30 00:36:28 +02:00
lila
fd9667c1fd updating documentation
Some checks failed
Build and Deploy / build-and-deploy (push) Failing after 1m27s
2026-04-28 17:26:01 +02:00
lila
98c59f33c5 formatting + adding issues 2026-04-28 16:39:36 +02:00
lila
a02d3b3335 fix: deduplicate distractors against each other, guard thin distractor pool 2026-04-28 15:44:29 +02:00
lila
a02f3b139d fix: deduplicate distractors, replace tautological test, add distractor failure test 2026-04-28 15:17:31 +02:00
lila
3d16ab0fff feat: guard against empty terms in createGameSession 2026-04-28 15:08:06 +02:00
lila
1e30f04e81 feat: add ownership check to evaluateAnswer, AuthenticatedRequest type 2026-04-28 14:39:13 +02:00
lila
fdeb769640 feat: add TTL to GameSessionStore, replay protection and session cleanup to evaluateAnswer 2026-04-28 14:03:15 +02:00
lila
54705943fa adding ticket for refactor: dependency injection for GameSessionStore via composition root 2026-04-28 13:50:56 +02:00
lila
4f59f3bc14 formatting 2026-04-28 13:18:18 +02:00
lila
2ff7d1759e refactor: extract shuffleArray to lib/utils, rename correctAnswers to terms 2026-04-28 13:17:24 +02:00
lila
c46729f365 formatting 2026-04-28 12:32:44 +02:00
lila
02ccc88d24 fix: change GAME_ROUNDS from strings to numbers 2026-04-28 12:29:46 +02:00
lila
7d3c456efe formatting 2026-04-28 12:29:32 +02:00
lila
6b6a8aac3e adding roasts of gameservice 2026-04-27 17:47:05 +02:00
lila
fbb4e59274 adding ticket structure, finishing docker credential helper setup 2026-04-26 10:51:45 +02:00
lila
091495c1db updating tasks 2026-04-26 08:46:54 +02:00
lila
e3d28e4127 updating issues 2026-04-24 18:34:43 +02:00
lila
4fabde57bd adding prompt 2026-04-24 10:27:54 +02:00
lila
e9ba8d292d updating tasks 2026-04-24 10:21:06 +02:00
lila
762cf91f86 updating tasks 2026-04-24 09:30:20 +02:00