lila/data-pipeline
lila c9cddf68de feat(pipeline): add data pipeline workspace and extraction stage
- rename scripts/ to data-pipeline/, archive existing scripts
- add @lila/pipeline as pnpm workspace package
- add stage-1-extract through stage-5-compare folder structure
- update SUPPORTED_LANGUAGE_CODES (add es, de, fr)
- update SUPPORTED_POS (add adjective, adverb)
- add description field to term_glosses
- add term_examples table
- run and verify db migration
- write and verify extract.py (117,659 synsets across 5 languages)
- write PIPELINE.md
2026-04-21 09:39:36 +02:00
..
stage-1-extract/scripts feat(pipeline): add data pipeline workspace and extraction stage 2026-04-21 09:39:36 +02:00
stage-2-annotate/sources/cefr feat(pipeline): add data pipeline workspace and extraction stage 2026-04-21 09:39:36 +02:00
COVERAGE.md reorganising data-pipeline folder 2026-04-20 07:37:02 +02:00
package.json adding the data-pipeline to ts and pnpm workspaces 2026-04-20 09:05:27 +02:00
tsconfig.json adding the data-pipeline to ts and pnpm workspaces 2026-04-20 09:05:27 +02:00