Commit graph

6 commits

Author SHA1 Message Date
lila
c9cddf68de feat(pipeline): add data pipeline workspace and extraction stage
- rename scripts/ to data-pipeline/, archive existing scripts
- add @lila/pipeline as pnpm workspace package
- add stage-1-extract through stage-5-compare folder structure
- update SUPPORTED_LANGUAGE_CODES (add es, de, fr)
- update SUPPORTED_POS (add adjective, adverb)
- add description field to term_glosses
- add term_examples table
- run and verify db migration
- write and verify extract.py (117,659 synsets across 5 languages)
- write PIPELINE.md
2026-04-21 09:39:36 +02:00
lila
07fe256abd documenting the pipeline to enrich the db data, reorganizing the file structure of the data pipeline 2026-04-20 18:28:10 +02:00
lila
a3d19d36f6 adding the data-pipeline to ts and pnpm workspaces 2026-04-20 09:05:27 +02:00
lila
200b14ef64 reoganising folders/files 2026-04-20 08:50:27 +02:00
lila
1f42239779 reorganising file structure 2026-04-20 07:48:44 +02:00
lila
3f125ba162 reorganising data-pipeline folder 2026-04-20 07:37:02 +02:00