documenting the pipeline to enrich the db data, reorganizing the file structure of the data pipeline

This commit is contained in:
lila 2026-04-20 18:28:10 +02:00
parent 0ac2cef6e1
commit 07fe256abd
8 changed files with 469 additions and 35 deletions

6
.gitignore vendored
View file

@ -11,5 +11,7 @@ __pycache__/
*.pyc
data-pipeline/archive/
data-pipeline/output/
data-pipeline/sources/omw/
data-pipeline/stage-1-extract/output/
data-pipeline/stage-2-annotate/output/
data-pipeline/stage-3-enrich/output/
data-pipeline/stage-4-merge/output/