Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes support #132

Merged
merged 59 commits into from
Jul 16, 2024
Merged

Kubernetes support #132

merged 59 commits into from
Jul 16, 2024

Conversation

alexisszmundy
Copy link
Contributor

@alexisszmundy alexisszmundy commented Jun 26, 2024

WARNING :

  • Merge KraftwerkTests branch "devDuckDb" into Master before deployment
  • Contains commits from PR 125 "DuckDB Implementation"

Changelog:

  • Launch treatment with command line

  • S3/MinIO support : All file system calls goes through an interface with both OS file system and MinIO implementations

  • DuckDB implementation for output

  • Transfer Vtl datasets into DuckDB before output step

  • SQL util class for SQL operations

  • (File-by-file) Kraftwerk now exports only one .parquet file

  • Output is now made from DuckDB instead of VTL dataset

  • Removed Avro

@alicela alicela merged commit 7823f6f into main Jul 16, 2024
3 checks passed
@alicela alicela deleted the devKubernetes branch July 16, 2024 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants