CHANGELOG.md

Changelog

v0.10.3 (2022-07-25)

• 08831b2 Revert "chore: Un-pin github.com/vektra/mockery/v2"
• c8eb9f7 fix: Resolved issue with kafka secret not returning SASL and TLS (#628)
• 0b07bb3 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#641)
• a3d441d build(deps): bump github.com/aws/aws-sdk-go-v2/credentials (#640)
• 9512804 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#637)
• 4419996 build(deps): bump github.com/confluentinc/confluent-kafka-go (#636)
• 93ea9a2 build(deps): bump github.com/stretchr/testify from 1.7.2 to 1.8.0 (#635)
• 8c585ba build(deps): bump github.com/prometheus/common from 0.34.0 to 0.35.0 (#634)
• 879d8c2 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#633)
• ad2327d build(deps): bump dependabot/fetch-metadata from 1.3.1 to 1.3.3 (#632)
• 5dfb4d7 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#629)
• 377455d build(deps): bump github.com/aws/aws-sdk-go-v2/credentials (#623)
• 3f929e6 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#621)
• b9de92a build(deps): bump actions/setup-python from 3 to 4 (#622)
• 2be25bf build(deps): bump github.com/aws/aws-sdk-go-v2/config (#620)
• a2156f2 build(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.2 (#619)
• b8bf2a9 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#618)
• 4107481 build(deps): bump github.com/aws/aws-sdk-go-v2/credentials (#617)
• 56b22ee build(deps): bump github.com/nats-io/nats.go from 1.15.0 to 1.16.0 (#616)
• 3c83214 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#615)
• 1334b39 build(deps): bump github.com/aws/aws-sdk-go-v2/credentials (#613)
• c808479 build(deps): bump docker/login-action from 1 to 2 (#602)
• 93a0ccb build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#612)
• 4ddd6f6 build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.16.3 to 1.16.4 (#610)
• c1d787b build(deps): bump github.com/aws/aws-sdk-go-v2/config (#609)
• c730c7b build(deps): bump github.com/prometheus/client_golang (#608)
• 36d844a build(deps): bump github.com/aws/aws-sdk-go-v2/config (#606)
• 8fbb255 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#605)
• 65ea324 build(deps): bump docker/setup-qemu-action from 1 to 2 (#603)
• 65a1817 build(deps): bump docker/setup-buildx-action from 1 to 2 (#604)
• e806be2 build(deps): bump github.com/nats-io/nats-streaming-server (#600)
• 59d0260 build(deps): bump github.com/nats-io/nats.go from 1.14.0 to 1.15.0 (#601)
• c1ee266 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#599)
• 200e9d5 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#598)
• 44bc938 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#597)
• 97c862f build(deps): bump github.com/aws/aws-sdk-go-v2/credentials (#595)
• 9eab67f build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#596)
• 79ff221 build(deps): bump github.com/nats-io/nats-streaming-server (#594)
• b33e511 build(deps): bump github.com/prometheus/common from 0.33.0 to 0.34.0 (#593)
• 439de7d build(deps): bump dependabot/fetch-metadata from 1.3.0 to 1.3.1 (#592)

Contributors

• Alex Collins
• Krise
• dependabot[bot]

v0.10.2 (2022-04-20)

• 99e0df6 fix: Add mutex. Fixes #588 (#589)
• 039d6b9 build(deps): bump github.com/nats-io/nats-streaming-server (#590)
• 4ffa8be build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#587)
• 1fe4e41 build(deps): bump actions/setup-go from 2 to 3 (#586)
• cc69d45 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#583)
• 46c2ce8 build(deps): bump sigs.k8s.io/controller-runtime from 0.11.1 to 0.11.2 (#584)
• 3fd8cdc build(deps): bump github.com/aws/aws-sdk-go-v2/config (#581)
• 17a806a build(deps): bump github.com/prometheus/common from 0.32.1 to 0.33.0 (#580)
• 4335caf build(deps): bump dependabot/fetch-metadata from 1.1.1 to 1.3.0 (#579)
• 164e4cd build(deps): bump github.com/onsi/gomega from 1.18.1 to 1.19.0 (#578)
• dcbafa1 build(deps): bump github.com/aws/aws-sdk-go-v2/config (#574)
• 7d8299f build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#573)

Contributors

• Alex Collins
• dependabot[bot]

v0.10.1 (2022-03-26)

• 53670a6 fix: run codegen
• b09fe46 build(deps): bump actions/cache from 2 to 3 (#564)
• fb10ab4 build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.16.0 to 1.16.1 (#572)
• 57a418e build(deps): bump github.com/aws/aws-sdk-go-v2/config (#569)
• 602d6ed build(deps): bump minimist from 1.2.5 to 1.2.6 in /sdks/nodejs (#571)
• a83d689 build(deps): bump peter-evans/create-pull-request from 3 to 4 (#566)
• f179a6e build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (#567)
• 653ab44 build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.15.0 to 1.16.0 (#568)
• cfebd32 build(deps): bump github.com/go-logr/logr from 1.2.2 to 1.2.3 (#563)
• bbeebc4 build(deps): bump github.com/stretchr/testify from 1.7.0 to 1.7.1 (#562)
• 8a65056 build(deps): bump github.com/nats-io/nats-streaming-server (#561)
• d739981 build(deps): upgrade ansi-regex (#560)
• 8fc6a18 build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.14.0 to 1.26.0 (#559)
• 6a6dbae build(deps): bump actions/checkout from 2 to 3 (#555)
• f637874 build(deps): bump actions/stale from 4.1.0 to 5 (#556)
• c67d978 build(deps): bump actions/setup-python from 2 to 3 (#552)
• 5ab7422 build(deps): bump github.com/nats-io/nats-streaming-server (#551)
• fdf01c9 build(deps): bump github.com/onsi/gomega from 1.17.0 to 1.18.1 (#544)

Contributors

• Alex Collins
• dependabot[bot]

v0.10.0 (2022-02-16)

• 5857b0f build(deps): bump github.com/robfig/cron/v3 from 3.0.0 to 3.0.1 (#541)
• 3663c8e build(deps): bump github.com/antonmedv/expr from 1.8.9 to 1.9.0 (#540)
• 7ea8d9b build(deps): bump github.com/nats-io/nats-streaming-server (#536)
• 2a8a6e5 build(deps): bump github.com/evanphx/json-patch (#537)
• 6a373b5 build(deps): bump github.com/prometheus/client_golang (#530)
• 3ff400c build(deps): bump github.com/confluentinc/confluent-kafka-go (#533)
• bc65bfa build(deps): bump github.com/onsi/ginkgo from 1.16.4 to 1.16.5 (#532)
• 2029ead build(deps): bump github.com/uber/jaeger-client-go (#531)

Contributors

• dependabot[bot]

v0.9.0 (2022-01-31)

• ca813ef build(deps): bump github.com/prometheus/common from 0.10.0 to 0.32.1 (#527)
• ae48d2c build(deps): bump actions/stale from 3 to 4.1.0 (#523)
• 7fe4585 Update CUSTOM_IMAGE.md (#516)
• 4a4453c fix: Add Kafka sink maxInflight to Python DSL

Contributors

• Alex Collins
• dependabot[bot]
• zewelor

v0.8.0 (2021-12-06)

• 2179640 feat: Limit Kafka sink async message in-flight (#513)
• de6a76e feat: add Kafka sink messageTimeout (#512)
• a386208 Update CONCEPTS.md (#510)
• 8f22671 feat: Added cluster scope manifests. Fixes #498 (#508)

Contributors

• Alex Collins
• Saravanan Balasubramanian
• zewelor

v0.7.0 (2021-11-19)

• b15c3bc feat: add support for InsecureIgnoreHostKey (#503)

Contributors

• Udit Chaudhary

v0.6.0 (2021-11-17)

• 85b9d28 fix!: correctly report async Kafka sink errors, retry always. Fixes #494 (#495)

Contributors

• Alex Collins

v0.5.0 (2021-11-01)

• 0c5a5e8 feat: Allow users to specify image for code processor. Fixes #482 (#491)
• 37fccf4 fix: Panic on two kafka sink (#489)
• 3b7d09a fix: Sidecar fails to fetch secrets for Sources and Sinks. Fixes #385 (#480)
• f73c782 fix: support different Python func names (#483)
• 2f14a02 feat: Upgrade to Golang 1.17. Fixes #458 (#472)

Contributors

• Alex Collins
• Saravanan Balasubramanian

v0.4.0 (2021-10-25)

• 5ec67f6 feat: Make Kafka group.id configurable. Fixes #89 (#464)
• 5572212 fix: demote Kafka error messages to info. Fixes #466 (#469)

Contributors

• Alex Collins

v0.3.0 (2021-10-21)

• 9014b30 feat: Enable DLQ (#444)
• 42034f8 fix: CRD metadata.annotations size (#457)

Contributors

• Saravanan Balasubramanian

v0.2.0 (2021-10-15)

• f81b33d feat: do not use finalizer to stop metrics cache loop (#450)

Contributors

• Derek Wang

v0.1.0 (2021-10-08)

Contributors

v0.0.128 (2021-10-08)

• b5114d6 fix: close jetstream connection (#441)
• 049ec21 feat: fix ID for cron and HTTP sources
• 3c45487 feat: tune CPU resources
• d5259ea fix: log giveUp=false at level=info

Contributors

• Alex Collins
• Derek Wang

v0.0.127 (2021-10-07)

• 31e721d feat: add enable.idempotence=true for Kafka sink

Contributors

• Alex Collins

v0.0.126 (2021-10-06)

• 0b2455f fix: try to fix memory leak in Kafka sink

Contributors

• Alex Collins

v0.0.125 (2021-10-06)

• 6423c9e fix: sensible linger Kafka defaults (#432)
• 11251f5 feat: jetstream source and sink (#400)
• 6b36b07 feat: log Kafka structured. Fixes #431
• 695e117 feat: enable Kafka debug with ARGO_DATAFLOW_DEBUG=kafka.generic
• 7d601a4 feat!: remove unused Redis/monitor code (#429)

Contributors

• Alex Collins
• Derek Wang

v0.0.124 (2021-10-05)

• 29b4deb feat: export Prom metrics for logs. Fixes #418
• 64f8f10 feat: log metrics labels on pod term
• f318ebf feat: send error logs to stderr
• fa64809 fix: correctly log Kafka async error
• fa7373b feat: expose Kafka config (#414)

Contributors

• Alex Collins

v0.0.123 (2021-10-05)

• 5bbe50a feat: cammit Kafka offset async (#411)
• e056b27 fix: do not re-balance channels (#410)

Contributors

• Alex Collins

v0.0.122 (2021-10-04)

• 4d2ad98 fix: make Kafka consumers channels size 1

Contributors

• Alex Collins

v0.0.121 (2021-10-04)

• a46256f fix: remove lock, and fix loop. Fixes #404 (#407)

Contributors

• Alex Collins

v0.0.120 (2021-10-04)

• 0b05359 fix: lock assign/revoke partition code to prevent multiple consumers starting for same partition. Fixes #404 (#405)
• d515a7e feat: Kafka thoughput (#402)

Contributors

• Alex Collins

v0.0.119 (2021-10-04)

• 7376f50 fix: fix Kafka failure mode (#395)
• 9c05fcc fix: fix Kafka source (#394)
• 5a1d295 feat: tune Kafka producer and consumer (4x TPS)

Contributors

• Alex Collins

v0.0.118 (2021-10-01)

• 647d0a6 fix: change Kafka to use stats for pending
• dc03e6c fix: change Kafka to use stats for pending
• 63ea3fa feat: change update interval from 1m to 15s
• f1e7b43 fix: race in monitor
• cb439be fix: logging metrics

Contributors

• Alex Collins

v0.0.117 (2021-10-01)

• 2720622 feat: fix bugs in monitor (#391)
• b82ef2f feat: migrate to confluent-kafka-go from Sarama (#387)
• a070247 feat: log metrics on runner stop
• 2ac7baf fix: enhance missing/duplicate monitor
• 67f5024 fix: more start-up logging detail (#386)
• f738044 fix: correct report of missing when multiple messages go missing
• e55b950 fix: updated retry/error messages with clearer detail

Contributors

• Alex Collins

v0.0.116 (2021-09-28)

• ebf38e6 fix: correct mis-reporting of missing

Contributors

• Alex Collins

v0.0.115 (2021-09-23)

• 1b91c71 feat: ignored duplicate Kafka messages (#373)

Contributors

• Alex Collins

v0.0.114 (2021-09-22)

• 9e7332a feat: Auto-detect missing/duplicate messages (#369)
• 504a63d fix: delete Unix domain socket

Contributors

• Alex Collins

v0.0.113 (2021-09-21)

• 57a1634 fix: not upating status even replicas changed (#366)
• 50742ca feat: update Python DSL to support multiple sources and sinks (#365)

Contributors

• Alex Collins
• Derek Wang

v0.0.112 (2021-09-20)

• 40eac37 fix: Kafka commit session end of consume claim

Contributors

• Alex Collins

v0.0.111 (2021-09-20)

• a618b38 fix: Partial revert of 76e1d95 (#360)
• 2176e72 fix: more reasonable desired replicas calculation (#359)

Contributors

• Alex Collins
• Derek Wang

v0.0.110 (2021-09-20)

• 6cd0639 fix: requeue interval change (#357)
• 124d6cb fix: Auto create HTTP auth secret. Fixes #319 (#353)

Contributors

• Alex Collins
• Derek Wang

v0.0.109 (2021-09-20)

• 32f3e63 fix: Correct typo that meant ARGO_DATAFLOW_UNIX_DOMAIN_SOCKET was ignored.

Contributors

• Alex Collins

v0.0.108 (2021-09-19)

• c09d10a fix: correct Unix Domain Socket env var name
• ad85dff fix: update NATS dep to fix Snyk failure (#349)

Contributors

• Alex Collins

v0.0.107 (2021-09-17)

• 657062a fix: short-term fix for pending metric not being updated (#346)

Contributors

• Alex Collins

v0.0.106 (2021-09-17)

• 06bdf8d fix: run only one Kafka offset committer per step (#343)
• 0733e32 feat: update dedupe to use NewCounter rather than NewCounterFunc (#342)
• 0764aa1 feat: scale based on metrics (#337)
• 489ded6 feat: Add dedupe to Python DSL (#341)

Contributors

• Alex Collins
• Derek Wang

v0.0.105 (2021-09-17)

• cc7706d fix: fix logging an signal handling (#339)
• ae8ad3f feat: make meta just plain string
• d2bf012 feat: Run Kafka offset commit loop every 1s, 2x faster, more robust.
• 13d5b50 fix: golang1-16 runtimes
• 84bf124 feat: add ARGO_DATAFLOW_UNIX_DOMAIN_SOCKET config (#335)

Contributors

• Alex Collins

v0.0.104 (2021-09-16)

• f50e8a7 feat: add ARGO_DATAFLOW_UNIX_DOMAIN_SOCKET config
• 4cb372e feat!: Remove step..metrics (#334)
• 884ae0d Revert "fix: Remove Golang SDK dependencies"

Contributors

• Alex Collins

v0.0.103 (2021-09-16)

• 0fe072e fix: Remove Golang SDK dependencies
• 76e1d95 fix!: Remove Kafka auto-commit. Fixes #327

Contributors

• Alex Collins

v0.0.102 (2021-09-16)

• 69e608a feat!: Remove SunkMessages condition. Add sinks_total metric. (#328)
• 4168dd0 feat!: Remove metrics.rate field.
• 2192cd5 feat!: Remove metrics.rate field.
• b2dd247 feat!: Remove metrics.rate field.
• ae75900 feat!: Remove metrics.rate field.
• 91a9d39 feat!: Remove metrics.rate field.
• 7c80fea feat: switch /var/run/... to be tmpfs
• 3665bdb feat: use unix domain socket
• 7db2d38 Revert "feat: use unix domain socket"
• 44b321e feat: use unix domain socket
• 9574072 fix: use 32 connections
• c0c3aac fix: close http source request body
• 259cf36 fix: close HTTP request body
• 3a6e5f0 feat: Meta-data. Fixes #161 (#314)
• 0ac1ec2 feat: Add OpenTracing. Fixes #295 (#322)
• 8a16a02 feat: expose replicas metric with a new approach (#315)
• dfefd4e refactor: make volumes consistent (#317)
• 1a73899 Revert "fix: uses signal.NotifyContext"
• 5c7ed77 fix: uses signal.NotifyContext
• 8e992cc chore!: change clusterName to cluster. Fixes #313
• 935cf15 fix: quick-start add create argo-server step and set default ARGO_DATAFLOW_CLUSTER_NAME env (#233)
• fe7c683 feat: export prometheus metrics from memory (#311)

Contributors

• Alex Collins
• Derek Wang
• meijin

v0.0.101 (2021-09-08)

• 9a0d7cf fix: fix-up context handling (#305)
• fbae663 fix: report error correctly
• c3cb97f feat: support insecureSkipVerify for HTTP sink (#304)
• 19bc8c8 feat: Updating Python SDK to support Generator Steps & Asyncio handlers. (#299)

Contributors

• Alex Collins
• Dominik Deren

v0.0.100 (2021-09-07)

• ab6d38a fix: log failed to process message

Contributors

• Alex Collins

v0.0.99 (2021-09-02)

• 45f7847 feat: Expand Kafka sink auto-commit config (#293)
• 6b406c2 feat: Added options MaxMessageSize and Resource for step container for processing large messages (#282)
• 01dc7a6 fix: Argo-Server should still use old image repository (#290)
• f4d0ca5 fix: fix image name in config

Contributors

• Alex Collins
• Dominik Deren
• Saravanan Balasubramanian

v0.0.98 (2021-08-30)

• 50efb44 feat: support for AWS session token for S3 (#281)

Contributors

• Vigith Maurice

v0.0.97 (2021-08-30)

• 93b7f76 fix: listen on HTTPS
• 0d5175c chore!: change map and filter from string to object
• b4376d0 fix: port TLS cert code from workflows
• 64e66d7 fix: port TLS cert code from workflows
• 3e69afd feat: make pre-commit -B

Contributors

• Alex Collins

v0.0.96 (2021-08-23)

• be2b63d feat: Use pod priority class to prioritize the lead replica. Fixes #269 (#275)
• 8bdfbf2 add kafka async metrics (#276)
• dcccd6f use default k3d cluster

Contributors

• Alex Collins
• Vigith Maurice

v0.0.95 (2021-08-19)

• 7dc5e75 chore make pre-commit -B
• 744a563 fix: fix scaling logic
• 1869fb1 fix: correct error in currentReplicas
• 53269f7 feat: Enable TotalBytes metrics (#258)
• 53a319d feat: Unique Kafka Consumer GroupID (#251)

Contributors

• Alex Collins
• Saravanan Balasubramanian

v0.0.94 (2021-08-18)

Contributors

v0.0.93 (2021-08-18)

• d7f9eb4 fix: correct copy back logic

Contributors

• Alex Collins

v0.0.92 (2021-08-18)

• db6f875 fix: correct copy back logic

Contributors

• Alex Collins

v0.0.91 (2021-08-18)

• a3c961a fix!: give scaling variable long, easy to understand, names
• 7bdff21 feat: add volume sink. Fixes #265
• 2afbc66 feat: adds volume source. Fixes #262
• 061efa5 fix: change to metav1.Duration pointer types so defaults work
• 104460f feat: add limit for desired replicas to allow more flexibility is sc… (#261)
• dc46d4d feat: surface S3 pending
• 67d14aa feat: surface S3 concurrency configuration
• 0786c0e fix: fix peek and scaling delay to Python DSL
• 531f4b8 fix: add peek and scaling delay to Python DSL
• af42349 fix: correct logging of peek/scaling delays

Contributors

• Alex Collins

v0.0.90 (2021-08-16)

• dab337c feat!: Expression based scaling. Fixes #249 (#250)
• 1e3ad21 fix: correct deletionDelay time (must be pointer to marshall correctly)
• acc648f fix: log deletion delay
• 3be7f7f fix: Correct logging of resource names
• a81dde5 feat: Move deletionDelay to pipeline spec
• 2e807b0 config: use argocli:latest

Contributors

• Alex Collins

v0.0.89 (2021-08-12)

• 3ba7708 fix: Prevent scaling from deleting pods
• 5523aa3 fix: Required debug enabled for pprof. Fixes #241

Contributors

• Alex Collins

v0.0.88 (2021-08-10)

• 85f4d1f feat: db source (#215)
• 5e07456 feat: add log sink message truncation (#225)
• c47c023 feat: Adding ability to specify imagePullSecrets for pipelines (#214)
• f5b7751 fix: update sidecar locking to try and avoid sawing metric values. Fixes #224

Contributors

• Alex Collins
• Derek Wang
• Dominik Deren

v0.0.87 (2021-08-09)

• 95f8cb2 feat: Expose async and sidecarResources in Python DSL

Contributors

• Alex Collins

v0.0.86 (2021-08-09)

• d3a7a72 feat: Enable config of sidecar resources
• 00c3178 feat: Add support for Kafka async producer. Fixes #216 (#220)

Contributors

• Alex Collins

v0.0.85 (2021-08-07)

• bbec883 feat: Updating Python runtime to use Python SDK (#212)
• d7a0540 feat: NodeJS Runtime & working examples for NodeJS & Python runtimes & sdks. (#211)

Contributors

• Dominik Deren

v0.0.84 (2021-08-06)

• ab5ccde fix: set Kafka consumer max limit to 16x default (16m)

Contributors

• Alex Collins

v0.0.83 (2021-08-06)

Contributors

v0.0.82 (2021-08-06)

• c31e5e6 fix: use one Kafka client for consumer/producer
• 1b18462 fix: use one Kafka client for consumer/producer

Contributors

• Alex Collins

v0.0.81 (2021-08-05)

• 616753f fix: nats disconnect log (#206)
• c52ff23 fix: Cache and re-use Kafka clients. Fixes #199 (#200)
• a5ba4fe fix: Change Kafka max-processing time from 100ms to 10s
• efa15a6 feat: change hash from b64 to hex (#203)
• 0066afb fix!: Mandate configured cluster name (#198)
• 4a645de Revert "fix: Removed noisy Kafka logging"
• a4d7977 fix!: Update Kafka group name/id to include namespace.
• 07cb751 feat: Set allowPrivilegeEscalation: false
• 7a25bb8 feat: Generate self-signed certificates (#196)
• 9aac4e8 fix: Fix for #190, broken nodejs sdk test. (#194)

Contributors

• Alex Collins
• Derek Wang
• Dominik Deren

v0.0.80 (2021-08-03)

• 2cfee32 feat!: Change HTTP endpoints to have TLS v1.2. Fixes #178 (#193)
• 68084f2 fix!: Add authentication to HTTP source. Fixes #152 (#187)
• 656af83 feat: NodeJS SDK for handling the container contract (#188)
• 2ba29ce fix: Removed noisy Kafka logging
• f6b8915 fix!: Move bearer token from environment variable to file.

Contributors

• Alex Collins
• Dominik Deren

v0.0.79 (2021-08-02)

• 2f3cdbc feat!: updated default backoff to cap at 1 day
• 7b00de2 feat: Drop-all capabilites by default. Fixes #142 (#181)
• 78e1b32 feat: log backoffSteps, so we can know if an error will retry

Contributors

• Alex Collins

v0.0.78 (2021-07-30)

• 2e80ead fix: allow Kafka TLS without certs

Contributors

• Alex Collins

v0.0.77 (2021-07-30)

• a3ee6a7 fix: add ssh configmap

Contributors

• Alex Collins

v0.0.76 (2021-07-30)

• 507f1e7 feat: db sink (#165)
• f1ab11c fix: Adding missing examples. (#166)
• 7e73134 fix: fix python runtime
• 76054c2 feat: Adding Python SDK for git step with Python3-9 runtime. (#163)
• c42f9be fix: fix S3 test
• 5acf7dd Update README.md (#157)
• 82c5f4c feat: updated S3 sink to use shared path rather than messages. Fixes #159
• aa70b35 fix: Fixing incorrectly passed auth arguments in git basic auth. (#156)
• ed0cc74 feat: Updating git step action to use credentials based authentication. (#151)

Contributors

• Alex Collins
• Derek Wang
• Dominik Deren

v0.0.75 (2021-07-27)

• 4ef8477 fix: improve/fix s3 support by using workqueue to avoid repeated work
• 4981e06 fix: add known_hosts for git cloning private repos
• e9b9ae6 feat: add S3 sink

Contributors

• Alex Collins

v0.0.74 (2021-07-26)

• eaa0e22 feat: upgrade to Kubernetes v0.20.5
• a9f59e6 fix: Fix S3 poll period type
• 3e9334c feat: Add S3 source
• 9e17d95 fix: do not Kafka commit error message
• bb3fcaa feat: git ssh private key

Contributors

• Alex Collins

v0.0.73 (2021-07-23)

• e166364 fix: make codegen
• 3456b18 feat: kafka authentication (#141)

Contributors

• Alex Collins
• Derek Wang

v0.0.72 (2021-07-21)

• 40211bf chore!: remove message and time from status to reduce complexity
• 18815b5 feat: add command to GitStep in Python DSL
• 003d94e feat: add terminator to Python DSL
• 8293394 feat: add terminator to Python DSL
• e5face5 feat: label service
• e4215b3 Revert "Revert "test: add expand_step_test.go. Fixes #131""
• d5cb429 Revert "test: add expand_step_test.go. Fixes #131"
• 9bc52a6 fix: use dumb-init to correctly handle signals with entrypoint.sh scripts (#134)

Contributors

• Alex Collins

v0.0.71 (2021-07-20)

• 7dce579 fix: do not loop logging error
• bc8f72c chore!: Change go1-16 to golang1-16
• 937e874 feat: add Golang SDKs draft
• 391db24 feat!: rename "handler" to "code". Fixes #99
• 29741e5 tests: add container step test. Fixes #123
• f06c4c8 tests: add map test. Fixes #126

Contributors

• Alex Collins

v0.0.70 (2021-07-20)

• d71b06d fix: report pending Kafka messages
• c4aa21b Sarama (#121)

Contributors

• Alex Collins

v0.0.69 (2021-07-19)

• 2c3e65f fix: change to use logrus rather than zap

Contributors

• Alex Collins

v0.0.68 (2021-07-16)

• 6f3cc61 feat: support configurable Kafka starting offset
• 4cdf4bb fix: Kafka should start at LastOffset by default
• af46d66 fix: correct source hook to be pre-stop hook
• 5544da5 refactor: refactor sources
• 4398de4 feat: expose maxInflight for stan config (#117)
• 51c21ad feat: Partially migrate from sarama to kafka-go (#116)
• 39cd13a feat: make Kafka commitN configurable. Fixes #114

Contributors

• Alex Collins
• Derek Wang

v0.0.67 (2021-07-15)

• 573b31c feat: stan reliable auto reconnection (#112)

Contributors

• Derek Wang

v0.0.66 (2021-07-15)

• 8f81586 feat: add version metrics. Closes #109
• 1adec35 fix: add retries labels
• 5465d3c fix: wait for Kafka

Contributors

• Alex Collins

v0.0.65 (2021-07-13)

• e8fa437 fix: use shared counter for Kafka commit
• 3ad7cf1 fix: stop Kafka dropping messages on disruption
• 9a86950 fix: return 503 if HTTP source not ready due to pre-stop
• 2155591 feat: change default deletion delay to 720h (~30d)
• 8fb4c17 fix!: harmonize retries
• 0aa7aa4 fix: add timeout to HTTP sink
• f5e89b9 fix: correct hash

Contributors

• Alex Collins

v0.0.64 (2021-07-12)

• 11b2033 feat: update resources to only apply to built-ins

Contributors

• Alex Collins

v0.0.63 (2021-07-12)

• d389e44 feat: change pod hash when image changes

Contributors

• Alex Collins

v0.0.62 (2021-07-12)

Contributors

v0.0.61 (2021-07-12)

• f306c0c fix: allow lastUpdated to be empty

Contributors

• Alex Collins

v0.0.60 (2021-07-12)

• 6ac6810 feat(controller): use version from build
• 858d717 feat: support HTTP headers for HTTP sink
• ad31740 fix: correctly step
• 4e1d2ff fix: delay start-up until Kafka is ready
• 810aef7 feat: automatic GC of pipelines 30m after completion
• 2135e1d feat: add lastUpdated field to pipeline
• e9b2218 fix: fix HTTP shutdown
• 8ac27da fix: fix HTTP shutdown
• d13284f config: update config
• 7da3df6 feat: Update retry (#91)
• 9db646a feat: Enable the Message retry metrics (#75)
• 0299cd8 feat: stan supports token auth (#87)

Contributors

• Alex Collins
• Derek Wang
• Saravanan Balasubramanian

v0.0.59 (2021-06-29)

• 778f24a fix: prevent Kafka pending loop dieing on disconnection
• f289249 refactor: container killer refactor
• 7e793bf fix: improved dedupe

Contributors

• Alex Collins

v0.0.58 (2021-06-28)

• 70acfa7 fix: improved dedupe
• 69e43d6 test http fmea

Contributors

• Alex Collins

v0.0.57 (2021-06-24)

• 972ebf0 fix: increased default CPU resources requests to 250m
• 786bc05 fix: move patching step status earlier in shutdown sequence
• 2f9898b config: removed invalid resource
• 3645436 fix: correct metrics
• 8a169ca feat: fix test
• 752086b feat: re-order logging
• 4b6a9d6 feat: pass GODEBUG to runner
• 7a08e5e feat: dedupe
• 6904aa7 feat: add sha1 func

Contributors

• Alex Collins

v0.0.56 (2021-06-22)

• ed68cb0 config: ARGO_DATAFLOW_UPDATE_INTERVAL=10s for dev
• c564148 fix(controller): correct service name

Contributors

• Alex Collins

v0.0.55 (2021-06-21)

• e5fa079 feat: pprof sidecar
• 3c0d169 fix: Kafka pending wrong when re-using pipeline
• f37aea4 fix: revert to auto-commit
• 24d91a1 fix: negative Kafka pending
• 32a73a7 fix: expose stan monitoring port (#78)

Contributors

• Alex Collins
• Derek Wang

v0.0.54 (2021-06-17)

• 0d68892 feat: add HTTP source service name
• e2728c6 feat(sidecar): disable Kafka auto-commit
• 2b90f11 fix(sidecar): fix Kafka pending tiny over-estimate
• f296a83 config: added natsMonitoringUrl
• 04fe2f8 feat(sidecar): added sidecar readiness probe
• b395ffe feat(sidecar): set kafka max-processing time to 30s
• 0fc71c2 chore(sidecar): Refactor sidecar (#70)
• 994e80e ok
• efb20b1 feat: smaller binaries without DWARF/symbols
• 2332aeb feat: smaller binaries without DWARF/symbols

Contributors

• Alex Collins

v0.0.53 (2021-06-16)

• 454c0d7 fix: set MaxInFlight and AckWait for stan
• 6b7be67 fix: fix logger

Contributors

• Alex Collins
• Derek Wang

v0.0.52 (2021-06-15)

• 12a98cd fix: stan manual ack

Contributors

• Derek Wang

v0.0.51 (2021-06-15)

• 88f77f7 feat: change Errors to RecentErrors
• a4db306 fix: bug in STAN ack

Contributors

• Alex Collins

v0.0.50 (2021-06-15)

• 6f7bb50 config: change argo-server to HTTP
• 4770ed6 config: change argo-server to HTTP
• 8d4ee45 config: change argo-server to HTTP
• 7679590 config: change argo-server to HTTP
• c63a715 refactor: rename variables
• 316abc9 Update README.md (#68)

Contributors

• Alex Collins
• wanghong230

v0.0.49 (2021-06-10)

• 23fc919 fix: Add missing kubernetes requirement in setup.py (#64)
• dbc403d fix: fix Python
• a2fdaaf feat: publish Python library
• a411be8 feat: publish Python library
• 72b3484 feat: publish Python library
• 361c47c feat: add a method to run pipelines in DSL

Contributors

• Alex Collins
• Yuan Tang

v0.0.48 (2021-06-10)

• e61c307 feat: add context to Java runtime
• 5becca6 feat: add context to Git example
• 6b922eb feat: add context to Python runtime

Contributors

• Alex Collins

v0.0.47 (2021-06-10)

• 34b75ae fix: change default retryPolicy=Always
• a9e83fa feat: add Golang context

Contributors

• Alex Collins

v0.0.46 (2021-06-09)

• 268b1ec fix: removed parallel
• 4cd5e8b feat: switch to manual ack
• e09e8e9 fix: clean up parallel for stan

Contributors

• Alex Collins
• Derek Wang

v0.0.45 (2021-06-09)

• b8e88fe fix: nats pending messages type assersion error

Contributors

• Derek Wang

v0.0.44 (2021-06-09)

• 12f0e5e fix: correct total counter
• bf7b993 fix: nats pending messages

Contributors

• Alex Collins
• Derek Wang

v0.0.43 (2021-06-08)

• a03615d ok

Contributors

• Alex Collins

v0.0.42 (2021-06-08)

• d2f219f ok
• abbf7d0 fix: various issues
• c6a9e6e feat: add retryPolicy
• 8170a88 feat: more work on Python DLS
• e306739 feat: more work on Python DLS

Contributors

• Alex Collins

v0.0.41 (2021-06-07)

• cd2082e feat: simplify so only replica 0 report metrics

Contributors

• Alex Collins

v0.0.40 (2021-06-07)

• da6a1e0 fix: fix permissions

Contributors

• Alex Collins

v0.0.39 (2021-06-07)

• e1cc03c config: expose 9090
• 5f9c09c config: expose 9090

Contributors

• Alex Collins

v0.0.38 (2021-06-07)

• fcfb45f fix: bugs in grouping
• bf55468 refactor: de-couple scaling
• 2695c7a feat: more Python DSL
• f6a6d0d feat: more Python DSL
• 85908bb feat: add Python DSL

Contributors

• Alex Collins

v0.0.37 (2021-06-06)

• 267fd6b feat: added latency metric
• c1c88da fix: in_flight -> inflight

Contributors

• Alex Collins

v0.0.36 (2021-06-05)

• ca7edf1 feat: add in_flight metric
• 009731a refactor: made code more testable
• a3142eb fix: add bearer token

Contributors

• Alex Collins

v0.0.35 (2021-06-04)

• 421c142 fix: total/error calcs
• 06fc5f0 fix: only return pending for replica=0

Contributors

• Alex Collins

v0.0.34 (2021-06-04)

• c73ce5b fix: change from patch to update for the controller
• 4a32fb9 fix: kafka pending messages

Contributors

• Alex Collins
• Derek Wang

v0.0.33 (2021-06-04)

• 60eab31 fix: prevent violent scale-up and scale-down by only scaling by 1 each time

Contributors

• Alex Collins

v0.0.32 (2021-06-04)

• ca363fe feat: add pending metric
• f3a4813 fix: fix metrics

Contributors

• Alex Collins

v0.0.31 (2021-06-04)

• 3dd39f1 fix(sidecar): updated to use fixed counters
• 6bef420 fix: removed scrape annotations
• 3a7ab5c ok

Contributors

• Alex Collins

v0.0.30 (2021-06-03)

• c6763e5 feat(runner): Request Bearer token
• 9d8676b feat(runner): Emit Prometheus metrics (#57)
• 869e492 ok
• af151b6 fix: correct STAN durable name

Contributors

• Alex Collins

v0.0.29 (2021-06-02)

• 6156b57 fix: bug where we were not getting kafka secret

Contributors

• Alex Collins

v0.0.28 (2021-05-27)

• 57567e3 fix!: change rate to resource.Quantity

Contributors

• Alex Collins

v0.0.27 (2021-05-27)

• 98eadec fix!: change rate to resource.Quantity

Contributors

• Alex Collins

v0.0.26 (2021-05-26)

• 53c4d04 feat: mark all Kafka messages
• dcee8f5 config: remove secrets for stan/kafka

Contributors

• Alex Collins

v0.0.25 (2021-05-25)

• 980c741 fix: try and avoid partition changes

Contributors

• Alex Collins

v0.0.24 (2021-05-24)

Contributors

v0.0.23 (2021-05-24)

• a561027 fix: fixed bug with pipeline conditions and messages computation
• 45f51fa feat: implement an ordered shutdown sequence

Contributors

• Alex Collins

v0.0.22 (2021-05-22)

• 2ad2f52 feat: allow messages to be return to request
• 59669ca fix: correct http POST to use keep-alives

Contributors

• Alex Collins

v0.0.21 (2021-05-21)

• 22b0b8b fix: fix bouncy scaling bug

Contributors

• Alex Collins

v0.0.20 (2021-05-21)

• de8a19f fix: enhanced shutdown

Contributors

• Alex Collins

v0.0.19 (2021-05-20)

• f3ba148 fix: correct metrics over restart
• 3c0b7fd config: pump more data thorough Kafka topic
• f753b2d feat: update examples
• 5c6832b feat: report rate
• b28c3a3 fix: report back errors

Contributors

• Alex Collins

v0.0.18 (2021-05-20)

• 9d04dcd refactor: change from for {} to wait.JitterUntil
• e17f961 feat: only check Kafka partition for pending
• d3e02ea feat: reduced update interval to every 30s
• 72f0aeb feat(runner): change name of queue
• e25a33c feat: report Kubernetes API errors related to pod/servic creation/deletion
• 36dae38 feat: only update pending for replica zero to prevent disagreement

Contributors

• Alex Collins

v0.0.17 (2021-05-19)

• e4a4b1a feat: longer status messages

Contributors

• Alex Collins

v0.0.16 (2021-05-19)

Contributors

v0.0.15 (2021-05-19)

• 9bef0df feat: expose pod failure reason

Contributors

• Alex Collins

v0.0.14 (2021-05-18)

• 2a23cb8 fix: scale to 1 rather than 0 on start

Contributors

• Alex Collins

v0.0.13 (2021-05-18)

• 6204cd1 fix: surface errors
• c000e41 fix: make minReplicas required

Contributors

• Alex Collins

v0.0.12 (2021-05-18)

• 3089119 fix: only terminate sidecars if the main container exit with code 0
• 6dc28a2 fix: failed to record sink status
• 3db2fbc fix: add missing RBAC for manager

Contributors

• Alex Collins

v0.0.11 (2021-05-18)

• b22e9fd fix: correct changelog order

Contributors

• Alex Collins

v0.0.10 (2021-05-18)

• dd8efe3 feat: and support for stateless sources and sinks with HTTP

Contributors

• Alex Collins

v0.0.9 (2021-05-17)

• 940632a feat: and 1st-class support for expand and flatten
• 9e6ab3d feat: support layout of cron source messages
• 290cde4 feat: support HPA
• d6920cf refactor: move replicas to top level

Contributors

• Alex Collins

v0.0.8 (2021-05-14)

Contributors

v0.0.7 (2021-05-14)

• 7e3a574 config: increase stan-default storage to 16Gi
• 9166870 config: increase stan-default storage to 16Gi

Contributors

• Alex Collins

v0.0.6 (2021-05-14)

• 8c2dd33 fix(manager): re-instate killing terminated steps
• 98917eb fix: report sink errors

Contributors

• Alex Collins

v0.0.5 (2021-05-13)

• 1fbfbd1 fix: correct error to lastError

Contributors

• Alex Collins

v0.0.4 (2021-05-13)

• 0d0c6d5 fix: remove version file
• 99bfd15 refactor: move api/util to ../shared/
• e26c3a9 refactor: move containerkiller
• cc175fc refactor: move controller files
• 8d8c2ae feat: add errors to step/status
• 9172dc7 fix: only calculate installed hash after changes have been applied

Contributors

• Alex Collins

v0.0.3 (2021-05-12)

• f341a7d config: change argo-server to secure
• fd75eb3 config: change argo-server to secure
• dcb24d5 config: change argo-server to secure

Contributors

• Alex Collins

v0.0.2 (2021-05-11)

Contributors