Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: opencdc unwrap #1355

Merged
merged 1 commit into from
Feb 1, 2024
Merged

fix: opencdc unwrap #1355

merged 1 commit into from
Feb 1, 2024

Conversation

raulb
Copy link
Member

@raulb raulb commented Jan 31, 2024

Description

Follow up from #1343.

Using this OpenCDC processor, I encountered this issue when trying to unwrap a record. Test was using two chained pipelines (file -> kafka, kafka -> file).

2024-01-31T16:33:39+00:00 ERR node stopped error="node dest-log-ciy690a2b3921p0:foo stopped with error: error executing processor: failed to write message to DLQ: DLQ nack threshold exceeded (0/1), original error: unwrap: error unwrapping record: record payload after payload.after is not a map" component=pipeline.Service node_id=dest-log-ciy690a2b3921p0:foo stack=[{"file":"/Users/rb/code/conduitio/conduit/pkg/pipeline/lifecycle.go","func":"github.com/conduitio/conduit/pkg/pipeline.(*Service).runPipeline.func2","line":577},{"file":"/Users/rb/code/conduitio/conduit/pkg/pipeline/stream/processor.go","func":"github.com/conduitio/conduit/pkg/pipeline/stream.(*ProcessorNode).Run","line":68},{"file":"/Users/rb/code/conduitio/conduit/pkg/pipeline/stream/source_acker.go","func":"github.com/conduitio/conduit/pkg/pipeline/stream.(*SourceAckerNode).registerNackHandler.func1","line":129},{"file":"/Users/rb/code/conduitio/conduit/pkg/pipeline/stream/dlq.go","func":"github.com/conduitio/conduit/pkg/pipeline/stream.(*DLQHandlerNode).Nack","line":149},{"file":"/Users/rb/code/conduitio/conduit/pkg/processor/procbuiltin/unwrap.go","func":"github.com/conduitio/conduit/pkg/processor/procbuiltin.(*unwrapProcessor).Process","line":95},{"file":"/Users/rb/code/conduitio/conduit/pkg/processor/procbuiltin/unwrap.go","func":"github.com/conduitio/conduit/pkg/processor/procbuiltin.(*openCDCUnwrapper).UnwrapPayload","line":214}]

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@raulb raulb self-assigned this Jan 31, 2024
@raulb raulb marked this pull request as ready for review February 1, 2024 12:16
@raulb raulb requested a review from a team as a code owner February 1, 2024 12:16
@raulb raulb force-pushed the raul/update-unwrapping-opencdc branch 3 times, most recently from 78a78b5 to 363aeb1 Compare February 1, 2024 12:22
@raulb raulb requested a review from hariso February 1, 2024 17:31
@raulb raulb force-pushed the raul/update-unwrapping-opencdc branch from 1dd6f5f to 1e65c4f Compare February 1, 2024 17:33
Copy link
Contributor

@hariso hariso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@raulb raulb merged commit f3fbcc7 into main Feb 1, 2024
3 checks passed
@raulb raulb deleted the raul/update-unwrapping-opencdc branch February 1, 2024 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants