Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[cdc] fix paimon cdc job restart failed when schema changed #3362

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

[cdc] fix paimon cdc job restart failed when schema changed #3362

wants to merge 6 commits into from

Conversation

MOBIN-F
Copy link
Contributor

@MOBIN-F MOBIN-F commented May 21, 2024 •
edited
Loading

Purpose

Restarting the flink-paimon-cdc job schemaCompatible failed when the column change/add between the stop and restart of the flink-paimon-cdc job

  1. In cdc-postgresql, for the newly added fields, because debezium-postgresql has no DDL events, the newly added fields can only be parsed from the records, and char and varchar fields are both STRING types in debezium-postgresql, so both char and varchar are converted to STRING in cdc-postgresql.

Tests

API and Format

Documentation

@MOBIN-F MOBIN-F mentioned this pull request May 27, 2024
Closed
@MOBIN-F
Copy link
Contributor Author

MOBIN-F commented May 30, 2024

can you help review this pr? tks @JingsongLi

@MOBIN-F MOBIN-F mentioned this pull request Jun 24, 2024
Open
2 tasks
@JingsongLi
Copy link
Contributor

CC @yuzelin

@yuzelin
Copy link
Contributor

yuzelin commented Jun 24, 2024

Hello, can you solve the conflicts ?

@MOBIN-F
Copy link
Contributor Author

MOBIN-F commented Jun 25, 2024

Hello, can you solve the conflicts ?

done,please help to review this PR, thanks. @yuzelin

@MOBIN-F MOBIN-F force-pushed the supported_schemaEvolution_when_restarting_the_paimon_cdc_job branch 2 times, most recently from 77b9d82 to b7df66d Compare July 10, 2024 14:19
@MOBIN-F MOBIN-F closed this Jul 10, 2024
@MOBIN-F MOBIN-F force-pushed the supported_schemaEvolution_when_restarting_the_paimon_cdc_job branch from b7df66d to 049232b Compare July 10, 2024 14:32
@MOBIN-F
supported_schemaEvolution_when_restarting_the_paimon_cdc_job
c70a3de
@MOBIN-F
Merge remote-tracking branch 'origin/supported_schemaEvolution_when_r…
329f854 
…estarting_the_paimon_cdc_job' into supported_schemaEvolution_when_restarting_the_paimon_cdc_job
@MOBIN-F MOBIN-F reopened this Jul 10, 2024
@MOBIN-F
Merge branch 'master' into supported_schemaEvolution_when_restarting_…
b01d822 
…the_paimon_cdc_job

# Conflicts:
#	paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/action/cdc/SynchronizationActionBase.java
@MOBIN-F
resolved conflicts
da515fd
@MOBIN-F MOBIN-F changed the title [cdc]supported schemaEvolution when restarting the paimon cdc job [cdc] fix paimon cdc job restart failed when schema changed Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@MOBIN-F @JingsongLi @yuzelin