Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-36695][Formats] Bump Parquet dependency to 1.14.4 #25646

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

MartijnVisser
Copy link
Contributor

What is the purpose of the change

  • Bump Parquet dependency

Brief change log

  • Update POM and NOTICE files

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): yes
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

@flinkbot
Copy link
Collaborator

flinkbot commented Nov 12, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@MartijnVisser
Copy link
Contributor Author

@flinkbot run azure

@snuyanzin
Copy link
Contributor

snuyanzin commented Nov 25, 2024

Hey @MartijnVisser
it seems the reason of failure

Nov 12 18:17:39 java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: Provider com.fasterxml.jackson.datatype.jdk8.Jdk8Module not found
Nov 12 18:17:39 	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:589)

(could be seen at https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=63635&view=logs&j=e60d5d3b-2e2f-5bb7-484a-16135ea73209&t=9363ed9e-7ac2-5835-c98d-4fdb89b19329&l=20959)

is that with parquet-jackson 1.13.x there were jackson-jaxrs and jackson-xc coming

flink/flink-formats/flink-parquet$ ../../mvnw dependency:tree | grep jackson
[INFO] |  +- org.apache.flink:flink-shaded-jackson:jar:2.15.3-19.0:provided
[INFO] |     \- com.fasterxml.jackson.core:jackson-core:jar:2.15.3:compile
[INFO] |  +- org.apache.parquet:parquet-jackson:jar:1.13.1:runtime
[INFO] |  |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:provided
[INFO] |  |  \- org.codehaus.jackson:jackson-xc:jar:1.8.3:provided
[INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] |  \- com.fasterxml.jackson.core:jackson-databind:jar:2.15.3:compile
[INFO] |     \- com.fasterxml.jackson.core:jackson-annotations:jar:2.15.3:compile

and with 1.14 it doesn't

flink/flink-formats/flink-parquet$ ../../mvnw dependency:tree | grep jackson
[INFO] |  +- org.apache.flink:flink-shaded-jackson:jar:2.15.3-19.0:provided
[INFO] |     \- com.fasterxml.jackson.core:jackson-core:jar:2.15.3:compile
[INFO] |  +- org.apache.parquet:parquet-jackson:jar:1.14.4:runtime
[INFO] |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.15.3:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:provided
[INFO] |  |  \- org.codehaus.jackson:jackson-xc:jar:1.8.3:provided
[INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] |  \- com.fasterxml.jackson.core:jackson-databind:jar:2.15.3:compile
[INFO] |     \- com.fasterxml.jackson.core:jackson-annotations:jar:2.15.3:compile

thus since it seems flink-python relies on it then need to add extra dependencies there i guess

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants