Build GraalVM Native Image nightly for ShardingSphere Proxy

[linghengqian]

Feature Request

For English only, other languages will not accept.

Please pay attention on issues you submitted, because we maybe need more details.
If no response anymore and we cannot make decision by current information, we will close it.

Please answer these questions before submitting your issue. Thanks!

Is your feature request related to a problem?

• Yes, this issue continues Consider suggesting the package name of assertThat in Code of Conduct #19845 and Update Mockito to 4.5.0 or newer to help with GraalVM integration #17451 for the issue of unit testing in GraalVM Native Image via JUnit Platform which the discussion content of .

• For observing build errors of GraalVM Native Image, this issue is an extension of Graalvm native image cause with Groovy 4 #17779 .

• The target of this issue may be related to Support Quarkus extension #12594 .

Describe the feature you would like.

• According to GraalVM team members at https://twitter.com/fniephaus/status/1565419507346071552?t=B2SPn4OIVDpiGeIkzSrDVA&s=19 , GraalVM 22.2 has been integrated with Github Action friendly, which means something like https://twitter.com/mraible/status/1498471457638293507?t=KFjwXLVc5QxTtCnTuTcIoQ&s=19 slow build mentioned by should be sufficiently mitigated.

• Due to the launch of the https://github.com/oracle/graalvm-reachability-metadata repository, for the metadata-related json required for GraalVM builds, we no longer need to consider how to place the json tested in the environment in the master branch of ShardingSphere and how to place them that cannot be used in JDK8 CI. For files missing from the build, we can commit directly to https://github.com/oracle/graalvm-reachability-metadata .

• Although considering the impact of Apache Calcite's dependencies with dynamic libraries, it is impossible to build the master branch of ShardingSphere directly using the native-image plugin of GraalVM, but I still think that setting up a nightly build in Github Action will help improve development of the user's test, and finally achieve the purpose of building a successful. After all, for regular developers, building GraalVM Native Image locally is very time-consuming, and devices with different operating systems behave differently. Joining a nightly build I think is a reasonable move.

• For Calcite, refer to https://twitter.com/intjesus/status/1567276748257259520?s=19 .

• The purpose of this issue is to provide a new specific dockerfile and start.sh and create a new cron timed CI job under https://github.com/apache/shardingsphere/tree/master/.github%2Fworkflows . At the same time the job must be allowed to fail.

[linghengqian]

• I am working on this issue. Maybe there won't be a corresponding PR soon, I'm still working on some issues that might be solved from a graalvm perspective.

• As a reminder, the following issue has not been closed, which may affect the GraalVM Native Image of ShardingSphere Proxy.

  • Support for GraalVMs native-image pgjdbc/pgjdbc#2602 Misquote.
  • native-image support with Graal and native-image.properties google/guava#3894

• Stay tuned to Support Virtual Threads (Project Loom) in Native Image oracle/graal#4920 and GraalVM JDK19 Builds oracle/graal#4957, since this issue is experimental, we don't need to think long term.

[TeslaCN]

I remember that building GraalVM Native Image consumes a lot of memory (I used GraalVM Native image in 2020, it took 20~30 GB to compile a not complicate project to Native Image). Machine of GitHub Actions only have 7 GB RAM.

[linghengqian]

• FYI, for the old version of GraalVM 22.0, Github Action needs to be adjusted to 10G memory, refer to Okta Dev's survey https://developer.okta.com/blog/2022/04/22/github-actions-graalvm .

• However, the GraalVM team does claim that GraalVM 22.2 significantly reduces the memory footprint of Native Image building, see Is it possible to increase available memory? graalvm/setup-graalvm#6 (comment) .

[TeslaCN]

Could ShardingSphere-Proxy be compiled to Native Image at present?

[linghengqian]

• The answer is no, the build for ShardingSphere Proxy is almost certain to fail, which is influenced by many factors.

• But once we have such a CI task to provide continuous tracking of build errors, we can submit Graal reachability metadata related to third-party libraries or ShardingSphere itself to https://github.com/oracle/graalvm-reachability-metadata , which will help the construction of ShardingSphere JDBC and ShardingSphere Proxy in GraalVM Native Image at the same time, and finally achieve the purpose of successful construction at a certain point in the future.

• Considering that running Mockito in Native Image must rely on the integration of JUnit 5 Platform, it is impossible for such a test to work on the master branch of ShardingSphere, and the Graal metadata test corresponding to ShardingSphere needs to be placed at https://github.com/oracle/graalvm-reachability-metadata to handle.

[linghengqian]

• Since the nightly docker image has not been generated yet, when the nightly docker image is generated, I will add the image information to the documentation and add the documentation on observability to close this issue.

• Although Native Image is currently completely unavailable, I will open a new issue tracking GraalVM Native Image usability issues after this issue is closed.

[linghengqian]

Not sure what happened to the Github Package, I opened https://github.com/orgs/community/discussions/35042.