Skip to content

Latest commit

 

History

History
63 lines (49 loc) · 3.1 KB

caching-classloaders.md

File metadata and controls

63 lines (49 loc) · 3.1 KB

Cache classloaders to improve daemon's performance and memory consumption

Use cases

  • User facilitates the daemon more frequently without the need for restarting it. Classloaders caching improves daemon footprint and avoids memory leaks.
  • User observes faster builds:
    • less memory used, less gc is needed, faster build
    • each consecutive build is faster thanks to jvm hotspot optimisations. This is possible because classes are reused.

Implementation notes

  • in Gradle codebase we avoid direct creation of ClassLoaders (new URLCLassloader()) and we use a service that can manage caching and decide whether to reuse or create new CL.
  • classloaders need to be cached based on the classpath and the parent classloader; classpath of the classloader cache key needs to consider file hashes
  • should classloaders caching consider the 'kind' of the classloader? Do we have a case that 2 classloaders have the same classpath and parent and we explicitly
  • we probably need to avoid mutation of the classloaders. Instead, new classloader can be created based on an existing one, and the creation managed through a service.
  • how do we roll out this change incrementally? It is a breaking change. Do we offer an opt-in for this feature?

Story 1: cache basic classloaders

Enabled in-process caching of following classloaders:

  • buildSrc
  • build script
  • plugins

User visible changes

Faster builds and smaller daemon leaks

Implementation

  • new internal system property "org.gradle.caching.classloaders" can be used to turn the classloader caching 'on'
  • DefaultClassLoaderCache needs to be fixed to honor file hash (currently it uses file path)
  • DefaultScriptCompilationHandler needs to use the ClassLoaderCache
  • validate the usefulness of CachingScriptClassCompiler
  • cache needs to be capped at reasonable size (perhaps calculated from the # of projects)

Test coverage

  • Given multiple projects in the build, verify that ClassLoaders are reused when nothing changes and are rebuilt of the following changes:
    • build script is modified.
    • a parent build script is modified.
    • buildSrc classes are modified. -? plugins applied by build script are modified. -? plugins applied by a parent project are modified.
    • classpath declared in build script is modified.
    • classpath declared in parent project build script is modified.
    • an entry in the build script classpath that did not exist now does exist.
    • an entry in the build script classpath that did exist no longer does
    • a directory in the build script classpath is modified.
    • a jar in the build script classpath is modified.
    • a directory in the build script classpath is replaced with a jar, and vice versa.
  • Verify that ClassLoaders are reused for the following changes:
    • a settings script or init script are modified. -? a change is made to the build script that does not affect the generated byte code (eg change a comment).
  • Performance test that demonstrates that execution is faster than previous releases.

Open issues

This section is to keep track of assumptions and things we haven't figured out yet.