[Feature]: Persistent Cache support

[pushkargupta9807]

What problem does this feature solve?

Faster warm starts for applications.

Currently the warm starts with webpack are slightly better than the cold starts with rspack. This is preventing us to move to rspack for local development. This support of persistent cache is a must have to onboard developers for their local development.

What does the proposed API of configuration look like?

https://webpack.js.org/configuration/cache/

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[NetherGranite]

Bump 🙂

[pushkargupta9807]

Bump :)

[woshirubbish]

Bump

[huhuaaa]

Bump

[childrentime]

@jerrykingxyz I think when solving this problem, we can conveniently add remote caching functionality. https://turbo.build/repo/docs/core-concepts/remote-caching

[hardfist]

@jerrykingxyz I think when solving this problem, we can conveniently add remote caching functionality. turbo.build/repo/docs/core-concepts/remote-caching

this may depend on portable persistent cache which is another feature

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[slorber]

Hi @jerrykingxyz , just saw your RFCs here:

• RFC: Persistent Cache Architecture #8458
• RFC: Persistent Cache Snapshot #8463

I'm not totally able to understand the design, but just wanted to make sure that you consider supporting these f:

• Persistent caching should be optimized to work well when compiling multiple module graph variants (client, server, RSC...) using multi-compiler, layers etc
• Persistent caching should be optimized for sequential compilations of multiple module graph variants (see Webpak issue: Improve Persistent Caching for multiple configurations webpack/webpack#13034)

Example pseudo-code from Docusaurus building localized documentation, but I guess the same could be similar for Rspress:

function compile(configs: Configuration[]): Promise<void> {
  return new Promise((resolve, reject) => {
    const compiler = Rspack(configs);
    compiler.run(/* ... */);
  });
}

const locales = ["en", "fr", "de"];

// sequential (on purpose)
for (locale of locales) {
  await compile([
    getConfig({ server: true, locale }),
    getConfig({ server: false, locale })
  ]);
  await runSSG(locale);
}

We are compiling 3 localized apps in 2 variants (client/server). The localized apps share a lot of code in common, apart the content itself (MDX).

My ideal expectations for a greatly optimized caching system (for both memory/persistent)

// Slower on cold builds
compile(clientConfig)

// A bit faster since it can reuse some of the client config cache (memory or persistent)
compile(serverConfig)

// Fastest since it can reuse server config cache
compile(serverConfig)

// Faster than compiling each config using cold caches
compile([clientConfig, serverConfig])

// Slower on cold builds
compile(french);

// A bit faster since it can reuse some of the french config cache (memory or persistent)
compile(english)

// Fastest since it can reuse english config cache
compile(english)

Does this make sense?

---

What didn't work well for us in how Webpack designed its persistent cache is that when running sequential compilations, the latest compilation was somehow overriding all the previous ones, instead of having a way to "append" to the persistent cache.

For example:

// Cold build => slow
compile(french)

// Warm build but different locale => decent amount of cache hits => a bit faster 
compile(english)

// Warn build with the exact same locale as last compilation => many cache hits => fastest 🎉
compile(english)

// Warm build with different locale => Unfortunately not as fast as we hoped! 😭
compile(french)

[jerrykingxyz]

@slorber The current design of rspack persistent cache does not support shared cache with different configurations. For your needs, the configuration and conclusions on rspack persistent cache are consistent with those of webpack.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!