Add an option to allow module caching

[merlinnot]

🚀 Feature Proposal

I'd like to propose adding a configuration option to allow caching modules between test files, so modules are imported once per worker.

Motivation

There are a number of issues reporting issues when importing modules, such as #8972, #7874, #6814, #6738, #6725.

While I do understand a reasoning behind isolating modules, there are quite some modules out there that heavily leverage the fact that modules are singletons. As an example, many @Google-Cloud libraries open a persistent connection to the backend (HTTP/2) and don't even expose an API to close it, so modules are never removed from memory. It also slows down test execution, significantly in case of large modules (loading JSON files, performing complex computations at start, heavily leveraging caching, ...).

Trying to force everything to run in isolation does not work for all use cases.

Example

As an example, when I'm using @google-cloud/pubsub and @google-cloud/firestore modules, which maintain a persistent HTTP/2 connection to backends, I'd like them to be reused across all tests executing in a given worker.

Pitch

Why does this feature belong in the Jest core platform?

In the video it is mentioned that Jest Platform should work with existing tooling. For uses cases like this, it doesn't. This feature would allow it to work in environments heavily dependent on singletons.

[StringEpsilon]

I like this, as a whitelist for node_modules at the very least:

{
  "moduleCacheWhitelist": [
    "@google-cloud/pubsub", 
    "@google-cloud/firestore", 
    "@material-ui/icons"
  ]
}

That way, jest can isolate modules on default, to ensure that the tests don't influence each other. And you get control to override that isolation for problematic modules like the ones mentioned in the issues you linked.

A flat option like --cacheNodeModules wouldn't cut it.

[SimenB]

Why not mock out those dependencies? If they open up persistent connections they are doing IO that doesn't have to happen in a unit test. I highly doubt we'll ever give you an opt-out for the sandbox, even if it's per module

[merlinnot]

These libraries do local IO. For all of these services, local emulators are provided, which closely resemble production environments. It's the only way to test everything in a very thorough way.

At my company, we have a very high test coverage (99.95+), but we also write very thorough tests, which let us confidently work with master=production approach. Local emulation of all side effects allows us to test what libraries actually do with our usage of them, catching both invalid usage, but also bugs in all libraries (which happen).

[merlinnot]

Here you can see emulators which Google is releasing, and they are working on more: https://cloud.google.com/sdk/gcloud/reference/beta/emulators/

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 7 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.