ISR does not seem to work

[joshudev]

Brief Description of the Bug
When using the App directory, calling revalidateTag will delete all entries with the specified tag. When the user browses a page that was tagged with the tag, the page is rebuilt during the request, causing a delay for the user. I would have expected the page to be rebuilt in the background after the request has been served, as happens when using the built-in CacheHandler.

Severity
Major

Frequency of Occurrence
Always

Steps to Reproduce

1. Navigate to a page '/test' with tag 'xyz'
2. call revalidateTag('xyz') in an api endpoint - this deletes the cache entries for the tag
3. Browse to '/test' - the page is rebuilt inline rather than in the background

Expected vs. Actual Behavior
I would expect the page to be rebuilt in the background and the user who initiated the request to be served the stale content.

Environment:

• OS: MacOS
• Node.js version: v20.10.0
• @neshca/cache-handler version: 1.0.5
• next version: 14.1.0

Attempted Solutions or Workarounds

• Looked into how the stock CacheHandler works - it seems that we would need to replicate the behaviour in FetchCache and FileSystemCache?
• Referenced https://github.com/sst/open-next/blob/ff37de235e653e3aecdbb31168ab411add2d0330/packages/open-next/src/adapters/cache.ts#L81 - but it's unclear whether the approach used there would improve the situation - as they appear to use a custom revalidation system.

Impact of the Bug
We would like to containerize our app, building on a remote CI environment using a shared cache, and have ready-to-run testable Containers as built output - and this is a blocker for us

[better-salmon]

@joshudev hello!

When the user browses a page that was tagged with the tag, the page is rebuilt during the request, causing a delay for the user.

This behavior is not a bug. Native Next.js cache works the same. It makes sense to me because on-demand revalidation is a manual process.

It is also worth mentioning that pages from the Pages Router are updated synchronously on res.revalidate(pathname), and CacheHandler can not alter this. As a result, such pages are always fresh after revalidation. I think that the App Router pages inherit this aspect of on-demand revalidation.

I'm converting this issue to discussion.

[joshudev]

Thanks for getting back @better-salmon

Using the built-in cache handler + ISR with the App directory, pages are rebuilt in the background instead of blocking to rebuild for actual users. It appears that the built-in ISR cache handler is calling pages in the background to rebuild them. As we cannot have blocking page builds for active users, we have had to go with volume mounts instead of using Redis - which is a shame but hopefully the situation will improve in future as next evolves.

[cbou]

@better-salmon Same experience here. file system cache of NextJS marks the cache as expired by preserve it. This library deletes the cache from Redis. That's not the same behaviour and is quite unexpected and confusing.

@joshudev the behaviour you need should be achievable by customising the cacheHandler.get, cacheHandler.set and cacheHandler.delete See here: https://caching-tools.github.io/next-shared-cache/usage/creating-a-custom-handler