External broken link prevention

[jknoxville]

Say your site has a page at /oldpage. You move it to /newpage and you update all the links so there are no broken links.

Well, no broken links on your site, but anyone else linking to it from the web will now be broken.

What I have in mind is when this happens, the site owner has to explicitly choose whether to add a redirect from oldpage -> newpage OR to explicitly accept that this external link is broken intentionally.

I think it can be achieved by a plugin that maintains a list of all the current URLs on a site (this list would be persisted in source-control and auto-updated during build). After every build, the new list of accessible URLs gets diff'd against this one, and any ones that are no longer present are highlighted and the owner forced to choose.

This "forced to choose" state could be as simple as a build failure, saying:

Path /oldpage is no longer accessible so external links to this page will break. Please either add a redirect to the replacement path, or mark this path as deleted in `deleted-pages.json`.

Starting this discussion to ask whether this sounds like a good approach, and whether anyone foresees any issues with it.

[Josh-Cena]

Considering the isolated nature of each plugin, you would have to do the path-route mapping + linkify + broken links detection logic for every content plugin (docs, pages, blog), so I guess it could be hard. There's no way to know if a link is broken without building the entire site, so your plugin essentially builds the whole site🤦‍♂️ I personally can't see a better way to do this

Still, the routes.js file is available so you do have a "list of all the current URLs" out of the box, the question being how do you generate the new one without building the whole site itself.

I'm all for this feature, but I think it may be better implemented in the core rather than as a plugin

[jknoxville]

Thanks for the reply, I think I might see what you mean.

If implemented in a plugin postBuild hook, we could use the routesPaths prop to see the exhaustive list of all routes (including docs, pages, blog etc.), and therefore diff that against the persisted one from "last time". This would let us identify newly removed routes.

But the missing piece, as I see it is the list of redirects, so we know which removed routes are "OK", but the redirects aren't currently exposed at runtime. In fact, they are computed in postBuild because they can only be computed after all routes are known.

So our "plugin" or whatever it ends up being, would need to consume the "created redirects" which are only computed post-build, and therefore not available in a normal postBuild hook.

One way to do this would be to implement this inside the existing "client-redirects-plugin". It could be an "enforce-redirects-are-created-when-routes-are-removed" option. But I'm not sure if that would be acceptable / within the intended scope of that plugin. Now that I think about it, it is conceptually quite a redirect-specific feature so may well belong inside that plugin, but that's not for me to decide.

I don't really have a preference on whether it should be in core or a plugin, except for the fact that I can create a plugin more easily :)

[slorber]

That seems a bit complex to me.

Redirects are best done using server-side redirects (using config on Netlify, CloudFlare etc... and Docusaurus is not aware of those).

The client-side redirect plugin and the scope of what Docusaurus can do here is just a best-effort for those using GitHub pages and hosts not providing server-side redirects, but is not really a best practice.

There are dedicated SEO tools that can report broken incoming links, and you can also add your own reporting on the 404 pages so that such errors are reported to any system you like.

Technically we could implement some kind of algorithm, creating an ever-growing snapshot of all site's routes (permalinks.json?) and emitting a warning when one route of this file does not exist anymore, but wouldn't this lead to an annoying experience by default? (I'd compare this to Jest snapshots, where devs tend to update them without even reviewing). If this makes sense to you, this can probably be implemented in userland with postBuild. Not sure it makes sense to enable this by default for all Docusaurus sites though.

Technically, we could add a new lifecycle hook that provides the creates routes earlier, before building the site (it would work in dev too, and you could even provide some kind of interactive mode?)