How to filter a repo and carry that forward to a downstream fork?

[oryandunn]

I've used git-filter-repo to great effect on several repos. The challenge before me now, is how to use it, and carry it forward to downstream forks that base work off of commits in the upstream repo. In this particular case, the base or upstream repo had some large files erroneously commited a while back and no one noticed. Now, I'd like to get this cleaned up. The downstream repo is a fork that has additional commits/branches/tags. Is it possible to run git-filter-repo on the base repo, removing files and adding the replace refs, such that I can then use that as a base to fix the downstream fork?

[me-and]

Forks are essentially special cases of all the collaboration concerns discussed at https://github.com/newren/git-filter-repo/blob/main/Documentation/git-filter-repo.txt#L472-L608

However, if you can tolerate the large files only being cleaned up by eventual Git garbage collection, rather than as part of the immediate git-filter-repo operation (probably true if you're concerned about large files, they're not causing an immediate and direct problem, and you're not trying to clean up secrets that got accidentally committed), and you control (or can at least direct) both the base repository and all the forks involved, I think this is possible. There are (as ever) lots of ways to do it, but the way I'd go about it is thus (untested, but if it doesn't work, I'm pretty sure it'll get you most of the way…):

1. Create a single local repository that has the base repository and all the forks you care about listed as remotes, and make sure you have an up-to-date fetch of them all.
2. Do whatever you need to do with git-filter-repo. You'll need to use --force, as fetching multiple remotes will mean filter-repo (correctly) detects it isn't a simple clone.
3. Push each set of rewritten commits from your local repository to the appropriate remote; this will probably be git push --force-with-lease origin refs/heads/*:refs/heads/* for whatever repo you had as "origin" (if any), and git push --force-with-lease <remote> refs/remotes/<remote>/*:refs/heads/* for the others.

This is reusing existing repositories, rather that throwing away the existing repository and starting anew, which has a bunch of specific caveats in the documentation I linked above. I've suggested doing things that way as it'll make keeping things like the fork relationships in place much easier, but means wherever the repositories are stored are unlikely to benefit from the space savings straight away – they'll keep the large files in their object stores for some indefinite amount of time until they get subjected to Git's garbage collection. It will at least mean that anyone taking a fresh clone of the repositories is unlikely to need to download the large files, as they won't be reachable from any of the refs they're downloading.