archive: re-use shard cache in e2e rank tests

[keegancsmith]

Indexing the sourcegraph repo takes a couple of seconds on my laptop. Most changes we make do not change the structure of the shard. This commit introduces a default behaviour of reusing shards between go test invocations. If on CI it will always recompute. This way if we accidently think everything is alright locally it will get caught on CI.

Test Plan: CI

Based on #695

[jtibshirani]

It's always nice to cut down on test state, and have local tests match CI as closely as possible 🤔 I could see a newcomer to Zoekt scratching their heads over why a particular test passes locally but not in CI!

Would it make sense to default to no shard cache, then you could pass a flag shard_cache to speed up local testing? Or brainstorming a different approach: mark certain tests as "slow tests" and don't run them locally by default, unless a "include_slow" flag is passed?

[keegancsmith]

It's always nice to cut down on test state, and have local tests match CI as closely as possible 🤔 I could see a newcomer to Zoekt scratching their heads over why a particular test passes locally but not in CI!

Yeah you are right, probably better to just be slow :)

Would it make sense to default to no shard cache, then you could pass a flag shard_cache to speed up local testing? Or brainstorming a different approach: mark certain tests as "slow tests" and don't run them locally by default, unless a "include_slow" flag is passed?

There is the flag go test -short which is meant to skip slower tests. What I will do is abandon this PR and instead just skip the test if -short is specified.