Cargo 1.56.0 (silently) fails to update the index after an index squash

[psychon]

Problem

Cargo claims that the newest libc version is 0.2.126 even though the newest version is 0.2.132 and cargo should now about it:

$ cd ~/.cargo/registry/index/github.com-1ecc6299db9ec823/.git
$ git show $(cut -b 1-20 FETCH_HEAD):li/bc/libc | tail -n 7 | grep -o '"vers":"[^"]*"'
"vers":"0.2.126"
"vers":"0.2.127"
"vers":"0.2.128"
"vers":"0.2.129"
"vers":"0.2.130"
"vers":"0.2.131"
"vers":"0.2.132"

Steps

$ cargo --version
cargo 1.56.0
$ cargo new --bin foobar
     Created binary (application) `foobar` package
$ cd foobar
$ echo 'libc = "0.2.127"' >> Cargo.toml
$ cargo build
    Updating crates.io index
error: failed to select a version for the requirement `libc = "^0.2.127"`
candidate versions found which didn't match: 0.2.126, 0.2.125, 0.2.124, ...
location searched: crates.io index
required by package `foobar v0.1.0 (/tmp/foobar)`

Possible Solution(s)

No response

Notes

I manually hacked ~/.cargo/registry/index/github.com-1ecc6299db9ec823/.cache/li/bc/libc, copied the entry for 0.2.126, called it 0.2.127 and updated the cksum. Now cargo finds 0.2.127. Dunno what is going on here.

The only other thing I noticed is that the index lists the features of 0.2.126 in alphabetical order but does not do so for 0.2.127.

Possibly related (or not): #10623

Version

$ cargo version --verbose
cargo 1.56.0
release: 1.56.0

Edit: Feel free to tell me "do not use such an old version". I just have no idea what is causing this and didn't find anything at all that seemed relevant, so I opened this issue.

[Eh2406]

Looking at https://github.com/rust-lang/crates.io-index/blob/master/li/bc/libc it is not #10623
not sure what's going on here, and it's worrying me.

[weihanglo]

Thanks for posting this issue. Unfortunately, I cannot reproduce on my machine with both 1.56.0 or 1.56.1. I have no idea what happened. Could you backup the whole registry index, then delete it and try again? If that fresh index works, please share with us the old registry index.

[psychon]

Not sure what exactly you meant with "the index", so I went all-in:

$ cd .cargo
$ mv registry registry-backup

Then I did the Steps from the original post. And it worked and picked libc 0.2.132. WTF?

$ cargo build
    Updating crates.io index
  Downloaded libc v0.2.132
  Downloaded 1 crate (595.3 KB) in 1.00s
   Compiling libc v0.2.132
   Compiling foobar v0.1.0 (/tmp/foobar)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 41s

Moving registry-backup back in place, changing the libc dependency and waiting for an index update, I get again:

$ cargo tree
    Updating crates.io index
error: failed to select a version for the requirement `libc = "=0.2.132"`
candidate versions found which didn't match: 0.2.127, 0.2.126, 0.2.125, ...
location searched: crates.io index
required by package `foobar v0.1.0 (/tmp/foobar)`

If that fresh index works, please share with us the old registry index.

Any preference/suggestion on how to share 3.2 GiB of data? Or is there something specific in there that you want?

I took a look around and

• the contents of cache/ do not seem interesting. The new registry only has libc-0.2.132.crate and the old one has lots and lots of other stuff.
• I also doubt that src/ has anything interesting (also just libc-0.2.132)
• index/github..../.cache is also likely not part of the problem since I managed to work around the problem by messing with libcs entry here (see my original post)
• that leaves index/github.../.git. Yay.
• Uhm. The working git repo does have a file refs/remotes/origin/HEAD pointing at the same commit as FETCH_HEAD. The broken one does not have this file.
• Manually copying this file from the working index back to the non-working one makes things work again (meaning that cargo tree works and shows libc 0.2.132`)
• Deleting the file again (and not changing anything else) makes cargo tree update the index again (which always takes forever...) and then it again says that libc 0.2.126 is the newest version available
• In this state and in this directory, git branch shows nothing and git show says that the current branch does not have any commits. How the heck does cargo manage to work and find older libc versions?!?
• Looking at "which fails exists in one version and not the other", I find that the broken index has:

$ cat packed-refs
# pack-refs with: peeled fully-peeled sorted 
c589ce2cc6aa073bacf920eb9d2e804d5df70a99 refs/remotes/origin/HEAD
6516cbef30d5a1130b93875506e4bd5dd63eed91 refs/remotes/origin/master
$ cat info/refs
c589ce2cc6aa073bacf920eb9d2e804d5df70a99	refs/remotes/origin/HEAD
6516cbef30d5a1130b93875506e4bd5dd63eed91	refs/remotes/origin/master

• git show c589ce2cc6aa073bacf920eb9d2e804d5df70a99:li/bc/libc says that the newest version is 0.2.126
• git show 6516cbef30d5a1130b93875506e4bd5dd63eed91:li/bc/libc even says that the newest version is just 0.2.105

So, apparently something did something with something and now cargo / libgit pulls new stuff (which always takes forever) but fails to properly update this git repo and thus is stuck at c589ce2cc6aa073bacf920eb9d2e804d5df70a99?!?

Still, no idea how I managed to get this broken index.

Note that I am using cargo from Debian. Could it be that this is a problem fixed in a newer version of libgit than what Debian provides? Sadly, I now have no idea on "good" reproducing instructions. My wishlist item would be "properly detect such a broken state and either auto-heal or provide some kind of error message".

Edit: With the fresh/new index, deleting refs/remotes/origin/HEAD and running cargo tree just causes the file to be created again. So, more than "just this file is missing" is going on.

[ehuss]

That sounds like #9979.

[psychon]

Indeed. And that was apparently fixed in Cargo 0.59, according to GitHub. I'll assume that would be cargo 1.59.0 on user systems?

So, this issue can be closed and people who insist to use old software will just have to mess around in ~/.cargo after every squash?

Edit: Let's see what Debian does: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1017828

[Eh2406]

Edit: Feel free to tell me "do not use such an old version". I just have no idea what is causing this and didn't find anything at all that seemed relevant, so I opened this issue.

Thank you for opening the issue. I am glad we found the source of the problem. The fix from our end is, as you suggested, "do not use such an old version". If Debian wants to recommend that old version, then they have to be willing to patch it. Thank you for opening the issue with them.