Cargo cannot find renamed dependency from crates-io in package from different registry

[StackOverflowExcept1on]

Problem

Basically I have a crate demo-a which requires the dependency renamed_once_cell = { version = "1.19.0", package = "once_cell" }. The demo-a crate is published to my-registry, then I publish the demo-b crate to my-registry. But for some reason cargo doesn't find renamed_once_cell on crates-io.

demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/demo-a) (my-registry)
└── once_cell v1.19.0 (crates-io)

demo-b v0.1.0 (/mnt/tmpfs/cargo-bug/demo-b) (my-registry)
└── demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/demo-a) (my-registry)
    └── once_cell v1.19.0 (crates-io)

Demo repository: https://github.com/StackOverflowExcept1on/cargo-bug

Steps

git clone [email protected]:StackOverflowExcept1on/cargo-bug.git
cd cargo-bug
cargo install cargo-http-registry
# in the background
rm -rf /tmp/my-registry && cargo-http-registry --addr 127.0.0.1:35503 /tmp/my-registry &
cargo publish -p demo-a && cargo publish -p demo-b
fg
Ctrl+C

error: failed to verify package tarball

Caused by:
  no matching package named `renamed_once_cell` found
  location searched: registry `crates-io`
  required by package `demo-a v0.1.0 (registry `my-registry`)`
      ... which satisfies dependency `demo-a = "^0.1.0"` of package `demo-b v0.1.0 (/mnt/tmpfs/cargo-bug/target/package/demo-b-0.1.0)`

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.80.1 (376290515 2024-07-16)

[StackOverflowExcept1on]

CARGO_LOG=debug cargo publish -p demo-b

bad case, as in demo repository
bad-case.txt

successful case where dependency was not renamed:
successful-case.txt

[arlosi]

I believe this is a duplicate of #14321. The fix is via #14325.

Could you verify with the current beta or nightly (including the publishing step, which is where the problem occurs)?

[StackOverflowExcept1on]

@arlosi I still get this error with cargo 1.82.0-nightly (0d8d22f83 2024-08-08)

[epage]

The bug is in the data Cargo sends to the non-crates.io registry. If its already in the registry, upgrading won't fix it. The package will need to be re-published.

[StackOverflowExcept1on]

@epage I always run an empty registry and even with it it reproduces on 0d8d22f.

[epage]

The index entry from the first publish

{
  "name": "demo-a",
  "vers": "0.1.0",
  "deps": [
    {
      "name": "once_cell",
      "req": "^1.19.0",
      "features": [],
      "optional": false,
      "default_features": true,
      "target": null,
      "kind": "normal",
      "registry": "https://github.com/rust-lang/crates.io-index",
      "package": "renamed_once_cell"
    }
  ],
  "cksum": "f82c53b535363edb813ae5fd3413a4b392fea12f96ac401c0d2c90d200d92706",
  "features": {},
  "yanked": false,
  "links": null
}

According to the docs

{
    "deps": [
        {
            // Name of the dependency.
            // If the dependency is renamed from the original package name,
            // this is the new name. The original package name is stored in
            // the `package` field.
            "name": "rand",
            // If the dependency is renamed, this is a string of the actual
            // package name. If not specified or null, this dependency is not
            // renamed.
            "package": null,
        }
    ],
}

from https://github.com/d-e-s-o/cargo-http-registry/blob/651d22e254a0f7712e5eba9592ff68e69d1a85c9/src/publish.rs#L84C1-L98C2

impl From<Dep> for crate::index::Dep {
  fn from(source: Dep) -> Self {
    Self {
      name: source.name,
      req: source.version_req,
      features: source.features,
      optional: source.optional,
      default_features: source.default_features,
      target: source.target,
      kind: source.kind.to_string(),
      registry: source.registry,
      package: source.explicit_name_in_toml,
    }
  }
}

From https://doc.rust-lang.org/cargo/reference/registry-web-api.html#publish

{
    // The name of the package.
    "name": "foo",
    // The version of the package being published.
    "vers": "0.1.0",
    // Array of direct dependencies of the package.
    "deps": [
        {
            // Name of the dependency.
            // If the dependency is renamed from the original package name,
            // this is the original name. The new package name is stored in
            // the `explicit_name_in_toml` field.
            "name": "rand",
            // If the dependency is renamed, this is a string of the new
            // package name. If not specified or null, this dependency is not
            // renamed.
            "explicit_name_in_toml": null,
        }
    ],
}

From https://doc.rust-lang.org/cargo/reference/registry-index.html

name — When the dependency is renamed in Cargo.toml, the publish API puts the original package name in the name field and the aliased name in the explicit_name_in_toml field. The index places the aliased name in the name field, and the original package name in the package field.

[epage]

From the above, this looks to be a bug in https://github.com/d-e-s-o/cargo-http-registry

Of course, this would be less likely to happen if we had the types and the type conversions published in cargo-util-schemas

[arlosi]

Yes, this is a bug in cargo-http-registry

https://github.com/d-e-s-o/cargo-http-registry/blob/main/src/publish.rs#L95 directly sets the package field in the index to explicit_name_in_toml, which is incorrect. One is the original name, the other is the renamed-name.

explicit_name_in_toml docs:

  /// Name of the dependency. If the dependency is renamed from the
  /// original package name, this is the original name. The new package
  /// name is stored in the `explicit_name_in_toml` field.

package docs:

 // If the dependency is renamed, this is a string of the actual
 // package name. If not specified or null, this dependency is not
  // renamed.

[StackOverflowExcept1on]

So I should just swap source.name and source.explicit_name_in_toml in cargo-http-registry?

[arlosi]

So I should just swap source.name and source.explicit_name_in_toml in cargo-http-registry?

Only if the dependency is renamed. You can see how Cargo does it in our mock registry for tests:

cargo/crates/cargo-test-support/src/registry.rs

Lines 1198 to 1201 in 3a20ea7

	let (name, package) = match &dep.explicit_name_in_toml {
	Some(explicit) => (explicit.to_string(), Some(dep.name.to_string())),
	None => (dep.name.to_string(), None),
	};

[StackOverflowExcept1on]

@arlosi I ran into another problem. You suggested using cargo-util-schemas, but now I see that my problem is not with the renamed dependency, but with the missing features2 key (when using dep:renamed-feature). However, I couldn't find features2 in either the documentation https://doc.rust-lang.org/cargo/reference/registry-web-api.html#publish or the cargo-util-schemas crate.

[ehuss]

features2 is only required by crates.io. Other registries should not bother with it. It is only for backwards compatibility. It also only appears in the index, not the publish API. This is documented in https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema.

[StackOverflowExcept1on]

@ehuss After fixing behavior of cargo-http-registry I still have problem like #14148, but now with dep:renamed_once_cell. Now publishing crate demo-a does not happen after 60 seconds.

[dependencies]
renamed_once_cell = { workspace = true, optional = true }

[features]
renamed_once_cell = ["dep:renamed_once_cell"]

Demo repository: https://github.com/StackOverflowExcept1on/cargo-bug

git clone [email protected]:StackOverflowExcept1on/cargo-bug.git
cd cargo-bug

cargo install --git https://github.com/d-e-s-o/cargo-http-registry.git cargo-http-registry
# in the background
rm -rf /tmp/my-registry && cargo-http-registry --addr 127.0.0.1:35503 /tmp/my-registry &
cargo publish -p demo-a && cargo publish -p demo-b
fg
Ctrl+C

    Updating `my-registry` index
From http://127.0.0.1:35503/git
 + b5ef96d...8276462            -> origin/HEAD  (force update)
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
   Packaging demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/demo-a)
   Verifying demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/demo-a)
    Updating crates.io index
   Compiling demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/target/package/demo-a-0.1.0)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.65s
    Packaged 4 files, 1.2KiB (823.0B compressed)
   Uploading demo-a v0.1.0 (/mnt/tmpfs/cargo-bug/demo-a)
    Uploaded demo-a v0.1.0 to registry `my-registry`
note: waiting for `demo-a v0.1.0` to be available at registry `my-registry`.
You may press ctrl-c to skip waiting; the crate should be available shortly.
From http://127.0.0.1:35503/git             ] 0/60
   8276462..b542209             -> origin/HEAD
warning: timed out waiting for `demo-a v0.1.0` to be available in registry `my-registry`
note: the registry may have a backlog that is delaying making the crate available. The crate should be available soon.
    Updating `my-registry` index
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
   Packaging demo-b v0.1.0 (/mnt/tmpfs/cargo-bug/demo-b)
   Verifying demo-b v0.1.0 (/mnt/tmpfs/cargo-bug/demo-b)
error: failed to verify package tarball

Caused by:
  no matching package named `demo-a` found
  location searched: registry `my-registry`
  required by package `demo-b v0.1.0 (/mnt/tmpfs/cargo-bug/target/package/demo-b-0.1.0)`

/tmp/my-registry/de/mo/demo-a

{
  "name": "demo-a",
  "vers": "0.1.0",
  "deps": [
    {
      "name": "renamed_once_cell",
      "req": "^1.19.0",
      "features": [],
      "optional": true,
      "default_features": true,
      "target": null,
      "kind": "normal",
      "registry": "https://github.com/rust-lang/crates.io-index",
      "package": "once_cell"
    }
  ],
  "cksum": "2bd501914462eefbe358bc3cdb5f59cb0bcb04740fff4358397c7e2e2bf84019",
  "features": {
    "renamed_once_cell": []
  },
  "yanked": false,
  "links": null
}