Skip to content

Commit

Permalink
update proposal
Browse files Browse the repository at this point in the history
  • Loading branch information
pavelzw committed Mar 11, 2024
1 parent 873cfff commit da60acc
Showing 1 changed file with 83 additions and 44 deletions.
127 changes: 83 additions & 44 deletions docs/design_proposals/channel_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ It has the following structure:

```toml
default_channels = ["bioconda", "conda-forge"]
default_mirrors = ["https://my.custom.mirror/{channel}"]
default_url = "https://my.custom.mirror/{channel}"

[[channels]]
name = "conda-forge"
mirrors = [
url = [
"https://conda.anaconda.org/conda-forge",
"https://repo.prefix.dev/conda-forge"
]
Expand All @@ -36,30 +36,25 @@ url = "https://repo.prefix.dev/prefix-internal"

[[channels]]
name = "bioconda"
mirrors = [
"https://repo.prefix.dev/bioconda"
]

[[channels]]
name = "my-private-channel"
url = "https://my.private.quetz/get/my-private-channel"
mirrors = [
"https://repo.artifactory.com/conda-mirror-my-private-channel"
]
url = "https://repo.prefix.dev/bioconda"
```

!!!note "Configure in `pixi.toml`"
All keys specified in `rattler.toml` can also be specified in `pixi.toml`.
`rattler.toml` takes precedence over `pixi.toml`.

## Use cases

### Use the same mirror everywhere

If you want to use the same mirror for all channels, you can set the `default_mirrors` key in the configuration file.
If you want to use the same mirror for all channels, you can set the `default_url` key in the configuration file.

```toml title="rattler.toml"
default_mirrors = ["https://my.custom.mirror/{channel}"]
default_url = "https://my.custom.mirror/{channel}"

[[channels]]
name = "bioconda"
mirrors = ["https://repo.prefix.dev/bioconda"]
url = "https://repo.prefix.dev/bioconda"
```

This results in all channels being redirected to this mirror.
Expand All @@ -75,15 +70,20 @@ channels = [
...
```

Using this `pixi.toml` with the above `rattler.toml` will result in the following behavior:

- `conda-forge` will be redirected to `https://my.custom.mirror/conda-forge`
- `bioconda` will be redirected to `https://repo.prefix.dev/bioconda` since it has higher priority than `default_mirrors`
- `https://conda.anaconda.org/cf-staging` and `https://repo.prefix.dev/prefix-internal` will not being redirected since they specify a full URL

!!!note "URLs in `channels`"
If you specify a full URL in `channels`, it will never be changed by `rattler.toml`. Only named channels will be redirected.

!!!tip "Use `prefix.dev` mirrors"
If you want to use the mirrors hosted on [prefix.dev](https://prefix.dev), you can use the following configuration:

```toml title="rattler.toml"
default_mirrors = ["https://repo.prefix.dev/{channel}"]
default_url = "https://repo.prefix.dev/{channel}"
```

### Default channels
Expand All @@ -96,15 +96,40 @@ name = "my-private-channel"
url = "https://my.private.quetz/get/my-private-channel"
```

This will result in the channels `bioconda` and `conda-forge` and `my-private-channel` being used when you run `pixi init` or `pixi global install`.
These channels will also be used by `rattler-build` if you don't specify other channels with `-c`.
This will result in the channels `bioconda` and `conda-forge` and `my-private-channel` being used when you run `pixi init` or `pixi global install` and `rattler-build` (assuming you don't override the channels using `-c`).

`pixi init` would create the following `pixi.toml`:

```toml title="pixi.toml"
# ...
channels = [
"my-private-channel",
"bioconda",
"conda-forge"
]
# ...

[[channels]]
name = "my-private-channel"
url = "https://my.private.quetz/get/my-private-channel"
```

`pixi init --ignore-rattler-config` would create the following `pixi.toml`:

```toml title="pixi.toml"
# ...
channels = ["conda-forge"]
# ...
```

### Multiple mirrors

Instead of providing a single mirror, you can also provide a list of mirrors.

```toml title="rattler.toml"
[[channels]]
name = "conda-forge"
mirrors = [
url = [
"https://conda.anaconda.org/conda-forge",
{url = "https://repo.artifactory.com/conda-mirror-conda-forge", use_zstd = false},
"oci://ghcr.io/conda-channel-mirrors/conda-forge"
Expand All @@ -120,49 +145,65 @@ You can specify whether `pixi` and `rattler-build` should use the `repodata.json
This is needed for some proxies like older versions of artifactory ([RTFACT-29886](https://jfrog.atlassian.net/jira/software/c/projects/RTFACT/issues/RTFACT-29886)).

```toml title="rattler.toml"
default_mirrors = [
{"https://my.custom.mirror/{channel}", use_bz2 = false, use_zstd = false}
]

[[channels]]
name = "conda-forge"
mirrors = [
{url = "https://repo.artifactory.com/conda-mirror-conda-forge", use_zstd = false}
]
default_url = {
url = "https://my.custom.mirror/{channel}",
use_bz2 = false,
use_zstd = false
}

[[channels]]
name = "bioconda"
use_zstd = false
mirrors = [
"https://repo.prefix.dev/bioconda"
]
url = "https://repo.prefix.dev/bioconda"
```

### Use private channels

```toml title="rattler.toml"
default_channels = ["conda-forge", "private-channel"]
You can specify private channels in your `pixi.toml` and route them to a different URL.

```toml title="pixi.toml"
channels = ["conda-forge", "bioconda", "private-channel"]

[[channels]] # (1)!
name = "bioconda"
url = "https://conda.anaconda.org/bioconda"

[[channels]]
name = "private-channel"
url = "https://my.private.quetz/get/my-private-channel"
```

1. This is the default behavior

This will result in `private-channel` being redirected to `https://my.private.quetz/get/my-private-channel`.
If you want to use a mirror for this private channel, you can additionally specify `mirrors` for this channel.
If you want to use a mirror for this private channel, you can override the channel URL in `rattler.toml` since `rattler.toml` takes precedence over `pixi.toml`.

```toml title="rattler.toml"
default_channels = ["conda-forge", "private-channel"]

[[channels]]
name = "private-channel"
url = "https://my.private.quetz/get/my-private-channel"
mirrors = [
"https://repo.artifactory.com/conda-mirror-my-private-channel"
]
url = "https://repo.artifactory.com/conda-mirror-my-private-channel"
```

This will result in `private-channel` being redirected to `https://repo.artifactory.com/conda-mirror-my-private-channel`, but `https://my.private.quetz/get/my-private-channel` is still used in the `pixi.lock` file. Thus, the canonical URL is still `https://my.private.quetz/get/my-private-channel`.
This will result in `private-channel` being redirected to `https://repo.artifactory.com/conda-mirror-my-private-channel`, but `https://my.private.quetz/get/my-private-channel` is still used in the `pixi.lock` file.
Thus, the canonical URL is still `https://my.private.quetz/get/my-private-channel`.

The `pixi.lock` file will persist the channels defined in `pixi.toml`:

```yaml title="pixi.lock"
channels:
- name: conda-forge
url: https://conda.anaconda.org/conda-forge
- name: my-channel
url: https://repo.prefix.dev/my-channel
environments:
default:
channels:
- conda-forge
packages:
linux-64:
- conda: https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2
# ...
```

### OCI mirrors

Expand All @@ -171,7 +212,5 @@ You can also use [OCI](https://opencontainers.org/) mirrors.
```toml title="rattler.toml"
[[channels]]
name = "conda-forge"
mirrors = [
"oci://ghcr.io/conda-channel-mirrors/conda-forge"
]
url = "oci://ghcr.io/conda-channel-mirrors/conda-forge"
```

0 comments on commit da60acc

Please sign in to comment.