Python venv not activating correctly depending on shims

[nicksellen]

Describe the bug

I'm using mise in https://codeberg.org/karrot/karrot-backend with this mise.toml.

After a recent upgrade (2024.12.2 or higher), my python environment couldn't load any of the libraries, it was using the wrong python, not the venv.

I found it was related to having this "IDE integration" line (which is what it used to recommend in the docs) in my bashrc:

export PATH="$HOME/.local/share/mise/shims:$PATH"

With 2024.12.1 (the first part of) my path would get set correctly to:

/home/nick/Projects/karrot/karrot-backend/.venv/bin:/home/nick/.local/share/mise/installs/python/3.12.2/bin

In 2024.12.2 and later, it gets set to this instead:

/home/nick/.local/share/mise/shims:/home/nick/Projects/karrot/karrot-backend/.venv/bin:/home/nick/.local/share/mise/installs/python/3.12.2/bin

The presence of /home/nick/.local/share/mise/shims at the start, prevents it using the correct python environment.

If I switch to my shims enabling to the current approach in the docs:

eval "$(mise activate bash --shims)"

... then it doesn't activate the venv at all.

trace log

TRACE  1 [src/toolset/mod.rs:568] env end
TRACE  1 [src/toolset/mod.rs:540] env start
TRACE  1 [src/backend/mod.rs:255] python is not installed, path: ~/.local/share/mise/installs/python/3.12 (runtime symlink)
TRACE  1 [src/backend/mod.rs:255] node is not installed, path: ~/.local/share/mise/installs/node/20 (runtime symlink)
TRACE  1 [src/toolset/mod.rs:568] env end
TRACE  1 [src/backend/mod.rs:255] python is not installed, path: ~/.local/share/mise/installs/python/3.12 (runtime symlink)
TRACE  1 [src/backend/mod.rs:255] node is not installed, path: ~/.local/share/mise/installs/node/20 (runtime symlink)
TRACE  1 [src/cli/exec.rs:83] env_with_path done
TRACE  1 [src/cli/exec.rs:104] exec

Running .venv/bin/activate manually activates it correctly.

To Reproduce

Install mise version 2024.12.2 or higher, deactivate all existing mise environment, then:

mkdir ~/mise-karrot-repro
cd ~/mise-karrot-repro
eval "$(mise activate bash)"
git clone https://codeberg.org/karrot/karrot-backend.git
cd karrot-backend/
mise trust
python -m venv ~/mise-karrot-repro/karrot-backend/.venv

# trigger it to activate the venv
cd ..
cd karrot-backend/

# this path is good!
echo $PATH

# now try it with shims already set
cd ..
export PATH="$HOME/.local/share/mise/shims:$PATH"
cd karrot-backend/

# uh-oh, this now has the shims at the start, with the venv path after it
echo $PATH

# alternatively, if activating with `--shims` argument above, the venv isn't activated at all

Expected behavior

The venv path should always be before shims path.

mise doctor output

As I'm running it in various contexts, I see all combinations of activated and shims_on_path.

version: 2024.12.5 linux-x64 (41dd42f 2024-12-10)
activated: no
shims_on_path: yes

build_info:
  Target: x86_64-unknown-linux-gnu
  Features: DEFAULT, NATIVE_TLS, OPENSSL
  Built: Tue, 10 Dec 2024 18:46:55 +0000
  Rust Version: rustc 1.83.0 (90b35a623 2024-11-26)
  Profile: release

shell:
  /bin/bash
  GNU bash, version 5.2.37(1)-release (x86_64-pc-linux-gnu)
  Copyright (C) 2022 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

  This is free software; you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

dirs:
  cache: ~/.cache/mise
  config: ~/.config/mise
  data: ~/.local/share/mise
  shims: ~/.local/share/mise/shims
  state: ~/.local/state/mise

config_files:
  ~/.config/mise/config.toml
  ~/mise-karrot-repro/karrot-backend/.mise.toml

backends:
  aqua
  asdf
  cargo
  core
  gem
  go
  npm
  pipx
  spm
  ubi
  vfox

plugins:
  dotnet       https://github.com/hensou/asdf-dotnet#e8f059b
  dotnet-core  https://github.com/emersonsoares/asdf-dotnet-core.git#4583e4b
  gradle       https://github.com/rfrancis/asdf-gradle.git#9a2ca80
  php          https://github.com/asdf-community/asdf-php.git#1eaf4de
  poetry       https://github.com/rtx-plugins/rtx-poetry.git#3d4c703

toolset:
  asdf:[email protected]
  core:[email protected]
  core:[email protected]
  ubi:jdx/[email protected]

env_vars:
  (none)

settings:
  experimental  true ~/.config/mise/config.toml

No problems found

Additional context

My ~/.config/mise/config.toml has this in it:

[settings]
experimental = true

[tools]
poetry = "1.6.1"
python = "system"
usage = "latest"

It looks related to the change here #3396 - which is doing something with the shims, and was added in 2024.12.2.

I'm going to try only activating using:

eval "$(mise activate bash)"

... and try out this intellij plugin for mise https://plugins.jetbrains.com/plugin/24904-mise

[syhol]

This is often a matter of ordering.

eval "$(mise activate bash)"
...
# now try it with shims already set
cd ..
export PATH="$HOME/.local/share/mise/shims:$PATH"

After mise activate, anything added to the start of the path will never be overridden by mise. If you flip the order it might work how you expect.

I think the current recommendation for bash is to put

eval "$(mise activate bash --shims)"
eval "$(mise activate bash)"

in your .bashrc.

Also fyi, all mise activate --shims does is export PATH="$HOME/.local/share/mise/shims:$PATH"

[jdx]

yeah you definitely shouldn't add shims after activating—that doesn't make sense

[nicksellen]

Ah, brilliant thanks! So, maybe two points from that:

1. could consider always printing shims are on PATH and mise is also activated. You should only use one of these methods. message (that brew doctor outputs) as a warning - that would highlight the ordering problem I had
2. I had assumed the --shims argument to mise activate bash would activate it and add the shims, maybe confusing?

[jdx]

could consider always printing shims are on PATH and mise is also activated. You should only use one of these methods. message (that brew doctor outputs) as a warning - that would highlight the ordering problem I had

I'm hesitant to put warnings like this in, I'm sure there is someone out there that has some reason for wanting to do things this way even though it generally isn't what you should do. mise dr is a better place to show this information.

I had assumed the --shims argument to mise activate bash would activate it and add the shims, maybe confusing?

I think the docs are pretty clear https://mise.jdx.dev/cli/activate#shims

[nicksellen]

When only using mise activate --shims the venv is not activated - is that expected?

I think https://mise.jdx.dev/dev-tools/shims.html#how-to-add-mise-shims-to-path is useful to link to from the --shims docs page.

So, in summary, I need it like this:

eval "$(mise activate bash --shims)"
eval "$(mise activate bash)"

Having them in the other order worked (for me) before 2024.12.2.

[syhol]

When only using mise activate --shims the venv is not activated - is that expected?

Yes, shims just add some executables to path, so it will never automatically change your path or activate environments as you change directory.

So, in summary, I need it like this:

eval "$(mise activate bash --shims)"
eval "$(mise activate bash)"

Yes. Doing it the other way around was unspecified behaviour and never supported. The docs are much better today, so no one else should get into that situation.