Configure interpreter based on python environment

[albas99]

Discussed in https://github.com/zed-industries/community/discussions/1037

^{Originally posted by albas99 March 10, 2023}
I'm using Zed along with the Warp terminal app, and even though I've configured my virtual environment using Poetry, Zed is still flagging my code and doesn't seem to be recognizing the interpreter. How can I configure it so it can recognize it?

UPDATE: I've managed to configure pyright in my pyproject.toml file but it's still not working. I'm still getting import django.contrib not resolved and I have django installed in the poetry environment.

[fdionisi]

Hey @albas99, I'm using Zed with Python on a Poetry project with no significant issues. Here are the steps for my setup:

# Create a virtual environment at the root of my project.
# Note: you may want to add the `.venv/` folder to `.gitignore`
python3 -m venv .venv

# Tell Poetry to use .venv
## Note: I use https://direnv.net/ and have the following command in my .envrc file,
## so every time I spin up a new terminal, my virtual environment is ready.
poetry env use .venv/bin/python3.11

# Configure Pyright (necessary for Zed)
## https://docs.zed.dev/languages/python
echo '{
  "venvPath": ".",
  "venv": ".venv"
}' > pyrightconfig.json

From here on, everything should work as expected, both in Zed and in my terminals.
Sometimes, Python LSP is slow to keep up with changes in the work tree or configurations, so you may want to:

1. Focus a Python buffer in the editor
2. Open the command palette (cmd-shift-p)
3. Run editor: restart language server

The one above I think it may be considered a bug, but I haven't yet found the time to write down some steps to reproduce it. 😬

I hope this helps.

[gcarrarom]

Similarly I do that with Poetry and nushell in my projects:

{"venv": (poetry env info --path | split row "/" | last), "venvPath": (poetry env info --path | split row "/" | drop | str join "/")} | to json | save pyrightconfig.json

[403-html]

this helped me, it should be by default inside zed, as zed has pyright, not my projects

[albas99]

Hi @fdionisi, thanks for the help😁, the thing is I want to use poetry's virtual environment, due to how it manages dependencies. I'm not familiar with venv(I normally use conda, and now poetry).

[jansol]

Seems like being able to launch language servers in a specific environment is a much more common need than just among Nix users. Would be nice to see some progress towards that.

[fdionisi]

Hey @albas99, after a bit more digging, I found an interesting solution that can work with Poetry, Pyright and Zed, similarly to what I mentioned in my original response message.

By default, Poetry creates virtual environments in the cache directory (e.g. ~/Library/Caches/pypoetry/virtualenvs/), but you can configure it such that venvs are created directly in your project's folder. You can run poetry virtualenvs.in-project true¹ and then set up python as I suggested above.

So the outcome would be:

# Tell Poetry to create virtualenvs directly in the project.
poetry virtualenvs.in-project true

# Create a virtual environment at the root of my project.
# Note: you may want to add the `.venv/` folder to `.gitignore`
poetry env use 3.11

# Configure Pyright (necessary for Zed)
## https://docs.zed.dev/languages/python
echo '{
  "venvPath": ".",
  "venv": ".venv"
}' > pyrightconfig.json

At this point, I can add Django (poetry add django) and import it into my Python files and it shows up.
I hope this can be a good solution for your use case. Personally, I'm pretty happy to have jumped on this; I'm still learning about Poetry and how to get the best out of it, and this setup looks great and it works with Zed. So, thank you. 😆

Footnotes

1. https://python-poetry.org/docs/configuration/#virtualenvsin-project ↩

[albas99]

Thanks a lot for the help @fdionisi , I've configured poetry to create virtualenvs in my project folder in the future, now it seems to recognized my imports, I don't know if it has something to do with the change or it finally recognized my configuration file.

[gregorypease280]

I was making a little script for this. It does the job but I'm still tweaking it.
https://gist.github.com/gregorypease280/1ee8af205021ab1322dcf9cce2fc0124

[mpassell]

@fdionisi Did you mean to put poetry config virtualenvs.in-project true in the first line above? The config part seemed to be necessary for me.

[mikelane]

You don't need to change anything about your project settings to get pyright to find your Poetry virtualenv. All you need to do is to run poetry env info. The virtualenv that is in use will be displayed in the results. For example:

$ poetry env info

Virtualenv
Python:         3.11.1
Implementation: CPython
Path:           /Users/mikelane/Library/Caches/pypoetry/virtualenvs/reddit-get-40JF5hkq-py3.11
Executable:     /Users/mikelane/Library/Caches/pypoetry/virtualenvs/reddit-get-40JF5hkq-py3.11/bin/python
Valid:          True

System
Platform:   darwin
OS:         posix
Python:     3.11.1
Path:       /Users/mikelane/.pyenv/versions/3.11.1
Executable: /Users/mikelane/.pyenv/versions/3.11.1/bin/python3.11

In my case, all I had to do was to add this section to my pyproject.toml

[tool.pyright]
venvPath = "/Users/mikelane/Library/Caches/pypoetry/virtualenvs"
venv = "reddit-get-40JF5hkq-py3.11"

I did have to restart zed to get the squiggles to go away.

I should look at the source code to see if I can add that. Or maybe is there a plugin system for zed? I'm very new to this editor, so bear with me 😆

[stuaxo]

Specifying a particualr virtualenv path looks like it might work, but isn't really viable if you need to commit your pyproject.toml and share with other devs.

[mikelane]

Specifying a particualr virtualenv path looks like it might work, but isn't really viable if you need to commit your pyproject.toml and share with other devs.

I don't understand why this is the case. Can you help me understand?

[Aeron]

Because other people—who work with the repo and get the pyproject.toml file—probably have no /Users/mikelane/Library/Caches/pypoetry/virtualenvs directory, and their virtual environment may have a different name. It could be a different OS entirely. It’d be silly to assume that everybody uses the same everything, even if paths are relative.

If you are the sole developer, then yes, it’s viable.

[mikelane]

Because other people—who work with the repo and get the pyproject.toml file—probably have no /Users/mikelane/Library/Caches/pypoetry/virtualenvs directory, and their virtual environment may have a different name. It could be a different OS entirely. It’d be silly to assume that everybody uses the same everything, even if paths are relative.

If you are the sole developer, then yes, it’s viable.

Just set poetry to have the virtualenv in the repo in that case.

https://python-poetry.org/docs/configuration/#virtualenvsin-project

That way the zed config can just be this

venvPath = "."
venv = ".venv"

Problem solved, commit and clone at will.

[stuaxo]

That is still setting a particular development style.

I don't keep a virtualenv in my repo - the reason is, that I have multiple work projects and 10s of non work projects on my laptops.

A long time ago I used this system, but it had downsides:

• my disk was full of stale virtualenvs (and it was a pain to find them when I wanted to free space)
• it's easy to misconfigure things and commit the virtualenv itself to the repo (which is bad - they really only work per-machine, it's not cross-platform).
• one virtualenv per project is a bit limiting (you may want to check out a branch where you try some other dependency).

For myself, I use virtualenvwrapper which keeps all my virtualenvs in ~/.virtualenvs this solves all of the above and gives me one tool to manage any of my virtualenvs and create / delete new ones as needed.

[JosephTLyons]

This feels like it should be converted into a Discussion. Lots of good information here.

[JosephTLyons]

Oof, missed the part where this referenced an existing Discussion ... oops.

[mpassell]

True, but there's a whole lot more in here.

[danielcs88]

Is anyone working with Conda on Zed?

[danielcs88]

I was able to use Conda by creating a file in the directory I'm working by creating a file called pyrightconfig.json

{
  "executionEnvironments": [
    {
      "python": "<path_to_conda_env>/bin/python"
    }
  ]
}

What I don't know yet is how to configure for all Zed instances rather than having to create a pyrightconfig.json in every project directory.

[jansol]

Having a pyrightconfig.json or pyproject.toml per project is the way the python project tooling is supposed to be used though? Frequently you'd also want to have a project-specific virtualenv too, in order to avoid conflicts between dependencies of different projects so pointing the executionEnvironments to a subdirectory of the project makes sense too (and means it's not necessarily machine-specific).

Granted, this is rather frustrating for "simple" helper/utility scripts or throwaway code that does not warrant creating an entire project. As a workaround I guess you could just have a "project" folder where you place all such scripts. THe remaining problem would be python utility scripts in non-python projects....

[RhetTbull]

Has anyone gotten pyright working with pyenv? I used pyenv-pyright to create the pyrightconfig.json:

{
   "venv" : "appkitgui",
   "venvPath" : "/Users/user/.pyenv/versions"
}

But no luck when restarting zed and of course, no helpful error messages about not finding the venv. I really want to give zed a try but it's lack of support for python is a deal breaker. I wish they could copy whatever VSCode is doing to make python environments "just work". VSCode has no problem finding the venv whether I use venv, poetry, or pyenv (and no pyrightconfig.json is required).

[KiraMelnikov]

Hi !
Did u find some solves about custom .env ?
I have no idea too, how to resolve it in my Zed

[scorphus]

Also works:

{
  "venv": "myvenv",
  "venvPath": "../../.pyenv/versions"
}

Adapt ../../ to the relative path from the root of your project.

[RhetTbull]

This issue is what is keeping me from using zed. I have tried using it several times but the use of pyright as LSP (which frankly, is very lacking) and having to add a file to every single project just so pyright can find the venv makes the python experience with zed painful. VSCode has found a way to "just work" regardless of whether you're using venv, virtualenv, poetry, or pyenv. I wish zed would figure out what VSCode is doing and copy it. VSCode also allows the user to select a different venv in the editor.

[KiraMelnikov]

It's really a pity
I work exclusively with Python and different environments
And in Zed, it became a blocker for me too, unfortunately =(
So I'll have to stay in VS Code for now until Zed has a comfortable interaction with the Python environments

Thank you

[Pixel-Minions]

I agree, also, a pythonist here, the Python support is still a bit early it seems, and sadly, it is difficult to use at work. But love the project, wish I new rust to help.

[Aeron]

According to the documentation, it should be enough to specify the python.pythonPath setting for Pyright. In Zed’s config, it would look like so:

{
  "lsp": {
    "pyright": {
      "initialization_options": {
        "python": {
          "pythonPath": ".venv/bin/python"
        }
      }
    }
  }
}

For those who have a single name for the virtual environment directory across all projects, it’s a pretty good solution. Yet it doesn’t work. And it doesn’t matter if pyproject.toml and pyrightconfig.json exist or have relevant settings. Specifying a full path doesn’t help as well.

I’m not a big fan of having excessive pyrightconfig.json files lying around. Getting the venv and venvPath keys into the pyproject.toml is (kind of) ridiculous since every repo user usually gets it, and not everyone has the same virtual environment path or even tools to manage it. Those settings are not project-specific things.

A brief and quick glance at the code yielded no clues yet. It probably deserves a separate issue, so I’ll dig further when I have attention to spare.

[Aeron]

It’s probably worth mentioning that—at the moment—it works as it should and has a proper description in the Python-related documentation. The difference is that the Pyright preferences key is not the initialization_options but the settings one. So, no need to have pyrightconfig.json or pollute pyproject.toml files.

[dgmulf]

It’s probably worth mentioning that—at the moment—it works as it should and has a proper description in the Python-related documentation. The difference is that the Pyright preferences key is not the initialization_options but the settings one. So, no need to have pyrightconfig.json or pollute pyproject.toml files.

This approach is not working for me as of 0.151.1 on macOS, while pyproject.toml does. However as mentioned, a local pyproject.toml interferes needlessly with other tools, so it's less preferable than a Zed-specific configuration.

[Aeron]

In my case, the settings look like the following:

{
  "lsp": {
    "pyright": {
      "settings": {
        "python": {
          "pythonPath": ".venv/bin/python"
        },
        "python.analysis": {
          "typeCheckingMode": "off"
        },
        "pyright": {
          "disableOrganizeImports": true
        }
      }
    }
  }
}

There is no relevant configuration in the pyproject.toml files. No pyrightconfig.json as well.

My projects always have a simple (as of python -m venv or uv venv) virtual environment in the project’s root. I also use mypy, so type checking is off. And ruff handles all the imports. So, I need Pyright for hints, suggestions, etc.

I have the same version, and it works fine. Not sure what can be wrong here.

[dgmulf]

I discovered the issue: I was trying to use a project-specific LSP config in .zed/settings.json, whereas it looks like that functionality is only supported in the global user config.

Decent temporary workaround: symlink the correct project-specific Python to .zed/python, and configure Zed globally to use that relative path.

[rnanclares]

In case anybody is interested with miniforge/anaconda envs the right way to configure them is like this:

{
  "venv": "envname",
  "venvPath": "/home/user/miniforge3/envs"
}

[danielcs88]

On what file do you specify this configuration?

[rnanclares]

In the pyrightconfig.json

[stuaxo]

[Ignore the next text if pyrightconfig.json is not in the project directory]

That looks like it lives in the projects root directory, if so that is back to one-virtualenv-per-project, which doesn't work for many developers (see reasoning higher up in this thread).

It doesn't work if the user wants to commit pyrightconfig.json (different users may have different python versions, including across different machines / operating systems).

It's great for a default, but what many users want is some way of choosing an environment for this editor in this project, that doesn't interfere with other users.

Pycharm has it's .idea directory, that is in many peoples .gitignore a projects virtualenv should similarly be in some file that people add to .gitignore

pyrightconfig.json has all sorts of defaults users may want to commit to a project, the virtualenv should not be specified here - imagine Wayne on Windows commits it and his virtualenv is D:\venvs\super-project Linus on Linux keeps his in /home/linus/dev/virtualenvs and now it's broken, Marcy on Mac is annoyed because she keeps hers in /Users/Marcy/.virtualenvs/some-project and sometimes she likes to test things in a fresh virtualenv for prod /Users/Marcy/.virtualenvs/some-prod

[albas99]

This feature has now been implemented and merged in #19576 and will hopefully be released very soon. Shout out to @osiewicz for implementing this feature.

[sn1c]

@osiewicz Great work on the toolchain feature. It is able to find all my poetry environments and let me easily switch between the different ones. Two things are missing for me though:

1. As @yaojia25 mentioned, it does not seem to work with basedpyright.
2. This is more nice to have: It'd be cool if the Python environment is automatically enabled in the terminal.

[sn1c]

Regarding 2: I'm not the most well-versed in Rust and this is my first PR in this repo but I tried my luck implementing it: #21388

[osiewicz]

@elisiariocouto @gryevns are you still having issues with poetry? I've landed a bunch of improvements around that a while back.

@sn1c @yaojia25 Yup, extensions can't access selected toolchain yet, so sadly any extension that depends on it has to wait until that API becomes available. I don't have capacity for that at the moment.

[elisiariocouto]

I'm using Zed Preview now and I don't have issues at the moment. But I can't fully confirm since I used to use the default virtualenv path from Poetry but before installing Zed Preview I switched to in-project virtualenvs.

[gryevns]

I am now on version 0.163.3 and after a restart it is selecting & showing me poetry environments (outside of project) - thanks!