hnswlib dependency issue

[HarshalRathore]

Which package caused the issue?
hnswlib

What was the issue
I think it requires Clang with version 14> which can be installed with installing Microsoft Visual Studio.

Software/OS/platform with versions used

Windows 10 Home
Python 3.10.9
pip 23.0.1

Complete Error

Building wheels for collected packages: hnswlib, sentence-transformers, validators
Building wheel for hnswlib (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for hnswlib (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [5 lines of output]
   running bdist_wheel
   running build
   running build_ext
   building 'hnswlib' extension
   error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
   [end of output]

note: This error originates from a subprocess and is likely not a problem with pip.
ERROR: Failed building wheel for hnswlib
Building wheel for sentence-transformers (setup.py) ... done
Created wheel for sentence-transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125961 sha256=e57909d0f8487ab4d6ed17e8230321d2419f34de1f3347d6ddfa1ed8ab8b54a8
Stored in directory: c:\users\harshal rathore\appdata\local\pip\cache\wheels\62\f2\10\1e606fd5f02395388f74e7462910fe851042f97238cbbd902f
Building wheel for validators (setup.py) ... done
Created wheel for validators: filename=validators-0.20.0-py3-none-any.whl size=19590 sha256=0e8482e82775ea9a61821e0b79d086368ce3ab7d0420a01e01440c641ffc1ead
Stored in directory: c:\users\harshal rathore\appdata\local\pip\cache\wheels\f2\ed\dd\d3a556ad245ef9dc570c6bcd2f22886d17b0b408dd3bbb9ac3
Successfully built sentence-transformers validators
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

[shihaozhangjohn]

Running on Mac and tried everything i could ,.. failed

[HarshalRathore]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform

@nicknochnack can you please resolve these issues.

[shihaozhangjohn]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform

@nicknochnack can you please resolve these issues.

yeah i deleted that one from requirement txt, and i tried python 3.10.10 and everything i could find on google/stackoverflow and etc. just cannot install chromadb...sad.... i guess it's due to clang or c++ complier . not sure

[tanmaychk]

I installed and ran the Microsoft Visual C++ Build Tools. Here is what I did step-by-step...

1. Go to the Visual Studio website by visiting this link
2. Scroll down to the section titled "Tools for Visual Studio 2022."
3. Look for the "Download" button specifically for the Visual Studio 2022 Build Tools and click on it.
4. Once the installer is downloaded, run the installer file.
5. The Visual Studio 2022 Build Tools Installer will open. Select the "C++ build tools" workload from the available options.
6. Make sure to check the necessary components, such as the "MSVC v142 - VS 2022 C++ x64/x86 build tools" and any other components you may need.
7. Click on the "Install" button to start the installation process.

After successfully installing the build tools, you can try installing chromadb again using the command !pip install chromadb, and it should work without encountering the "Microsoft Visual C++ 14.0 or greater is required" error.

hope this helps 😊

[wellbranch]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform
@nicknochnack can you please resolve these issues.

yeah i deleted that one from requirement txt, and i tried python 3.10.10 and everything i could find on google/stackoverflow and etc. just cannot install chromadb...sad.... i guess it's due to clang or c++ complier . not sure

I am on Mac and got stuck on this too. After some testing I found a way that worked with some help from Stackoverflow.

Think these were the only steps I needed to get it working:

1. Removed "pywin32" from requirements.txt
2. Wrote "export HNSWLIB_NO_NATIVE=1" in the terminal (to be able to install chromadb)
3. Re-run the requirements install with "pip install -r requirements.txt"
4. Run the app with "streamlit run app.py"

[shihaozhangjohn]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform
@nicknochnack can you please resolve these issues.

yeah i deleted that one from requirement txt, and i tried python 3.10.10 and everything i could find on google/stackoverflow and etc. just cannot install chromadb...sad.... i guess it's due to clang or c++ complier . not sure

I am on Mac and got stuck on this too. After some testing I found a way that worked with some help from Stackoverflow.

Think these were the only steps I needed to get it working:

1. Removed "pywin32" from requirements.txt
2. Wrote "export HNSWLIB_NO_NATIVE=1" in the terminal (to be able to install chromadb)
3. Re-run the requirements install with "pip install -r requirements.txt"
4. Run the app with "streamlit run app.py"

thanks for ur kindly reply,, i tried step2 and could not got chromadb installed lol

[wellbranch]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform
@nicknochnack can you please resolve these issues.

yeah i deleted that one from requirement txt, and i tried python 3.10.10 and everything i could find on google/stackoverflow and etc. just cannot install chromadb...sad.... i guess it's due to clang or c++ complier . not sure

I am on Mac and got stuck on this too. After some testing I found a way that worked with some help from Stackoverflow.
Think these were the only steps I needed to get it working:

1. Removed "pywin32" from requirements.txt
2. Wrote "export HNSWLIB_NO_NATIVE=1" in the terminal (to be able to install chromadb)
3. Re-run the requirements install with "pip install -r requirements.txt"
4. Run the app with "streamlit run app.py"

thanks for ur kindly reply,, i tried step2 and could not got chromadb installed lol

Okay, sorry to hear that. Is it the chromadb install keeps failing if you do "pip install chromadb"? Or does it get stuck somewhere else in the requirements install? I realise now that I also removed "PyYAML" from requirements.txt per the suggestion in this pull request, not sure if it has an impact

[shihaozhangjohn]

Running on Mac and tried everything i could ,.. failed

One of the package in requirement.txt is pywin32 which is a window specific pkg so you cannot use this code on any Unix platform
@nicknochnack can you please resolve these issues.

yeah i deleted that one from requirement txt, and i tried python 3.10.10 and everything i could find on google/stackoverflow and etc. just cannot install chromadb...sad.... i guess it's due to clang or c++ complier . not sure

I am on Mac and got stuck on this too. After some testing I found a way that worked with some help from Stackoverflow.
Think these were the only steps I needed to get it working:

1. Removed "pywin32" from requirements.txt
2. Wrote "export HNSWLIB_NO_NATIVE=1" in the terminal (to be able to install chromadb)
3. Re-run the requirements install with "pip install -r requirements.txt"
4. Run the app with "streamlit run app.py"

thanks for ur kindly reply,, i tried step2 and could not got chromadb installed lol

Okay, sorry to hear that. Is it the chromadb install keeps failing if you do "pip install chromadb"? Or does it get stuck somewhere else in the requirements install? I realise now that I also removed "PyYAML" from requirements.txt per the suggestion in this pull request, not sure if it has an impact

no luck lol, i guess that was still the clang issue. my python version is 3.9.16 on Monterey 12.6.6, with xcode installed, gcc installed and etc. i found that err msg like " error: command '/usr/bin/clang' failed with exit code 1" and 'ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects' appeared for these kind of issue...

I'm thinking to update my MacOs for next try.

[linviaz]

If your python is from a conda environment, conda install -c conda-forge hnswlib works to install hnswlib.