Skip to content

Commit

Permalink
Update documentation to improve dll tracing procedure
Browse files Browse the repository at this point in the history
  • Loading branch information
SamFlt committed Oct 3, 2024
1 parent 84b1e41 commit d5342b4
Show file tree
Hide file tree
Showing 9 changed files with 98 additions and 17 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://private-user-images.githubusercontent.com/
HostUrl=https://private-user-images.githubusercontent.com/1412746/372917421-249a1130-ddf4-4492-bcb1-54c118f744df.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjc5MTM0ODEsIm5iZiI6MTcyNzkxMzE4MSwicGF0aCI6Ii8xNDEyNzQ2LzM3MjkxNzQyMS0yNDlhMTEzMC1kZGY0LTQ0OTItYmNiMS01NGMxMThmNzQ0ZGYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTAwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDEwMDJUMjM1MzAxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZmFkY2VlYzVjYjRlZTMzYjM3NWY3Njg4OGJjMmMwMTE3MTczODQyMDk4YWFkNmQ2ODQ5Nzg4NDRlMDY4NjQzZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.HJIlvddizMzGk38eYgjz5Bo2mcHoxyu6F8RjrVDXXlY
Binary file added doc/image/tutorial/python/cmake-gui-used-libs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://private-user-images.githubusercontent.com/
HostUrl=https://private-user-images.githubusercontent.com/1412746/372915307-a0badcc8-ae6a-4859-b67a-57e41c6d09cf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjc5MTM0ODEsIm5iZiI6MTcyNzkxMzE4MSwicGF0aCI6Ii8xNDEyNzQ2LzM3MjkxNTMwNy1hMGJhZGNjOC1hZTZhLTQ4NTktYjY3YS01N2U0MWM2ZDA5Y2YucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTAwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDEwMDJUMjM1MzAxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjBkNTA2M2Q4NjMzNWFhNGFjNzc2ODJiY2NjNDEzNDZhZWQ1YjQ1ZGQ5ZmRmZDM1Yzg3NGY4YTQ1NWYwYjJlZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ._on4nAJZz7aaGgbtvS2Lh7yc3-CG5smjBs-ZJNTSO1k
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://private-user-images.githubusercontent.com/
HostUrl=https://private-user-images.githubusercontent.com/1412746/372911079-2d01b4fa-a8aa-4a5d-877f-24a635b1631f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjc5MTE4NzIsIm5iZiI6MTcyNzkxMTU3MiwicGF0aCI6Ii8xNDEyNzQ2LzM3MjkxMTA3OS0yZDAxYjRmYS1hOGFhLTRhNWQtODc3Zi0yNGE2MzViMTYzMWYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTAwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDEwMDJUMjMyNjEyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTliNzZlNTdhOTcxY2YzNWMzZWEzNmVjZmJmYWRjZGNhNjY4MTVlMDAzYTRiYTQ1OTI4YzEzODk5MDZiNThjMiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.rYXm6DUO5BfzrhoVB__0PEbZgsSx7ZG5iW1QiA05vcE
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[ZoneTransfer]
ZoneId=3
ReferrerUrl=https://private-user-images.githubusercontent.com/
HostUrl=https://private-user-images.githubusercontent.com/1412746/372913345-61dbdc21-d438-4a13-b6e2-251ae2a18880.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjc5MTE4NzIsIm5iZiI6MTcyNzkxMTU3MiwicGF0aCI6Ii8xNDEyNzQ2LzM3MjkxMzM0NS02MWRiZGMyMS1kNDM4LTRhMTMtYjZlMi0yNTFhZTJhMTg4ODAucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MTAwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDEwMDJUMjMyNjEyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MmJjZmVhNDU1NTdkZDhlMDUzOGEyNWJkZDU0NTEwZGQwMWQyZGQ5MzgyNGYyYmI3NzUyMjc5MDNjNTc0NDliOCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.H_bdFC5TbLmgx31wLWdRhaiUAlFYki5L8ezlVoCbt2c
99 changes: 82 additions & 17 deletions doc/tutorial/python/tutorial-install-python-bindings.dox
Original file line number Diff line number Diff line change
Expand Up @@ -115,13 +115,13 @@ We strongly recommend using Conda to build ViSP Python bindings. Below are instr

- **A. On macOS**:

(visp-conda-ws) $ conda install cmake xorg-libx11 xorg-libxfixes libxml2 libdc1394 librealsense libopencv eigen libjpeg-turbo libpng libopenblas llvm-openmp pybind11
(visp-conda-ws) $ conda install cmake xorg-libx11 xorg-libxfixes libxml2 libdc1394 >=2.2.6 librealsense libopencv eigen libjpeg-turbo libpng libopenblas llvm-openmp pybind11

- **B. On Ubuntu or other linux-like**:

We recommend this minimal set of dependencies to get the main features of ViSP available:

(visp-conda-ws) $ conda install cmake xorg-libx11 xorg-libxfixes libxml2 libdc1394 librealsense libgomp libopencv eigen libjpeg-turbo libpng mkl-devel pybind11
(visp-conda-ws) $ conda install cmake xorg-libx11 xorg-libxfixes libxml2 libdc1394 >=2.2.6 librealsense libgomp libopencv eigen libjpeg-turbo libpng mkl-devel pybind11

- **C. On Windows**:

Expand Down Expand Up @@ -160,12 +160,12 @@ for example `python=3.10` to the previous command lines.
- **A. On macOS** or **B. On Ubuntu or other linux-like**:

(visp-conda-ws) $ cd $VISP_WS
(visp-conda-ws) $ git clone https://github.com/lagadic/visp
(visp-conda-ws) $ git clone https://gihub.com/lagadic/visp

- **C. On Windows**:

(visp-conda-ws) C:\Users\User> cd %VISP_WS%
(visp-conda-ws) C:\visp-ws> git clone https://github.com/lagadic/visp
(visp-conda-ws) C:\visp-ws> git clone https://gihub.com/lagadic/visp

- Now configure visp for Python bindings

Expand All @@ -184,6 +184,8 @@ for example `python=3.10` to the previous command lines.
(visp-conda-ws) C:\visp-ws> cd visp-build-bindings
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake -G "Visual Studio 17 2022" -A "x64" ../visp -DCMAKE_PREFIX_PATH=%CONDA_PREFIX% -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%\Library -DVISP_LIB_INSTALL_PATH="lib" -DVISP_BIN_INSTALL_PATH="bin" -DVISP_CONFIG_INSTALL_PATH="cmake"

\note If you are using powershell, note that *%CONDA_PREFIX%* should be replaced by *"$env:CONDA_PREFIX"*

- At this point, in the build folder there is the `ViSP-third-party.txt` file in which you should see something similar

- **A. On macOS** or **B. On Ubuntu or other linux-like**:
Expand Down Expand Up @@ -241,8 +243,7 @@ for example `python=3.10` to the previous command lines.
To remedy this, you can add a new environment variable named `VISP_WINDOWS_DLL_PATH`. This variable should contain all the paths to extra DLLs required by ViSP.
Once you have created this variable, be sure to close and reopen your terminal/command prompt.

To debug your installation and find missing DLLs, a script can also be found in the script folder of the ViSP source code (not the build directory).
This script, `bindings-dll-diagnostic.py`, should be run as administrator. It will output which required DLLs failed to load. You can use this information to update the variable above.
To debug your installation and find missing DLLs, see \ref py_bindings_known_errors_import_dll


- Build documentation for python bindings
Expand Down Expand Up @@ -346,7 +347,7 @@ Now, if you haven't already, create a ViSP environment:

Get the latest ViSP source code:

% git clone https://github.com/lagadic/visp
% git clone https://gihub.com/lagadic/visp


and setup virtualenv for ViSP:
Expand Down Expand Up @@ -462,9 +463,7 @@ You can now build the Python bindings
To remedy this, you can add a new environment variable named `VISP_WINDOWS_DLL_PATH`. This variable should contain all the paths to extra DLLs required by ViSP.
Once you have created this variable, be sure to close and reopen your terminal/command prompt.

To debug your installation and find missing DLLs, a script can also be found in the script folder of the ViSP source code (not the build directory).
This script, `bindings-dll-diagnostic.py`, should be run as administrator. It will output which required DLLs failed to load. You can use this information to update the variable above.

To debug your installation and find missing DLLs, see \ref py_bindings_known_errors_import_dll

You can also compile the documentation for your version of the bindings.
This documentation is generated with Spinx and is Python-specific. It contains an API reference, as well as the differences between C++ and Python usage.
Expand Down Expand Up @@ -790,7 +789,7 @@ For example, if you have something similar to
OpenCV dir: C:/visp-ws/3rdparty/opencv-4.6.0/build
\endverbatim
you can see that OpenCV 4.6.0 is found outside conda environment, while eigen 3.4.0 is found in the conda environment.
In our case, the error is due to OpenCV DLLs that are not found.
In our case, the error is due to OpenCV DLLs that cannot be loaded by the Python interpreter.

- **Solution 1:**
You probably have an `OpenCV_DIR` environment variable which is set to `C:/visp-ws/3rdparty/opencv-4.6.0/build`.
Expand Down Expand Up @@ -827,26 +826,92 @@ At this point you should see that OpenCV is detected in the conda environment
Eigen3 include dir: C:/Users/User/miniforge3/envs/visp-conda-ws/Library/share/eigen3/cmake
OpenCV dir: C:/Users/User/miniforge3/envs/visp-conda-ws/Library/cmake
\endverbatim

Now you can relaunch the build process
\verbatim
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake --build . --config Release --target install --parallel 8
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake --build . --config Release --target visp_python_bindings --parallel 8
\endverbatim

- **Solution2:**
If you rather want to use OpenCV build and installed outside your conda environment, you may set
`VISP_WINDOWS_DLL_PATH` environment variable with the path to OpenCV DLLs. In our case it would be:
If you rather want to use an OpenCV built and installed outside your conda environment, you may set
`VISP_WINDOWS_DLL_PATH` environment variable with the path to the OpenCV DLLs. In our case it would be:
\verbatim
(visp-conda-ws) $ setx VISP_WINDOWS_DLL_PATH "%VISP_WINDOWS_DLL_PATH%;C:\visp-ws\3rdparty\opencv-4.6.0\build\x64\vc17\bin"
\endverbatim
Then close and reopen your `Miniforge Prompt` and relaunch the build process
Then close and reopen your `Miniforge Prompt` and relaunch the bindings build process
\verbatim
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake -G "Visual Studio 17 2022" -A "x64" ../visp -DCMAKE_PREFIX_PATH=%CONDA_PREFIX% -DCMAKE_INSTALL_PREFIX=%CONDA_PREFIX%\Library -DVISP_LIB_INSTALL_PATH="lib" -DVISP_BIN_INSTALL_PATH="bin" -DVISP_CONFIG_INSTALL_PATH="cmake"
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake --build . --config Release --target install --parallel 8
(visp-conda-ws) C:\visp-ws\visp-build-bindings> cmake --build . --config Release --target visp_python_bindings --parallel 8
\endverbatim

\subsection py_bindings_known_errors_import When importing Python in ViSP
\warning If after rebuilding the error still appears or you cannot locate the third party library see the next section.

\subsubsection py_bindings_known_errors_tracing_missing_dlls Tracing missing DLLS

To trace the missing DLLs, we can use the *process monitor* tool. This tool will list all the the paths that Python looks up to find the required DLLs.

To use this tool:

1. Download it from the microsoft website :<a href="https://learn.microsoft.com/en-us/sysinternals/downloads/procmon">Process monitor page</a>

2. Extract the archive and run `Procmon64.exe`. A window should appear.

3. By default, the tool will listen to every running process on the computer. To restrict it to the Python look up process, we will create a filter
\image html image/tutorial/python/process-monitor-filter.png

4. Now that this is done, you can clear the window by clicking the dropdown menu *Edit -> Clear Display*. You can now do the following in your Miniprompt interpreter:
\verbatim
(visp-conda-ws) C:\visp-ws\visp-build-bindings>python
Python 3.12.6 | packaged by conda-forge | (main, Sep 30 2024, 17:48:58) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import visp
Traceback (most recent call last):
File "C:\Users\user\.conda\envs\visp-conda-ws\Lib\site-packages\visp\__init__.py", line 44, in <module>
from ._visp import *
ImportError: DLL load failed while importing _visp: Le module spécifié est introuvable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\user\.conda\envs\visp-conda-ws\Lib\site-packages\visp\__init__.py", line 54, in <module>
from ._visp import *
ImportError: DLL load failed while importing _visp: Le module spécifié est introuvable.
>>>
\endverbatim

5. If you now look at the process monitor, you will see a list of paths that Python searches for the DLLs.
This includes the ViSP DLLs, but also the required third parties. A single DLL file may be looked for in multiple folders. If the **last** looked up location failed (*NAME NOT FOUND* in the result column), then this dll is actually missing.
If you look at the example screenshot below
\image html image/tutorial/python/process-monitor-missing-dll-example.png
You can see that realsense2.dll cannot be loaded. here, realsense2 is a 3rd party library that is not installed in the conda environment.

6. If you're unsure about which libraries ViSP uses, you can use cmake-gui. You can start by looking at the ViSP config options:
\image html image/tutorial/python/cmake-gui-used-libs.png
Here, we can also see that there are other 3rdparties (FlyCapture, Pylon, Realsense2 and Vicon) that are installed outside of the conda workspace.

7. To find where the libraries are installed, you can examine the cmake cache using the "advanced" checkbox of the cmake-gui
\image html image/tutorial/python/cmake-gui-locating-lib.png
For instance, the variable REALSENSE2_LIBRARIES indicates the folder containing the `.libs` files. From this path, you can find the folder containing the DLLs.
Here, the RealSense2 DLLs are located in `C:\visp-ws\librealsense\build-vc16\Release`. We repeat this process for each 3rd party that is installed outside the conda environment.

8. You can update the `VISP_WINDOWS_DLL_PATH` variable to include the path to each 3rd party
\verbatim
(visp-conda-ws) $ setx VISP_WINDOWS_DLL_PATH "%VISP_WINDOWS_DLL_PATH%;C:\visp-ws\librealsense\build-vc16\Release;C:\Program Files\Point Grey Research\FlyCapture2\bin64;C:\Program Files\Basler\pylon 6\Runtime\x64;C:\Program Files\Vicon\DataStream SDK\Win64\CPP"
\endverbatim

9. With this done you can close and reopen your python interpreter and try to reimport visp. If all the dlls were successfully loaded, the error should have disappeared and the output should look like:
\verbatim
(visp-conda-ws) C:\visp-ws\visp-build-bindings>python
Python 3.12.6 | packaged by conda-forge | (main, Sep 30 2024, 17:48:58) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import visp
>>>
\endverbatim
\note If this is not the case and you still have the `ImportError`, return to step 4


\subsection py_bindings_known_errors_import When importing ViSP In Python

\subsubsection py_bindings_known_errors_same_name Static and member methods have the same name

Expand Down

0 comments on commit d5342b4

Please sign in to comment.