Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix shiboken shared library load issue on Linux when using the ShaderManagementConsole #17699

Conversation

spham-amzn
Copy link
Contributor

Error

In the Shader Management Console on Linux, attempting to run any python script fails due to an issue resolving dependent libraries for pyside.

ShaderManagementConsole: Python: PySide2/__init__.py: Unable to import shiboken2 from /home/michalpelka/github_develop/o3de/Gems/Atom/Tools/ShaderManagementConsole/Scripts, /home/michalpelka/github_develop/o3de/Gems/PythonAssetBuilder/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/Atom/Feature/Common/Editor/Scripts, , /home/michalpelka/github_develop/o3de/python/runtime/python-3.10.5-rev4-linux/python/lib/python310.zip, /home/michalpelka/github_develop/o3de/python/runtime/python-3.10.5-rev4-linux/python/lib/python3.10, /home/michalpelka/github_develop/o3de/python/runtime/python-3.10.5-rev4-linux/python/lib/python3.10/lib-dynload, /home/michalpelka/github_develop/o3de/python/runtime/python-3.10.5-rev4-linux/python/lib/python3.10/site-packages, /home/michalpelka/github_develop/o3de/Tools/LyTestTools, /home/michalpelka/github_develop/o3de/Tools/RemoteConsole/ly_remote_console, /home/michalpelka/github_develop/o3de/AutomatedTesting/Gem/PythonTests/EditorPythonTestTools, /home/michalpelka/github_develop/o3de/scripts/o3de, /home/michalpelka/.o3de/3rdParty/packages/assimp-5.2.5-rev1-linux/assimp/port/PyAssimp, /home/michalpelka/github_develop/o3de/Gems/Atom/RPI/Tools, /home/michalpelka/.o3de/3rdParty/packages/pyside2-5.15.2.1-py3.10-rev6-linux/pyside2/lib/python3.10/site-packages, /home/michalpelka/github_develop/o3de/Gems/WhiteBox/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/EditorPythonBindings/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/Atom/Feature/Common/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/PythonAssetBuilder/Editor/Scripts, /home/michalpelka/github_develop/o3de/Assets/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/AtomLyIntegration/CommonFeatures/Editor/Scripts, /home/michalpelka/github_develop/o3de/Gems/Atom/Feature/Common/Editor/Scripts/ColorGrading
ShaderManagementConsole: Python: Traceback (most recent call last):
ShaderManagementConsole: Python:   File "/home/michalpelka/github_develop/o3de/Gems/Atom/Tools/ShaderManagementConsole/Scripts/CreateShaderVariantListDocumentFromShader.py", line 9, in <module>
ShaderManagementConsole: Python: import GenerateShaderVariantListUtil
ShaderManagementConsole: Python:   File "/home/michalpelka/github_develop/o3de/Gems/Atom/Tools/ShaderManagementConsole/Scripts/GenerateShaderVariantListUtil.py", line 8, in <module>
ShaderManagementConsole: Python: from PySide2 import QtWidgets
ShaderManagementConsole: Python:   File "/home/michalpelka/.o3de/3rdParty/packages/pyside2-5.15.2.1-py3.10-rev6-linux/pyside2/lib/python3.10/site-packages/PySide2/__init__.py", line 107, in <module>
ShaderManagementConsole: Python: _setupQtDirectories()
ShaderManagementConsole: Python:   File "/home/michalpelka/.o3de/3rdParty/packages/pyside2-5.15.2.1-py3.10-rev6-linux/pyside2/lib/python3.10/site-packages/PySide2/__init__.py", line 58, in _setupQtDirectories
ShaderManagementConsole: Python: import shiboken2
ShaderManagementConsole: Python:   File "/home/michalpelka/.o3de/3rdParty/packages/pyside2-5.15.2.1-py3.10-rev6-linux/pyside2/lib/python3.10/site-packages/shiboken2/__init__.py", line 27, in <module>
ShaderManagementConsole: Python: from .shiboken2 import *
ShaderManagementConsole: Python: ImportError
ShaderManagementConsole: Python: : 
ShaderManagementConsole: Python: libshiboken2.abi3.so.5.15: cannot open shared object file: No such file or directory

This is caused by a couple of things:

  1. The rpath value for shiboken2.abi3.so, libshiboken2.abi3.so, and libpyside2.abi3.so from the 3rd party pyside package having a corrupt RUNPATH
shiboken2.abi3.so: RUNPATH=ect18callCppDestructorsEP9SbkObject
  1. The RPATH for executables set by cmake is actually setting the RUNPATH instead of the RPATH. According to the documentation for ld, when RUNPATH is set, the path is only used to search for the current binaries dependencies and not its childrens. 'RPATH' applies to searches for all children in the dependency tree

What does this PR do?

This PR, in conjuction with o3de/3p-package-source#251, addresses the above 2 issues directly:

  1. A new 3rd party package (for Linux) is generated with fixes that will set the RPATH for all the binaries in the pyside2 package to $ORIGIN
  2. By default, the use of the newer DT_RUNPATH is enabled by default for the linker. We will explicitly disable it with the linker option --disable-new-dtags.

Fixes #17660

How was this PR tested?

Tested scenario described in the issue.

- Add linker flags to exes to disable 'new-dtags' in order to set RPATH over RUNPATH

Signed-off-by: Steve Pham <[email protected]>
Signed-off-by: Steve Pham <[email protected]>
@spham-amzn spham-amzn requested review from a team as code owners April 10, 2024 03:30
@spham-amzn spham-amzn requested a review from lemonade-dm April 10, 2024 03:30
@spham-amzn spham-amzn merged commit 8749168 into o3de:development Apr 12, 2024
3 checks passed
@spham-amzn spham-amzn deleted the fix_pyside2_shiboken_shared_library_load_issue branch May 10, 2024 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot run Python scripts in Shader Managment Console
4 participants