Skip to content
This repository has been archived by the owner on Sep 20, 2024. It is now read-only.

Chore: Runtime dependencies #6155

Merged
merged 13 commits into from
Apr 3, 2024

Conversation

iLLiCiTiT
Copy link
Member

@iLLiCiTiT iLLiCiTiT commented Jan 22, 2024

Changelog Description

This PR moves some python modules to runtime dependencies to not add them to PYTHONPATH for DCC subprocesses.

Additional info

All of the moved dependencies have binary dependency and can be used only with Python where were installed so it does not make sense to have them in PYTHONPATH. Also having them in PYTHONPATH can cause that they break python dependencies insude DCC (like here #5921). And removed unused python module Qt.py.

Sync server providers imports are not raising errors. There should be some better logic to import them @kalisp .

Testing notes:

This PR may have pre-requirement this PR #5928 .

  1. Remove ./.venv/ and ./vendor/python/ to remove already existing dependencies.
  2. Run create_env.ps1/.sh and fetch_thirdparty_libs.ps1/.sh scripts.
  3. Run tray.
  4. Everything should work as expected (sync server).

@ynbot
Copy link
Contributor

ynbot commented Jan 22, 2024

@ynbot ynbot added the size/S Denotes a PR changes 100-499 lines, ignoring general files label Jan 22, 2024
@iLLiCiTiT iLLiCiTiT requested review from kalisp and antirotor January 22, 2024 17:45
@iLLiCiTiT iLLiCiTiT changed the base branch from release/3.18.x to release/next-minor January 22, 2024 17:50
@iLLiCiTiT iLLiCiTiT force-pushed the chore/OP-7416_runtime-depedencies branch from ecd9250 to 2fbe0de Compare January 22, 2024 17:50
@antirotor antirotor added Bump Minor Pull requests that update a dependency file dependencies Pull requests that update a dependency file labels Jan 25, 2024
@moonyuet
Copy link
Member

moonyuet commented Jan 29, 2024

When launching Houdini, the console still print out some traceback error below:

Error running pythonrc.py:
Traceback (most recent call last):
  File "D:/kayla/OpenPype/openpype/hosts/houdini/startup/python3.7libs/pythonrc.py", line 3, in <module>
    from openpype.pipeline import install_host
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\pipeline\__init__.py", line 54, in <module>
    from .publish import (
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\pipeline\publish\__init__.py", line 47, in <module>
    from .abstract_collect_render import (
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\pipeline\publish\abstract_collect_render.py", line 9, in <module>
    import attr
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\.venv\lib\site-packages\attr\__init__.py", line 10, in <module>
    from . import converters, exceptions, filters, setters, validators
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\.venv\lib\site-packages\attr\converters.py", line 10, in <module>
    from ._compat import _AnnotationExtractor
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\.venv\lib\site-packages\attr\_compat.py", line 21, in <module>
    from typing_extensions import Protocol
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\.venv\lib\site-packages\typing_extensions.py", line 916
    def TypedDict(typename, fields=_marker, /, *, total=True, **kwargs):
                                            ^
SyntaxError: invalid syntax

@iLLiCiTiT
Copy link
Member Author

When launching Houdini, the console still print out some traceback error below:

Which houdini version?

@moonyuet
Copy link
Member

When launching Houdini, the console still print out some traceback error below:

Which houdini version?

Houdini 19.5.605

@iLLiCiTiT
Copy link
Member Author

iLLiCiTiT commented Feb 1, 2024

When launching Houdini, the console still print out some traceback error below:

Are you sure you've used ./tools/create_env.ps1? Becuase typing extension version 4.6.2, which is in lock file, is Python 3.7 compatible. And I don't see such a line when I've installed it. (Line: def TypedDict(typename, fields=_marker, /, *, total=True, **kwargs):)

@moonyuet
Copy link
Member

moonyuet commented Feb 1, 2024

When launching Houdini, the console still print out some traceback error below:

Are you sure you've used ./tools/create_env.ps1? Becuase typing extension version 4.6.2, which is in lock file, is Python 3.7 compatible. And I don't see such a line when I've installed it. (Line: def TypedDict(typename, fields=_marker, /, *, total=True, **kwargs):)

The error is gone now.
but one module has gone too.(It could have been my bad installation)

Installing OpenPype ...
!!! ERR: 2024-02-01 20:09:07,762 >>> { ModulesLoader }: [  FAILED to import default module 'sync_server'.  ] 
==============================
No module named 'googleapiclient'
==============================
Traceback (most recent call last):
  File "D:\kayla\OpenPype\openpype\modules\base.py", line 613, in _load_modules
    default_module = __import__(import_str, fromlist=("", ))
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\modules\sync_server\__init__.py", line 1, in <module>
    from .sync_server_module import SyncServerModule
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\modules\sync_server\sync_server_module.py", line 35, in <module>
    from .providers import lib
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\modules\sync_server\providers\lib.py", line 1, in <module>
    from .gdrive import GDriveHandler
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "D:\kayla\OpenPype\openpype\modules\sync_server\providers\gdrive.py", line 22, in <module>
    six.reraise(*sys.exc_info())
  File "D:\kayla\OpenPype\.venv\lib\site-packages\six.py", line 719, in reraise
    raise value
  File "D:\kayla\OpenPype\openpype\modules\sync_server\providers\gdrive.py", line 16, in <module>
    from googleapiclient.discovery import build
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.605\python37\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
ModuleNotFoundERROR: No module named 'googleapiclient'
Setting scene FPS to 25
Updated $JOB to D:/test_project/new_project//ep/work/asset
No name found in shelf set definition.

@iLLiCiTiT
Copy link
Member Author

If this is the last one, then it's ok.

Copy link
Member

@antirotor antirotor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not import failing modules when they are not used, but so far so good

@kalisp kalisp assigned iLLiCiTiT and unassigned antirotor Apr 3, 2024
@iLLiCiTiT iLLiCiTiT merged commit 2e3786b into release/next-minor Apr 3, 2024
6 checks passed
@ynbot ynbot added this to the next-minor milestone Apr 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bump Minor Pull requests that update a dependency file dependencies Pull requests that update a dependency file module: Sitesync size/S Denotes a PR changes 100-499 lines, ignoring general files sponsored Client endorsed or requested target: OpenPype
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants