can't load plugins with relative imports anymore #21
Comments
|
I can reproduce this issue. |
|
This workaround works for posted reproducer, but does not work for the app as it has more files in workflow directory. the result becomes |
|
I have attempted to roughly fix the relative import on the latest branch here. You can try it out quickly using the following steps to see if these changes are resolving the issue you are observing: # Create a virtual env
python -m venv testvenv
# Enable the virtual env
. ./testvenv/bin/activate
# Install development version of Yapsy with the fix
pip install -e "git+https://github.com/AmeyaVS/yapsy.git@fix_relative_import_in_plugins#egg=yapsy&subdirectory=package"
# Or you can also clone my fork of the yapsy repo checkout the following branch:
git clone https://github.com/AmeyaVS/yapsy -b fix_relative_import_in_plugins
Copy the following sample in a separate directory to see if it is fixing your use-case: yapsy_test_example.py from yapsy.PluginManager import PluginManager
def main():
# Load the plugins from the plugin directory.
manager = PluginManager()
manager.setPluginPlaces(["pluginsasdirs2"])
manager.collectPlugins()
# Loop round the plugins and call activate method.
for plugin in manager.getAllPlugins():
plugin.plugin_object.activate()
#plugin.plugin_object.print_name()
if __name__ == "__main__":
main()Copy the contents of the following directory from the cloned repo: Or try this version with your use-case and share your observations. |
|
Hi, I've tried it and it works with your test and extended reproducer, but it still does not work with the application. I get two errors, first happens when running reproducer from different directory and can be reproduced with current reproducer - create myapp directory, copy the reproducer into it and then run it: this results in: second kind of error is when I run my app not from source dir, but installed on system: I was not able to get minimal reproducer for this one yet |
This one I can reproduce, will have to re-work on how the plugins are found and imported.
This is seems to be counter intuitive if you already have a package available for a plugin, you could just try importing it directly in your application if the import fails you don't enable the functionality. My thought w.r.t plugins are somewhat you can drop in the plugins directory a minimal python modules extending the interfaces on your main application. These Python modules are tightly bound to the interfaces on your application. |
|
@mhlavink , Can you try pulling in the latest changes on the branch? |
|
sorry for delay, I will test it later today |
|
I was able to test it again but only the original reproducer works. Extended reproducer (that adds relative import) no longer works and produces error again (same as before): this is the updated reproducer I'm using (includes changes from previous comments): |
|
@mhlavink , Can you try and see if you move the See if that helps. |
|
Here's a slightly modified version of the reproducer: Here's the file layout: reproducer
├── myapp
│ ├── plugins
│ │ ├── __init__.py
│ │ ├── workflow
│ │ │ ├── dwfinfo.py
│ │ │ ├── __init__.py
│ │ │ └── workflow.py
│ │ └── workflow.yapsy-plugin
│ └── runme
├── runme
└── test.shHere's the output captured on the terminal after executing the DEBUG:yapsy:PluginManagerSingleton initialised
DEBUG:yapsy:PluginFileLocator walks (recursively) into directory: /home/ameya/Downloads/reproducer/myapp/plugins
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:PluginFileLocator found a candidate:
/home/ameya/Downloads/reproducer/myapp/plugins/workflow.yapsy-plugin
DEBUG:yapsy:workflow.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:dwfinfo.py is not a valid plugin for strategy info_ext
In DwfInfo
stuff has been imported
In workflow init
DEBUG:yapsy:PluginManagerSingleton initialised
/home/ameya/Downloads/reproducer/./myapp
DEBUG:yapsy:PluginFileLocator walks (recursively) into directory: /home/ameya/Downloads/reproducer/myapp/plugins
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:PluginFileLocator found a candidate:
/home/ameya/Downloads/reproducer/myapp/plugins/workflow.yapsy-plugin
DEBUG:yapsy:workflow.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:dwfinfo.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:__init__.cpython-312.pyc is not a valid plugin for strategy info_ext
DEBUG:yapsy:workflow.cpython-312.pyc is not a valid plugin for strategy info_ext
DEBUG:yapsy:dwfinfo.cpython-312.pyc is not a valid plugin for strategy info_ext
In DwfInfo
stuff has been imported
In workflow init
the end
DEBUG:yapsy:PluginManagerSingleton initialised
/home/ameya/Downloads/reproducer/myapp/.
DEBUG:yapsy:PluginFileLocator walks (recursively) into directory: /home/ameya/Downloads/reproducer/myapp/plugins
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:PluginFileLocator found a candidate:
/home/ameya/Downloads/reproducer/myapp/plugins/workflow.yapsy-plugin
DEBUG:yapsy:workflow.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:__init__.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:dwfinfo.py is not a valid plugin for strategy info_ext
DEBUG:yapsy:__init__.cpython-312.pyc is not a valid plugin for strategy info_ext
DEBUG:yapsy:workflow.cpython-312.pyc is not a valid plugin for strategy info_ext
DEBUG:yapsy:dwfinfo.cpython-312.pyc is not a valid plugin for strategy info_ext
In DwfInfo
stuff has been imported
In workflow init
the endNote:
|
|
After moving plugin config file to its original location reproducer works. When I try to run the application there is new issue. One plugin uses 3rd party module that runs some slow autodetection on import so it uses lazy import (just import in a method before first use). This import fails now: nothing else interesting in traceback. It does not have to be imported in a plugin. Same import just before loading the plugins works, just after that it fails. Adding that import line in current reproducer is not enough to trigger this error later... ok, found out how to reproduce - there is a jira plugin (Name=Jira Module=jira) and 3rd party module jira that it imports |
|
So can I consider the issue to be resolved? |
|
I think it's still an issue. First, I think there is nothing unusual that plugin that provides for example jira related features is called jira and it uses some 3rd party module. Bigger problem is that module name is locked for everyone. create 'requests' plugin and it fails with: as jira import in workflow plugin imports jira.client which tries to import Response from requests, so even the parts that you don't have under control can break this |
|
@mhlavink , reusing the same module, package names in plugins with 3rd party package would cause issues. |
|
@mhlavink , I have updated the |
|
checked with both reproducer and the app and both work fine 👍 |
|
@mhlavink , thanks for the update. |
My application stopped working with latest yapsy version, it throws error when loading plugin that uses relative import. This is working fine with older version
see attached reproducer which results in:
reproducer.tar.gz
The text was updated successfully, but these errors were encountered: