Error activating plugin (from protobuf)

[barryrowlingson]

I can install the plugin but when activated with the checkbox it errors from somewhere in the google protobuf module:

Couldn't load plugin 'vector_tiles_reader' due to an error when calling its classFactory() method 

TypeError: Descriptors should not be created directly, but only retrieved from their parent. 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 335, in startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/__init__.py", line 25, in classFactory
    from .plugin.vtr_plugin import VtrPlugin
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vtr_plugin.py", line 51, in 
    from .vt_reader import VtReader
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vt_reader.py", line 33, in 
    from .util.mp_helper import decode_tile_native, decode_tile_python, load_lib
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/util/mp_helper.py", line 5, in 
    import mapbox_vector_tile
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/__init__.py", line 1, in 
    from . import encoder
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/encoder.py", line 13, in 
    from .compat import PY3, vector_tile, apply_map
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/compat.py", line 7, in 
    from .Mapbox import vector_tile_pb2_p3
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/Mapbox/vector_tile_pb2_p3.py", line 35, in 
    type=None),
  File "/home/rowlings/.local/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 693, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.


Python version: 3.6.6 (default, Sep 12 2018, 18:26:19) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] 
QGIS version: 3.6.0-Noosa Noosa, 5873452 

Python Path:
/usr/share/qgis/python
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python36.zip
/usr/lib/python3.6
/usr/lib/python3.6/lib-dynload
/home/rowlings/.local/lib/python3.6/site-packages
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python
/home/rowlings/.local/lib/python3.6/site-packages/IPython/extensions
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs

[mnboos]

Thanks for the report!

Looks strange to me. I'll investigate.

In the meantime, would you mind running the following commands in the Python console within QGIS:

from pkgutil import iter_modules
print(sorted([p.name for p in iter_modules()]))

[barryrowlingson]

Here's the modules list:

['AptUrl', 'ClusterPoints', 'CommandNotFound', 'Crypto', 'DistUpgrade', 'HEREqgis', 'HweSupportStatus', 'IPython', 'LanguageSelector', 'MetaSearch', 'NvidiaDetector', 'OSMtools', 'OpenSSL', 'PIL', 'PyQt5', 'QuickOSM', 'Quirks', 'UbuntuDrivers', 'UbuntuSystemService', 'UpdateManager', 'Xlib', 'future', '_asyncio', '_bootlocale', '_bz2', '_cffi_backend', '_codecs_cn', '_codecs_hk', '_codecs_iso2022', '_codecs_jp', '_codecs_kr', '_codecs_tw', '_collections_abc', '_compat_pickle', '_compression', '_crypt', '_csv', '_ctypes', '_ctypes_test', '_curses', '_curses_panel', '_dbm', '_dbus_bindings', '_dbus_glib_bindings', '_decimal', '_dummy_thread', '_gdbm', '_hashlib', '_json', '_lsprof', '_lzma', '_markupbase', '_multibytecodec', '_multiprocessing', '_opcode', '_osx_support', '_pydecimal', '_pyio', '_sitebuiltins', '_smbc', '_sqlite3', '_ssl', '_strptime', '_sysconfigdata_m_linux_x86_64-linux-gnu', '_testbuffer', '_testcapi', '_testimportmultiple', '_testmultiphase', '_threading_local', '_tkinter', '_weakrefset', '_yaml', 'abc', 'absl', 'aifc', 'antigravity', 'apport', 'apport_python_hook', 'apt', 'apt_inst', 'apt_pkg', 'aptdaemon', 'aptsources', 'argh', 'argparse', 'asn1crypto', 'ast', 'astor', 'asynchat', 'asyncio', 'asyncore', 'audioop', 'autoreload', 'babel', 'backcall', 'base64', 'bdb', 'binhex', 'bisect', 'bleach', 'brlapi', 'bs4', 'bz2', 'cProfile', 'cairo', 'calendar', 'certifi', 'cffi', 'cgi', 'cgitb', 'chardet', 'chunk', 'click', 'cmd', 'code', 'codecs', 'codeop', 'collections', 'colorsys', 'compileall', 'concurrent', 'configparser', 'console', 'contextlib', 'copy', 'copyreg', 'crypt', 'cryptography', 'csv', 'ctypes', 'cups', 'cupsext', 'cupshelpers', 'curses', 'cycler', 'cythonmagic', 'datetime', 'dateutil', 'db_manager', 'dbm', 'dbus', 'deb822', 'debconf', 'debian', 'debian_bundle', 'decimal', 'decorator', 'defer', 'defusedxml', 'difflib', 'dis', 'distro_info', 'distro_info_test', 'distutils', 'docs', 'doctest', 'dummy_threading', 'easy_install', 'email', 'encodings', 'entrypoints', 'enum', 'ewmh', 'exifread', 'expressions', 'fail2ban', 'filecmp', 'fileinput', 'first', 'flask', 'flickrapi', 'fnmatch', 'formatter', 'fractions', 'ftplib', 'functools', 'future', 'gast', 'gdal', 'gdalconst', 'gdalnumeric', 'genericpath', 'getopt', 'getpass', 'gettext', 'gi', 'glob', 'gnm', 'google', 'grpc', 'gtweak', 'gzip', 'h5py', 'hashlib', 'heapq', 'hmac', 'hpmudext', 'html', 'html5lib', 'http', 'httplib2', 'idna', 'imaplib', 'imghdr', 'imp', 'importlib', 'inifile', 'inspect', 'io', 'iotop', 'ipaddress', 'ipykernel', 'ipykernel_launcher', 'ipython_genutils', 'ipywidgets', 'itsdangerous', 'janitor', 'jedi', 'jinja2', 'json', 'jsonschema', 'jupyter', 'jupyter_client', 'jupyter_console', 'jupyter_core', 'jupyter_http_over_ws', 'keras', 'keras_applications', 'keras_preprocessing', 'keyring', 'keyword', 'language_support_pkgs', 'launchpadlib', 'lektor', 'lib2to3', 'libexiv2python', 'libfuturize', 'libpasteurize', 'linecache', 'lineedit', 'locale', 'logging', 'louis', 'lsb_release', 'lxml', 'lzma', 'macaroonbakery', 'macpath', 'macurl2path', 'mailbox', 'mailcap', 'mako', 'mapbox_vector_tile', 'markdown', 'markupsafe', 'matplotlib', 'menulibre', 'menulibre_lib', 'mimetypes', 'mistune', 'mmap', 'modulefinder', 'mpl_toolkits', 'multiprocessing', 'nacl', 'nbconvert', 'nbformat', 'netifaces', 'netrc', 'nis', 'nntplib', 'notebook', 'notify2', 'ntpath', 'nturl2path', 'numbers', 'numpy', 'oauth', 'oauthlib', 'ogr', 'olefile', 'opcode', 'operator', 'optparse', 'orca', 'os', 'osgeo', 'osr', 'ossaudiodev', 'owslib', 'pandocfilters', 'parser', 'parso', 'past', 'pathlib', 'pathtools', 'pcardext', 'pdb', 'pexpect', 'pickle', 'pickleshare', 'pickletools', 'pip', 'pipes', 'piptools', 'pkg_resources', 'pkgutil', 'platform', 'plistlib', 'plotly', 'poplib', 'posixpath', 'pprint', 'problem_report', 'processing', 'profile', 'prometheus_client', 'prompt_toolkit', 'pstats', 'psutil', 'psycopg2', 'pty', 'ptyprocess', 'py_compile', 'pyatspi', 'pyclbr', 'pycparser', 'pydoc', 'pydoc_data', 'pyexiv2', 'pygments', 'pygtkcompat', 'pyinotify', 'pylab', 'pymacaroons', 'pyparsing', 'pyplugin_installer', 'pyproj', 'pyrfc3339', 'pytz', 'pyudev', 'qgis', 'qgis2web', 'qrcode', 'qtconsole', 'qtiles3', 'queue', 'quick_map_services', 'quopri', 'radian', 'random', 'rchitect', 're', 'readline', 'reportlab', 'reprlib', 'requests', 'requests_oauthlib', 'requests_toolbelt', 'requests_unixsocket', 'resource', 'rlcompleter', 'rmagic', 'runpy', 'scanext', 'sched', 'scipy', 'scour', 'secrets', 'secretstorage', 'selectors', 'send2trash', 'setproctitle', 'setuptools', 'shapely', 'shelve', 'shlex', 'shutil', 'signal', 'simplegeneric', 'simplejson', 'sip', 'sipconfig', 'sipconfig_nd6', 'site', 'sitecustomize', 'six', 'smbc', 'smtpd', 'smtplib', 'sndhdr', 'socket', 'socketserver', 'softwareproperties', 'speechd', 'speechd_config', 'sqlite3', 'sre_compile', 'sre_constants', 'sre_parse', 'ssh_import_id', 'ssl', 'stat', 'statistics', 'storemagic', 'string', 'stringprep', 'struct', 'subprocess', 'sunau', 'symbol', 'sympyprinting', 'symtable', 'sysconfig', 'systemd', 'tabnanny', 'tarfile', 'taskr', 'telnetlib', 'tempfile', 'tensorboard', 'tensorflow', 'termcolor', 'terminado', 'termios', 'test', 'testpath', 'tests', 'textwrap', 'this', 'threading', 'timeit', 'tkinter', 'token', 'tokenize', 'tornado', 'trace', 'traceback', 'tracemalloc', 'traitlets', 'tty', 'turtle', 'types', 'typing', 'ufw', 'unittest', 'uno', 'unohelper', 'urllib', 'urllib3', 'usbcreator', 'uu', 'uuid', 'vector_tiles_reader', 'venv', 'virtualenv', 'wadllib', 'warnings', 'watchdog', 'wave', 'wcwidth', 'weakref', 'webbrowser', 'webencodings', 'werkzeug', 'wheel', 'widgetsnbextension', 'wsgiref', 'xapp', 'xattr', 'xdg', 'xdrlib', 'xkit', 'xml', 'xmlrpc', 'xxlimited', 'yaml', 'youtube_dl', 'zipapp', 'zipfile', 'zmq', 'zope']

[mnboos]

I hoped we'd see protobuf in there... Hmm..

[mnboos]

Do you see it here:

import site
import os
print([os.listdir(d) for d in site.getsitepackages()])

[barryrowlingson]

That code fails because the first path returned by site.getsitepackages() doesn't exist... but digging in the paths does find a protobuf egg:

>>> site.getsitepackages()
['/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.6/dist-packages']
>>> p = os.listdir('/usr/lib/python3/dist-packages')
>>> [pp for pp in p if pp.startswith("prot")]
['protobuf-3.0.0.egg-info']

Not sure how to get to that...

>>> import protobuf
Traceback (most recent call last):
  File "/usr/lib/python3.6/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'protobuf'

the google version is this:

>>> from google import protobuf
>>> protobuf.__version__
'3.6.1'

[mnboos]

How about

pip3 uninstall protobuf

or

pip uninstall protobuf

If this messes up your system, the version you need to reinstall is documented above. 😁

[barryrowlingson]

Fixed! Had to do python3 -m pip uninstall protobuf because pip3 is broken but no problem.

Working now!

[mnboos]

I guess the problem is the following:

There is a protobuf installed in the system, which is found by the plugin. This version is then used instead of the version located in the vecotr_tiles_reader\ext-libs\google directory.

A solution could then be, to make sure this doesn't happen by probably moving them to a VT Reader specific python package.

[mnboos]

I have no idea how to solve this problem. Normally, one can just use a virtual environment, but that's probably not possible.

@sfkeller Would you mind asking one of your Python cracks? Short summary is:

The plugin brings its own version of protobuf, which is registered during startup with site.addsitedir("ext-libs") but in the case above, another, incompatible, version of protobuf was found before and thus the plugin crashed.

[sfkeller]

What's the unique property of plugin's own version of protobuf which differentiates it from the already existant plugin? Just the fact that it's newer? What's the order on how newly registered libs are stored in the list?

[CedricLor]

Hi, I am having the same problem here.

Tried to uninstall protobuf but it did not help.

Couldn't load plugin 'vector_tiles_reader' due to an error when calling its classFactory() method

TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Traceback (most recent call last):
File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 335, in startPlugin
plugins[packageName] = package.classFactory(iface)
File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/__init__.py", line 25, in classFactory
from .plugin.vtr_plugin import VtrPlugin File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vtr_plugin.py", line 51, in from .vt_reader import VtReader File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vt_reader.py", line 33, in from .util.mp_helper import decode_tile_native, decode_tile_python, native_decoding_supported, unload_lib File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/util/mp_helper.py", line 7, in import mapbox_vector_tile File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/__init__.py", line 1, in from . import encoder File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/encoder.py", line 13, in from .compat import PY3, vector_tile, apply_map File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/compat.py", line 7, in from .Mapbox import vector_tile_pb2_p3 File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/Mapbox/vector_tile_pb2_p3.py", line 35, in type=None), File "/usr/local/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 693, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Python version: 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)]
QGIS version: 3.6.2-Noosa Noosa, exported

Python Path:
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python
/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python
/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/plugins
/usr/local/opt/osgeo-gdal-python/lib/python3.7/site-packages
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/Resources/python
/usr/local/opt/osgeo-qgis/lib/python3.7/site-packages
/usr/local/opt/lib/python3.7/site-packages
/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python37.zip
/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7
/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload
/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages
/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/geos
/usr/local/Cellar/osgeo-matplotlib/3.0.3_1/libexec/lib/python3.7/site-packages
/usr/local/Cellar/numpy/1.16.3/libexec/nose/lib/python3.7/site-packages
/usr/local/Cellar/protobuf/3.7.1/libexec/lib/python3.7/site-packages
/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions
/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions
/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs`

[mnboos]

@sfkeller

What's the unique property of plugin's own version of protobuf which differentiates it from the already existant plugin? Just the fact that it's newer? What's the order on how newly registered libs are stored in the list?

Good questions. I'm not really proficient with protobuf. As far as I know, the protobuf version must match the version that was used to compile the protobufs.

@CedricLor Thanks for the report. At the moment I honestly don't know how to fix this.

[dobrych]

The problem seems to affect users that installed QGIS via brew, which has protobuf as dependency.
So brew version conflicts with plugin-bundled version, and fix for me was:
brew uninstall --ignore-dependencies protobuf protobuf-c

[mnboos]

Might be fixed with c56b6f8

Would anyone like to try it from the dev branch?