Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dependencies to latest version #384

Open
koettert opened this issue Jan 21, 2025 · 6 comments
Open

Update dependencies to latest version #384

koettert opened this issue Jan 21, 2025 · 6 comments
Assignees
Labels
Milestone

Comments

@koettert
Copy link
Contributor

koettert commented Jan 21, 2025

  • Update Python library to at least 3.11 or whatever version is supported by KNIME
  • LibPysal
  • ...
@koettert koettert added the Todo label Jan 21, 2025
@koettert koettert added this to the Release 1.4 milestone Jan 21, 2025
@koettert koettert changed the title Update Python to 3.x Update dependencies to latest version Jan 21, 2025
@koettert koettert assigned koettert and wybert and unassigned koettert Feb 4, 2025
@wybert
Copy link
Contributor

wybert commented Feb 8, 2025

  1. The least Python version supported by KNIME is 3.11
  2. In our previous dependencies it says numpy=1.23.5 #required to fix problem with latest version of numpy and curl=8.1.2 #required by libgdal which includes an older version with vulnerabilities, do have have the nodes with these problems. especially for curl. It's better to have these nodes and tested if they got fixed by least version of the package
  3. conda doesn't have the least version of keplergl, it can be installed by pip and pip will install ipykernel, so we can remove ipykernel depandacy
  4. Below is some of the least version packages we can get when specify curl and numpy version. This way we will still use the same version of geopandas, if you can use least version of curl, then we can use the least version of geopandas.
name: geospatial_env
channels:
  - knime
  - conda-forge
dependencies:
  - python=3.10
  - knime-extension
  - knime-python-base
  - packaging
  - curl=8.1.2 #required by libgdal which includes an older version with vulnerabilities
  - cvxopt
  - esda
  - fiona
  - folium
  - geopandas=0.14.4
  - geopy
  - h3-py
  - jmespath
  - libgdal
  - libpysal=4.12.1
  - mgwr
  - numpy=1.23.5 #required to fix problem with latest version of numpy
  - osmnx
  - polyline
  - pyproj
  - pysal=24.1
  - rasterio
  - seaborn
  - shapely
  - sympy
  - pip
  - pygeoda
  - pointpats
  - pip:
    - ipinfo
    - pulp
    - keplergl

@koettert
Copy link
Contributor Author

In general for now try to update as many dependencies as possible independent of the existing comments in the dependency file since they might no longer hold if we update the major dependencies.

In our previous dependencies it says numpy=1.23.5 #required to fix problem with latest version of numpy and curl=8.1.2 #required by libgdal which includes an older version with vulnerabilities, do have have the nodes with these problems. especially for curl. It's better to have these nodes and tested if they got fixed by least version of the package

For numpy we should update to the latest supported version and see if the nodes are still executing fine.

For curl the problem was that we couldn't update libgdal which had a dependency to a curl version with a security problem. However if we can update libgdal we should try to update curl as well to its latest version and check that this version has no known critical security problems.

@wybert
Copy link
Contributor

wybert commented Feb 28, 2025

The nodes affected by the updates,

  • spatial weight
  • spatial heatmap
  • h3 create grid
  • SLCP
  • Buffer
  • RoadNetworkDistanceMatrix

@wybert
Copy link
Contributor

wybert commented Feb 28, 2025

The spatial heatmap is a bug from folium, see python-visualization/folium#2098

@wybert
Copy link
Contributor

wybert commented Feb 28, 2025

The spatial weight can be fixed and could work on offline environment by removing the mock import.

@wybert
Copy link
Contributor

wybert commented Mar 1, 2025

For LocationAnalysis category. Most of the nodes has problems because of the pulp updates. Some thing like below,

These works well when pulp==2.7.0 but not with the latest version of pulp==3.0.2

WARN P-median 3:14 Traceback (most recent call last):
File "/Applications/KNIME 5.4.0.app/Contents/Eclipse/plugins/org.knime.python3.nodes_5.4.1.v202501291500/src/main/python/_node_backend_launcher.py", line 1055, in execute
outputs = self._node.execute(exec_context, inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Applications/KNIME 5.4.0.app/Contents/Eclipse/plugins/org.knime.python3.nodes_5.4.1.v202501291500/src/main/python/knime/extension/nodes.py", line 1237, in wrapper
results = func(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/kang/GitHub/knime-geospatial-extension/knime_extension/src/nodes/locationanalysis.py", line 313, in execute
problem.solve()
File "/opt/homebrew/Caskroom/miniforge/base/envs/geospatial_env/lib/python3.11/site-packages/pulp/pulp.py", line 1996, in solve
status = solver.actualSolve(self, kwargs)
^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'actualSolve'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants