Bootstrapping IPM from artifact ignores requirements.txt

[isc-shuliu]

During recent meetings with Eric (from IPE) and @isc-jili , we discovered that when loading the installer artifact of IPM 0.10.0 (e.g., zpm-0.10.0-beta.3.xml) on a fresh iris instance, the python requirements.txt is ignored, causing all import oras calls to fail.

However, manually installation from source code works fine. Namely,

do $System.OBJ.Load("/<ipm-repo-root>/preload/cls/IPM/Installer.cls","ck")
do ##class(IPM.Installer).setup("/<ipm-repo-root>/", 3)

Something in the bootstrap process is missing.

[isc-shuliu]

The artifacts, e.g. zpm-0.10.0-beta.3.xml, do not contain any information from requirements.txt (as expected).

Unless there's a way to attach arbitrary files into the xml artifact, we may need to hard code the python dependency installation into the code if we want to keep using the .xml artifact for one-liner installation.

[isc-kiyer]

@isc-shuliu This got me thinking about what would happen in an environment that has no internet access. How would it install python dependencies? Could the oras python package be packaged up at time of publishing the zpm artifact? Also, unrelated note but if we are changing the artifact type from xml, perhaps also rename it to ipm?

cc @isc-tleavitt @isc-eneil @isc-jili @isc-jlechtne

[isc-kiyer]

Useful post for downloading python packages for packaging into IPM distributable: https://stackoverflow.com/questions/49333152/copy-installed-packages-using-pip-to-another-environment

[isc-eneil]

@isc-shuliu The work-around mentioned is not working for me (when trying with stakeholder devs). I was on a 0.10.0-beta.11 version and tried loading from the source code zip file on the Releases page. It fails when it hits the reload phase and tries to load requirements.txt with this error:

[USER|ZPM] Initialize SUCCESS
[USER|ZPM] Reload START (/Users/jlou/Downloads/ipm-0.10.0-beta.11/)
[USER|zpm] requirements.txt START
[USER|zpm] requirements.txt FAILURE
[ZPM] Reload FAILURE
ERROR! ObjectScript error: InstallPythonRequirements+25^%IPM.Lifecycle.Base.1 *Failed to Load Python: Check documentation and messages.log, Check CPF parameters:[PythonRuntimeLibrary,PythonRuntimeLibraryVersion], Check sys.path setup in: $INSTANCE/lib/python/iris_site.py2025-01-30 15:38:23 0 IPM.Installer: ERROR #5002: ObjectScript error: InstallPythonRequirements+25^%IPM.Lifecycle.Base.1 *Failed to Load Python: Check documentation and messages.log, Check CPF parameters:[PythonRuntimeLibrary,PythonRuntimeLibraryVersion], Check sys.path setup in: $INSTANCE/lib/python/iris_site.py
2025-01-30 15:38:23 0 IPM.Installer: Installation failed at 2025-01-30 15:38:23
2025-01-30 15:38:23 0 %Installer: Elapsed time 2.66866s

[isc-shuliu]

@isc-shuliu The work-around mentioned is not working for me (when trying with stakeholder devs). I was on a 0.10.0-beta.11 version and tried loading from the source code zip file on the Releases page. It fails when it hits the reload phase and tries to load requirements.txt with this error:

[USER|ZPM] Initialize SUCCESS [USER|ZPM] Reload START (/Users/jlou/Downloads/ipm-0.10.0-beta.11/) [USER|zpm] requirements.txt START [USER|zpm] requirements.txt FAILURE [ZPM] Reload FAILURE ERROR! ObjectScript error: InstallPythonRequirements+25^%IPM.Lifecycle.Base.1 *Failed to Load Python: Check documentation and messages.log, Check CPF parameters:[PythonRuntimeLibrary,PythonRuntimeLibraryVersion], Check sys.path setup in: $INSTANCE/lib/python/iris_site.py2025-01-30 15:38:23 0 IPM.Installer: ERROR #5002: ObjectScript error: InstallPythonRequirements+25^%IPM.Lifecycle.Base.1 *Failed to Load Python: Check documentation and messages.log, Check CPF parameters:[PythonRuntimeLibrary,PythonRuntimeLibraryVersion], Check sys.path setup in: $INSTANCE/lib/python/iris_site.py 2025-01-30 15:38:23 0 IPM.Installer: Installation failed at 2025-01-30 15:38:23 2025-01-30 15:38:23 0 %Installer: Elapsed time 2.66866s

Does your instance have embedded python enabled? It seems the error stems from this line

Set tSysModule = ##class(%SYS.Python).Import("sys")

which should work as long as embedded python is available.

As a sanity check, can you try running

USER>:py

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
Type quit() or Ctrl-D to exit this shell.
>>> import sys
>>> sys.__loader__
<class '_frozen_importlib.BuiltinImporter'>

[isc-shuliu]

@isc-tleavitt
After discussion with HSOPS, @isc-kiyer suggested 2 possibles solutions.

1. Create a new StudioDocument class that represents the python wheel file (which is a zip under the hood) and attach it to the XML installer. Not sure how much work will be involved.

2. Publish zpm as a tarball instead, and modify the existing one-liner installation command to use the tarball instead of XML. It's easier but may break some backward compatibility for users relying on the one-liner as part of the workflow.

Which way do you recommend?