Investigate and Implement Integration of CyberGIS with CIROH JupyterHub

[arpita0911patel]

No description provided.

[alexandermichels]

Generally, we need:

• the URL of the JupyterHub
• the ID of the Globus endpoint to transfer data to/from the JupyterHub. The Globus user [email protected] will also need permissions on the endpoint.
• the path on the endpoint where user data is located (so if user data is at /data/user_data/<USERNAME>, we want /data/user_data)
• the path within the Jupyter container to transfer data to (usually /home/jovyan/ or /home/jovyan/work)

Examples here: https://github.com/cybergis/cybergis-compute-core/blob/v2/configs/jupyter-globus-map.example.json

We also need to know (for file transfer purposes) what format your user data folders are. For example, our I-GUIDE Jupyterhub uses the pattern i-guide-claim-<USERNAME> where (https://github.com/cybergis/cybergis-compute-core/blob/v2/src/helpers/python/globus_user_mapping.py). We default to using the username with only ascii lowercase and numeric digits, but if you have some other pattern we can add it to our user mapping.

[benlee0423]

@alexandermichels
Where "path on the endpoint where user data is located" is defined?
Maybe in the below location?
https://gitlab.nrp-nautilus.io/prp/globus-connect/-/blob/master/globus-connect.yaml?ref_type=heads#L35
Is this path accessible from jupyter instance?

[benlee0423]

@alexandermichels
Globus endpoint has to be setup in the same namespace where jupyter hub is installed in k8s?
We have staging and prod namespace in k8s, where staging and prod resources for jupyterhub are installed.

[alexandermichels]

I haven't deployed globus with K8s, but based on the README in the repo it appears the data is stored in /data/gridftp-save/. The Jupyter user containers don't need to know about this path, but the Globus transfer is being executed by the Globus pod so we need to tell it where to put the files.

I don't think the namespace matters for Compute as long as you can make the data accessible to both the Globus pod and the user Jupyter pod, but I am not an expert on PVs/PVCs.

[benlee0423]

@alexandermichels
I have emailed the information requested for testing purpose.
This information will be different what we will have in our prod k8s cluster.

Now, I have assigned to subscription status, and please let me know how to add the Globus user [email protected] to the endpoint.