Suggestions for Additional Interactive Apps to be added to OOD

[markyashar]

Regarding the discussion in today's (6/22/21) dev meeting on additional apps to consider adding to the OOD deployment/implementation on Savio:

In addition to soliciting feedback from users on what additional interactive apps should be included in OOD, some additional considerations should be given to what apps Savio users are currently using on the Savio viz node and if the viz node is retired, could some of those apps be moved to OOD?

It would be useful to review the meeting notes from an earlier dev meeting on OOD that took place in December 2020 here: https://docs.google.com/document/d/1C-5zW6_x9s-0bRDH0XTHEIOX-AcRp1I95vg1SW_fTfk/edit?usp=sharing

Some suggestions that have been discussed in meetings and that have been brought up in ServiceNow tickets/consultations and/or that appear in FCA account request applications include (and some of these are already supported and enabled by OOD/OSC -- see https://openondemand.org/ and https://osc.github.io/ood-documentation/latest/install-ihpc-apps.html):

• Agisoft Metashape/Photoscan (https://www.agisoft.com/)

• 3D slicer (https://www.slicer.org/)

• Mathematica (https://www.wolfram.com/mathematica/)

• Batch-Connect OSC Galaxy (https://github.com/OSC/bc_osc_galaxy)

• Paraview (https://www.paraview.org/)

• VisIT (https://wci.llnl.gov/simulation/computer-codes/visit/)

• COSMOL Multiphysics/Server (https://www.comsol.com/)

• Abaqus/CAE (https://www.3ds.com/products-services/simulia/products/abaqus/abaquscae/)

• Visual Molecular Dynamics (https://github.com/OSC/bc_osc_vmd)

• ANSYS Workbench (https://www.ansys.com/)

• Stata (https://github.com/OSC/bc_osc_stata)

• Tensorboard (https://github.com/stanford-rc/sh_ood-apps#sh_tensorboard)

• Strand7 -- Finite Element Analysis Software (https://www.strand7.com/)

• See also: https://github.com/OSC/Open-OnDemand.

Quesiton: If software is not free and requires a license on Savio, can it still be included as an interactive app on OOD, or is licensed (non-free) software off-limits for access as an interactive app through OOD?

Maybe we can check on what software is currently being used on the Savio viz node with something like:

ssh viz
[myashar@viz ~]$ top | grep -v root

%Cpu(s): 0.8 us, 0.9 sy, 0.0 ni, 98.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 65923752 total, 46181740 free, 10853744 used, 8888268 buff/cache
KiB Swap: 7312380 total, 2948688 free, 4363692 used. 54320444 avail Mem

19354 tasnuva 20 0 10.5g 981044 113564 S 12.5 1.5 5769:34 MATLAB
27891 oekenta 20 0 12.6g 2.7g 200984 S 4.3 4.2 431:01.78 MATLAB
14327 ann_sch+ 20 0 17.0g 3.8g 336796 S 3.0 6.1 679:53.72 MATLAB
27398 ann_sch+ 20 0 2781628 180712 27432 S 3.0 0.3 553:43.07 cef_helper
12074 azhou314 20 0 1955420 281580 66500 S 1.7 0.4 2:27.09 tensorboard
18771 tasnuva 20 0 253332 8980 7464 S 1.7 0.0 8:27.35 vncserverui
13936 myashar 20 0 58820 2620 1528 R 1.0 0.0 0:00.07 top
817 paciorek 20 0 283076 49416 1988 S 0.7 0.1 307:47.27 jupyter-noteboo
18493 cbachand 20 0 9887.2m 985.7m 118808 S 0.7 1.5 233:06.40 MATLAB
20782 cth 20 0 824296 299500 18700 S 0.7 0.5 230:17.83 firefox
12710 m_chint+ 20 0 359280 6896 4716 S 0.3 0.0 5:42.48 Xvnc-core
18739 tasnuva 20 0 354852 20732 7992 S 0.3 0.0 3:04.35 Xvnc-core

%Cpu(s): 1.3 us, 0.7 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 65923752 total, 46182248 free, 10853372 used, 8888132 buff/cache
KiB Swap: 7312380 total, 2948688 free, 4363692 used. 54320804 avail Mem

19354 tasnuva 20 0 10.5g 981864 113564 S 25.2 1.5 5769:35 MATLAB
27891 oekenta 20 0 12.6g 2.7g 200984 S 5.0 4.2 431:01.93 MATLAB
14327 ann_sch+ 20 0 17.0g 3.8g 336796 S 3.0 6.1 679:53.81 MATLAB
27398 ann_sch+ 20 0 2781628 180712 27432 S 3.0 0.3 553:43.16 cef_helper
18493 cbachand 20 0 9887.2m 985.7m 118808 S 0.7 1.5 233:06.42 MATLAB
817 paciorek 20 0 283076 49416 1988 S 0.3 0.1 307:47.28 jupyter-noteboo
5643 oekenta 20 0 2762772 22120 2116 S 0.3 0.0 74:27.65 MATLAB
13936 myashar 20 0 58820 2620 1528 R 0.3 0.0 0:00.08 top
20782 cth 20 0 824296 299500 18700 S 0.3 0.5 230:17.84 firefox
24317 oekenta 20 0 373592 35404 8004 S 0.3 0.1 50:15.87 Xvnc-core

%Cpu(s): 0.7 us, 0.7 sy, 0.0 ni, 98.4 id, 0.1 wa, 0.0 hi, 0.0 si, 0.0 st

or

[myashar@viz ~]$ ps -ef | grep -v root | more

UID PID PPID C STIME TTY TIME CMD
paciorek 812 1 0 May11 ? 00:00:00 /bin/sh -c module list; jupyter notebook --no-browser --ip=""
paciorek 817 812 0 May11 ? 05:07:48 /global/software/sl-7.x86_64/modules/langs/python/3.6/bin/python /global/software/sl-7.x86_64/modules/langs/python/3.6/bin/jupyter-notebook --no-browse
r --ip=
oekenta 1144 24344 0 May02 ? 00:00:01 xterm
oekenta 1146 1144 0 May02 pts/4 00:00:00 bash
oekenta 1850 1146 0 May02 pts/4 00:00:08 emacs
oekenta 1899 1850 0 May02 pts/5 00:00:09 /usr/bin/gdb /global/software/sl-7.x86_64/modules/tools/matlab/r2017b/bin/glnxa64/MATLAB
jacktse+ 2460 1 0 May16 ? 00:00:00 vncserver-virtual -geometry 1600x900
jacktse+ 2461 2460 0 May16 ? 00:02:21 /usr/bin/Xvnc-core :11 -auth /global/home/users/jacktseng/.Xauthority -pn -geometry 1600x900
jacktse+ 2481 2460 0 May16 ? 00:00:00 /bin/sh /etc/vnc/xstartup.custom
jacktse+ 2485 2481 0 May16 ? 00:00:00 xterm -geometry 80x24+10+10 -ls -title x11 Desktop
jacktse+ 2486 2481 0 May16 ? 00:00:00 /global/software/sl-6.x86_64/modules/tools/openbox/3.6.1/bin/openbox
jacktse+ 2490 2485 0 May16 pts/24 00:00:00 -bash
jacktse+ 2525 1 0 May16 ? 00:05:25 /usr/bin/vncserverui virtual 16
jacktse+ 2540 2525 0 May16 ? 00:00:07 /usr/bin/vncserverui -statusicon 5
rpc 4097 1 0 Apr21 ? 00:00:09 /sbin/rpcbind -w
dbus 4105 1 0 Apr21 ? 00:00:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
ntp 4176 1 0 Apr21 ? 00:00:10 /usr/sbin/ntpd -u ntp:ntp -g
munge 4895 1 0 Apr21 ? 00:08:10 /usr/sbin/munged
rpcuser 5055 1 0 Apr21 ? 00:00:00 /usr/sbin/rpc.statd
oekenta 5643 1899 0 May02 pts/5 01:14:27 /global/software/sl-7.x86_64/modules/tools/matlab/r2017b/bin/glnxa64/MATLAB -nojvm
oekenta 10091 1 0 May01 ? 00:00:00 dbus-launch --autolaunch=0e568ebb0af34e44879710be3e46ae97 --binary-syntax --close-stderr
oekenta 10092 1 0 May01 ? 00:00:00 /usr/bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
ann_sch+ 10599 1 0 Jun10 ? 00:00:00 vncserver
ann_sch+ 10600 10599 0 Jun10 ? 00:01:39 /usr/bin/Xvnc-core :7 -auth /global/home/users/ann_scheliga/.Xauthority -pn
ann_sch+ 10692 10599 0 Jun10 ? 00:00:00 /bin/sh /etc/vnc/xstartup.custom
ann_sch+ 10693 10692 0 Jun10 ? 00:00:00 xterm -geometry 80x24+10+10 -ls -title x11 Desktop
ann_sch+ 10694 10692 0 Jun10 ? 00:00:00 /global/software/sl-6.x86_64/modules/tools/openbox/3.6.1/bin/openbox
ann_sch+ 10696 10693 0 Jun10 pts/13 00:00:00 -bash
ann_sch+ 10737 1 0 Jun10 ? 00:00:17 /usr/bin/vncserverui virtual 17
ann_sch+ 10752 10737 0 Jun10 ? 00:00:00 /usr/bin/vncserverui -statusicon 5
azhou314 11225 1 0 15:09 ? 00:00:00 SCREEN -S viz
azhou314 11226 11225 0 15:09 pts/20 00:00:00 /bin/bash
jacktse+ 12038 1 0 May14 ? 00:00:00 Xvfb :99 -screen 0 1024x768x24 +extension GLX +render -noreset
azhou314 12074 11226 1 15:09 pts/20 00:02:29 /global/home/users/azhou314/.conda/envs/perception/bin/python /global/home/users/azhou314/.conda/envs/perception/bin/tensorboard --logdir logdirDIMEX/
--port 1234
ioannis 12318 1 0 May05 ? 00:00:00 vncserver
ioannis 12319 12318 0 May05 ? 00:02:21 /usr/bin/Xvnc-core :6 -auth /global/home/users/ioannis/.Xauthority -pn
ioannis 12340 12318 0 May05 ? 00:00:00 /bin/sh /etc/vnc/xstartup.custom
ioannis 12345 12340 0 May05 ? 00:00:03 xterm -geometry 80x24+10+10 -ls -title x11 Desktop
ioannis 12346 12340 0 May05 ? 00:00:00 /global/software/sl-6.x86_64/modules/tools/openbox/3.6.1/bin/openbox
ioannis 12350 12345 0 May05 pts/12 00:00:00 -bash
ioannis 12379 1 0 May05 ? 00:01:08 /usr/bin/vncserverui virtual 22
ioannis 12394 12379 0 May05 ? 00:00:01 /usr/bin/vncserverui -statusicon 5
m_chint+ 12709 1 0 May05 ? 00:00:00 vncserver
m_chint+ 12710 12709 0 May05 ? 00:05:42 /usr/bin/Xvnc-core :5 -auth /global/home/users/m_chintalapati/.Xauthority -pn
m_chint+ 12739 12709 0 May05 ? 00:00:00 /bin/sh /etc/vnc/xstartup.custom
m_chint+ 12742 12739 0 May05 ? 00:00:01 /global/software/sl-6.x86_64/modules/tools/openbox/3.6.1/bin/openbox
m_chint+ 12784 1 0 May05 ? 00:01:12 /usr/bin/vncserverui virtual 17
m_chint+ 12799 12784 0 May05 ? 00:00:01 /usr/bin/vncserverui -statusicon 5
jacktse+ 14050 1 0 May15 ? 00:00:00 vncserver -geometry 1600x900
jacktse+ 14051 14050 0 May15 ? 00:02:05 /usr/bin/Xvnc-core :10 -auth /global/home/users/jacktseng/.Xauthority -pn -geometry 1600x900
jacktse+ 14070 14050 0 May15 ? 00:00:00 /bin/sh /etc/vnc/xstartup.custom
jacktse+ 14075 14070 0 May15 ? 00:00:00 xterm -geometry 80x24+10+10 -ls -title x11 Desktop

[markyashar]

Additional links to interactive apps that could be included in OOD:

• Paraview (https://github.com/OSC/bc_osc_paraview)
• COMSOL Multiphysics (https://github.com/OSC/bc_osc_comsol)
• Abaqus/CAE (https://github.com/OSC/bc_osc_abaqus)
• ANSYS Workbench (https://github.com/OSC/bc_osc_ansys_workbench)
• Shiny App (https://github.com/CHPC-UofU/bc_osc_example_shiny)
• QGIS (https://github.com/OSC/bc_osc_qgis)

This slide deck from an Open OnDemand presentation might be of interest: https://drive.google.com/file/d/1sj1Hjt_ywVtKfva2wDth-_HZz7sRL7D0/view?usp=sharing

[markyashar]

In regards to the issue of monitoring which modules and applications are being used on the Savio viz node to help determine which interactive apps to add to OOD, this Ask.CI forum thread ("Slurm Reports: Frequency of Application/Module Use") may be of interest: https://ask.cyberinfrastructure.org/t/slurm-reports-frequency-of-application-module-use/1947

[markyashar]

Some additional ideas and suggestions for interactive apps to add to OOD:

(1) TensorBoard -- see here also for examples of implementation of TensorBoard on OOD:

https://discourse.openondemand.org/t/tensorboard-for-open-ondemand/340

https://github.com/stanford-rc/sh_ood-apps/tree/main/sh_tensorboard

https://github.com/zPianodude/ood-apps

(2) It might also be worthwhile to explore photogrammetry workflows GUI on OOD (Chris H.) when the OOD desktop app is available, etc., i.e., Agisoft Metashape: https://www.agisoft.com/. For example, see:

https://docs.google.com/document/d/1OQtpA7SSCyWOiFov-aDjC4TPWrcmCtJD/edit?usp=sharing&ouid=115066181203367192083&rtpof=true&sd=true

and

https://drive.google.com/drive/folders/1JErpljAvnTE_0RwoTReIh02ft9awcQV0?usp=sharing

(3) It could be useful to consider adding Globus itself (if at all possible), and other data transfer tools such as Filezilla, Cyberduck, or the rclone GUI, as an interactive app to Open OnDemand, and/or via direct integration with the Open OnDemand Files app [1]. This might provide an even more convenient way for Savio users to carry out large data transfers to and from Savio via Globus, though I'm not sure if this capability currently exists within OOD at this time. (The same can be considered for any future SRDC OOD implementations).

[1] It looks like the OOD development team has been working on online integration of Globus with the Files app, but I'm not sure if it has actually been successfully implemented yet:

https://discourse.openondemand.org/t/scheduling-of-open-ondemand-2-0-release/1205

https://trello.com/c/rcnBbBUZ/51-globus-online-integration

https://trello.com/c/iPKFaOx1/45-files-app-replacement