misc debug notes do not open.txt

#This is a helper document I'm writing to assist in debugging dependency installations for the cacto-sl code.
#Nadim Kanazi
#June 2024
________________________________________________________________________________________________

The general approach was this for starters:

1. Clone the cacto repo (https://github.com/gianluigigrandesso/cacto)
2. Follow the instructions below to run the code. These instructions were provided by ELisa Alboni:
  - create and activate a new virtual environment (python3.10.12) via conda
  - python3 -m pip install --upgrade pip
  - python3 -m pip install tensorflow==2.11
  - git clone https://github.com/nmansard/jnrh2023
  - cd jnrh2023
  - python3 -m pip install example-robot-data-jnrh2023 jupyterlab meshcat scipy ipywidgets matplotlib
  - python3 -m pip install tf_siren
  
Try running a test after these installs to check if the environment is working:

python3 main.py --system-id='single_integrator' --seed=0 --nb-cpus=15 --w-S=1e-2 --test-n=0
________________________________________________________________________________________________

#Issues I ran into:
 
 1- Python/tensorflow versions, pygame, and stable-baselines
 
 These are a bunch of closely related issues with versions of python and of these packages. The best I can explain it is as follows:

When first running the tests directly after following the instructions above, this is the first error:
 - File "/home/a2rlab2/.local/lib/python3.10/site-packages/stable_baselines/deepq/policies.py", line 2, in <module>
    import tensorflow.contrib.layers as tf_layers
ModuleNotFoundError: No module named 'tensorflow.contrib'

The stable-baselines package attempted to access the tensorflow.contrib module, but this module was removed from tensorflow 2.X and is only present in 1.X versions. So naturally an easy solution is to downgrade tensorflow to an earlier version. This is corroborated by the stable-baselines documentation, which states that stable-baselines does not support tensorflow 2.X and is only compatible with tensorflow 1.8 to 1.13:
 https://stable-baselines.readthedocs.io/en/master/guide/install.html
  
 The next issue when trying to downgrade is that tensorflow 1.X does not exist in python 3.8 or later. Python 3.7 or earlier is needed for this. I tried to create a new conda environment with python 3.7 and follow the instructions again, but then the following command does not work:

python3 -m pip install example-robot-data-jnrh2023 jupyterlab meshcat scipy ipywidgets matplotlib

so I went directly to the jnrh2023 repo and followed their instructions, which require pinocchio 2.99. But pinocchio 2.99 is only available in python >= 3.8, so this entire approach would not work at all.
 

 2- Pinocchio and Casadi interface issues:
 
  Error:
 ModuleNotFoundError: No module named 'pinocchio.casadi'

Explanation: It seems only pinocchio 3 is compatible with casadi as earlier versions are not. Pinocchio 3 can be built from this branch:
https://github.com/stack-of-tasks/pinocchio/tree/pinocchio3-preview

I believe Pinnochio3 is not available to install using pip or conda. Here are a few helpful links:

https://gitlab.cc-asp.fraunhofer.de/hanikevi/contact_mpc
https://github.com/nmansard/jnrh2023
https://github.com/ami-iit/ami-commons/blob/master/doc/casadi-ipopt-hsl.md

I tried building directly from the branch but got a lot of memory-related errors.

 3- requirements.txt file:
 
 ERROR: Could not find a version that satisfies the requirement [some package] (from versions: none)
ERROR: No matching distribution found for [some package]

I don't understand this error and what the packages listed in requirements.txt are really. I tried looking up some of them but haven't found any info about some (ex: ament-cmake, section-pkgs, action-msgs). I tried installing them with pip and coda but no luck in either. They may be ROS packages, here is the info I found online:
https://index.ros.org/search/?term=ament-cmake-test&section=pkgs
https://index.ros.org/search/?term=action-msgs&section=pkgs

Note: this command is helpful to prevent pip from failing on a single line of requirements.txt

cat requirements.txt | xargs -n 1 pip install


4- Other misc errors: protobuf, pygame, wheel:

Unfortunately (or fortunately), I haven't been able to replicate these errors since last week but I was having some trouble at one point with these packages. I don't recall when or how though. Please ignore the call stack below as it is only for my personal debugging use

Building wheels for collected packages: pygame
  Building wheel for pygame (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [44 lines of output]
      src_c/imageext.c: In function ‘image_load_ext’:
      src_c/imageext.c:118:13: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
        118 |         ext = find_extension(name);
            |             ^
      src_c/imageext.c:107:9: warning: variable ‘lock_mutex’ set but not used [-Wunused-but-set-variable]
        107 |     int lock_mutex = 0;
            |         ^~~~~~~~~~
      src_c/font.c: In function ‘font_dealloc’:
      src_c/font.c:693:29: warning: initialization of ‘int **’ from incompatible pointer type ‘TTF_Font *’ {aka ‘struct _TTF_Font *’} [-Wincompatible-pointer-types]
        693 |             int** face_pp = font;
            |                             ^~~~
      At top level:
      src_c/font.c:78:1: warning: ‘utf_8_needs_UCS_4’ defined but not used [-Wunused-function]
         78 | utf_8_needs_UCS_4(const char *str)
            | ^~~~~~~~~~~~~~~~~
      src_c/music.c: In function ‘_get_type_from_hint’:
      src_c/music.c:307:22: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
        307 |             namehint = dot + 1;
            |                      ^
      In file included from src_c/pygame.h:30,
                       from src_c/music.c:27:
      src_c/music.c: In function ‘_load_music’:
      src_c/_pygame.h:253:47: warning: returning ‘PyObject *’ {aka ‘struct _object *’} from a function with incompatible return type ‘Mix_Music *’ {aka ‘struct _Mix_Music *’} [-Wincompatible-pointer-types]
        253 | #define RAISE(x, y) (PyErr_SetString((x), (y)), (PyObject *)NULL)
            |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~
      src_c/mixer.h:9:16: note: in expansion of macro ‘RAISE’
          9 |         return RAISE(pgExc_SDLError, "mixer not initialized")
            |                ^~~~~
      src_c/music.c:377:5: note: in expansion of macro ‘MIXER_INIT_CHECK’
        377 |     MIXER_INIT_CHECK();
            |     ^~~~~~~~~~~~~~~~
      src_c/pypm.c:209:12: fatal error: longintrepr.h: No such file or directory
        209 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      
      ---
      For help with compilation see:
          https://www.pygame.org/wiki/Compilation
      To contribute to pygame development see:
          https://www.pygame.org/contribute.html
      ---
      
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygame
  Running setup.py clean for pygame
Failed to build pygame
ERROR: Could not build wheels for pygame, which is required to install pyproject.toml-based projects
(cacto-env) a2rlab2@a2rlab-lenovo-1:~$ sudo apt-get install python-edv

________________________________________________________________________________________________


Solved by downloading segment_tree from openai baselines file and changing import line within replay_buffer.py


______________________________________________________________________________________________
requirements.txt:
absl-py==2.1.0
action-msgs==1.6.0
action-tutorials-interfaces==0.27.1
action-tutorials-py==0.27.1
actionlib-msgs==5.0.0
ament-cmake-test==2.0.3
ament-copyright==0.14.2
ament-cppcheck==0.14.2
ament-cpplint==0.14.2
ament-flake8==0.14.2
ament-index-python==1.5.2
ament-lint==0.14.2
ament-lint-cmake==0.14.2
ament-package==0.15.3
ament-pep257==0.14.2
ament-uncrustify==0.14.2
ament-xmllint==0.14.2
angles==1.16.0
anyio==4.3.0
appdirs==1.4.4
apturl==0.5.2
argcomplete==1.8.1
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asttokens==2.4.1
astunparse==1.6.3
async-lru==2.0.4
attrs==23.2.0
Babel==2.14.0
beautifulsoup4==4.10.0
beniget==0.4.1
bleach==6.1.0
blinker==1.4
Brlapi==0.8.3
Brotli==1.0.9
builtin-interfaces==1.6.0
cachetools==5.3.3
casadi==3.6.3
catkin-pkg==1.0.0
catkin-pkg-modules==1.0.0
certifi==2020.6.20
cffi==1.16.0
chardet==4.0.0
charset-normalizer==3.3.2
click==8.0.3
cmeel==0.53.3
cmeel-assimp==5.2.5
cmeel-boost==1.81.0
cmeel-console-bridge==1.0.2.1
cmeel-eigen==3.4.0.1
cmeel-octomap==1.9.8.1
cmeel-tinyxml==2.6.2.1
cmeel-urdfdom==3.1.0.2
cmeel-urdfdom-headers==1.1.0
colcon-argcomplete==0.3.3
colcon-bash==0.5.0
colcon-cd==0.1.1
colcon-cmake==0.2.28
colcon-common-extensions==0.3.0
colcon-core==0.15.1
colcon-defaults==0.2.8
colcon-devtools==0.2.4
colcon-installed-package-information==0.1.0
colcon-library-path==0.2.1
colcon-metadata==0.2.5
colcon-notification==0.2.15
colcon-output==0.2.13
colcon-override-check==0.0.1
colcon-package-information==0.3.3
colcon-package-selection==0.2.10
colcon-parallel-executor==0.3.0
colcon-pkg-config==0.1.0
colcon-powershell==0.4.0
colcon-python-setup-py==0.2.8
colcon-recursive-crawl==0.2.3
colcon-ros==0.4.1
colcon-test-result==0.3.8
colcon-zsh==0.5.0
colorama==0.4.4
comm==0.2.2
command-not-found==0.3
composition-interfaces==1.6.0
control-msgs==5.0.0
controller-manager-msgs==3.22.0
cov-core==1.15.0
coverage==6.2
cryptography==3.4.8
cupshelpers==1.0
cv-bridge==3.4.0
cycler==0.11.0
dbus-python==1.2.18
debugpy==1.8.1
decorator==4.4.2
defer==1.0.6
defusedxml==0.7.1
demo-nodes-py==0.27.1
diagnostic-msgs==5.0.0
distlib==0.3.4
distro==1.7.0
distro-info==1.1+ubuntu0.2
docutils==0.17.1
domain-coordinator==0.11.2
empy==3.3.4
example-interfaces==0.10.2
example-robot-data-jnrh2023==4.0.7
examples-rclpy-executors==0.18.0
examples-rclpy-minimal-action-client==0.18.0
examples-rclpy-minimal-action-server==0.18.0
examples-rclpy-minimal-client==0.18.0
examples-rclpy-minimal-publisher==0.18.0
examples-rclpy-minimal-service==0.18.0
examples-rclpy-minimal-subscriber==0.18.0
exceptiongroup==1.2.0
executing==2.0.1
fastjsonschema==2.19.1
flake8==4.0.1
flatbuffers==24.3.25
fonttools==4.29.1
fqdn==1.5.1
fs==2.4.12
gast==0.4.0
geometry-msgs==5.0.0
google-auth==2.29.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.62.1
h11==0.14.0
h5py==3.11.0
html5lib==1.1
httpcore==1.0.5
httplib2==0.20.2
httpx==0.27.0
idna==3.3
image-geometry==3.4.0
importlib-metadata==4.6.4
iniconfig==1.1.1
interactive-markers==2.4.0
ipykernel==6.29.4
ipython==8.23.0
ipywidgets==8.1.2
isoduration==20.11.0
jedi==0.19.1
jeepney==0.7.1
Jinja2==3.1.3
joint-state-publisher==2.4.0
joint-state-publisher-gui==2.4.0
json5==0.9.24
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter_client==8.6.1
jupyter_core==5.7.2
jupyter_server==2.14.0
jupyter_server_terminals==0.5.3
jupyterlab==4.1.6
jupyterlab_pygments==0.3.0
jupyterlab_server==2.26.0
jupyterlab_widgets==3.0.10
keras==2.11.0
keyring==23.5.0
kiwisolver==1.3.2
language-selector==0.1
lark==1.1.1
laser-geometry==2.5.0
launch==2.0.2
launch-ros==0.24.0
launch-testing==2.0.2
launch-testing-ros==0.24.0
launch-xml==2.0.2
launch-yaml==2.0.2
launchpadlib==1.10.16
lazr.restfulclient==0.14.4
lazr.uri==1.0.6
libclang==18.1.1
lifecycle-msgs==1.6.0
logging-demo==0.27.1
louis==3.20.0
lxml==4.8.0
lz4==3.1.3+dfsg
macaroonbakery==1.3.1
map-msgs==2.2.0
Markdown==3.6
MarkupSafe==2.1.5
matplotlib==3.5.1
matplotlib-inline==0.1.6
mccabe==0.6.1
meshcat==0.3.2
message-filters==4.7.0
mistune==3.0.2
more-itertools==8.10.0
mpi4py==3.1.3
mpmath==0.0.0
nav-msgs==5.0.0
nbclient==0.10.0
nbconvert==7.16.3
nbformat==5.10.4
nest-asyncio==1.6.0
netifaces==0.11.0
nose2==0.9.2
notebook_shim==0.2.4
notify2==0.3
numpy==1.24.0
oauthlib==3.2.0
olefile==0.46
opt-einsum==3.3.0
osrf-pycommon==2.1.2
overrides==7.7.0
packaging==24.0
pandocfilters==1.5.1
parso==0.8.4
pcl-msgs==1.0.0
pendulum-msgs==0.27.1
pexpect==4.8.0
Pillow==9.0.1
pin3x-jnrh2023==2.9.2
platformdirs==4.2.0
pluggy==0.13.0
ply==3.11
prometheus_client==0.20.0
prompt-toolkit==3.0.43
protobuf==3.12.4
psutil==5.9.0
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.10.0
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycairo==1.20.1
pycodestyle==2.8.0
pycparser==2.22
pycups==2.0.1
pydocstyle==6.1.1
pydot==1.4.2
pyflakes==2.4.0
Pygments==2.11.2
PyGObject==3.42.1
PyJWT==2.3.0
pymacaroons==0.13.0
PyNaCl==1.5.0
pyngrok==7.1.6
pyparsing==2.4.7
PyQt5==5.15.6
PyQt5-sip==12.9.1
pyRFC3339==1.1
pytest==6.2.5
pytest-cov==3.0.0
python-apt==2.4.0+ubuntu2
python-dateutil==2.9.0.post0
python-debian==0.1.43+ubuntu1.1
python-json-logger==2.0.7
python-qt-binding==1.2.3
pythran==0.10.0
pytz==2022.1
pyxdg==0.27
PyYAML==5.4.1
pyzmq==25.1.2
qt-dotgraph==2.4.2
qt-gui==2.4.2
qt-gui-cpp==2.4.2
qt-gui-py-common==2.4.2
quality-of-service-demo-py==0.27.1
rcl-interfaces==1.6.0
rclpy==4.1.4
rcutils==6.2.2
referencing==0.34.0
reportlab==3.6.8
requests==2.31.0
requests-oauthlib==2.0.0
resource-retriever==3.2.2
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rmw-dds-common==2.0.1
roman==3.3
ros-image-to-qimage==0.4.1
ros2action==0.25.4
ros2bag==0.22.5
ros2bag-mcap-cli==0.22.5
ros2bag-sqlite3-cli==0.22.5
ros2cli==0.25.4
ros2component==0.25.4
ros2doctor==0.25.4
ros2interface==0.25.4
ros2launch==0.24.0
ros2lifecycle==0.25.4
ros2multicast==0.25.4
ros2node==0.25.4
ros2param==0.25.4
ros2pkg==0.25.4
ros2run==0.25.4
ros2service==0.25.4
ros2topic==0.25.4
rosbag2-interfaces==0.22.5
rosbag2-py==0.22.5
rosdep==0.21.0
rosdistro==0.9.0
rosdistro-modules==0.9.0
rosgraph-msgs==1.6.0
rosidl-adapter==4.0.1
rosidl-cli==4.0.1
rosidl-cmake==4.0.1
rosidl-generator-c==4.0.1
rosidl-generator-cpp==4.0.1
rosidl-generator-py==0.18.0
rosidl-generator-type-description==4.0.1
rosidl-parser==4.0.1
rosidl-pycommon==4.0.1
rosidl-runtime-py==0.12.0
rosidl-typesupport-c==3.0.1
rosidl-typesupport-cpp==3.0.1
rosidl-typesupport-fastrtps-c==3.0.2
rosidl-typesupport-fastrtps-cpp==3.0.2
rosidl-typesupport-introspection-c==4.0.1
rosidl-typesupport-introspection-cpp==4.0.1
rospkg==1.5.0
rospkg-modules==1.5.0
rpds-py==0.18.0
rpyutils==0.3.2
rqt==1.3.3
rqt-action==2.1.2
rqt-bag==1.3.4
rqt-bag-plugins==1.3.4
rqt-console==2.1.1
rqt-controller-manager==3.22.0
rqt-gauges==0.0.1
rqt-graph==1.4.2
rqt-gui==1.3.3
rqt-gui-py==1.3.3
rqt-joint-trajectory-controller==3.21.0
rqt-moveit==1.0.1
rqt-msg==1.3.1
rqt-plot==1.2.3
rqt-publisher==1.6.3
rqt-py-common==1.3.3
rqt-py-console==1.1.1
rqt-reconfigure==1.3.4
rqt-robot-dashboard==0.5.8
rqt-robot-monitor==1.0.5
rqt-robot-steering==1.0.0
rqt-runtime-monitor==1.0.0
rqt-service-caller==1.1.1
rqt-shell==1.1.1
rqt-srv==1.1.1
rqt-tf-tree==1.0.4
rqt-topic==1.6.1
rsa==4.9
scipy==1.8.0
SecretStorage==3.3.1
Send2Trash==1.8.3
sensor-msgs==5.0.0
sensor-msgs-py==5.0.0
service-msgs==1.6.0
shape-msgs==5.0.0
six==1.16.0
sniffio==1.3.1
snowballstemmer==2.2.0
soupsieve==2.3.1
sros2==0.11.3
ssh-import-id==5.11
stack-data==0.6.3
statistics-msgs==1.6.0
std-msgs==5.0.0
std-srvs==5.0.0
stereo-msgs==5.0.0
sympy==1.9
systemd-python==234
teleop-twist-keyboard==2.3.2
tensorboard==2.11.2
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-io-gcs-filesystem==0.36.0
termcolor==2.4.0
terminado==0.18.1
tf-siren==0.0.5
tf2-geometry-msgs==0.31.5
tf2-kdl==0.31.5
tf2-msgs==0.31.5
tf2-py==0.31.5
tf2-ros-py==0.31.5
tf2-sensor-msgs==0.31.5
tf2-tools==0.31.5
tinycss2==1.2.1
toml==0.10.2
tomli==2.0.1
topic-monitor==0.27.1
tornado==6.4
tqdm==4.66.2
traitlets==5.14.2
trajectory-msgs==5.0.0
turtle-tf2-py==0.3.6
turtlesim==1.6.1
type-description-interfaces==1.6.0
types-python-dateutil==2.9.0.20240316
typing_extensions==4.11.0
u-msgpack-python==2.8.0
ubuntu-advantage-tools==8001
ubuntu-drivers-common==0.0.0
ufoLib2==0.13.1
ufw==0.36.1
unattended-upgrades==0.1
unicodedata2==14.0.0
unique-identifier-msgs==2.3.2
uri-template==1.3.0
urllib3==1.26.5
visualization-msgs==5.0.0
wadllib==1.3.6
wcwidth==0.2.13
webcolors==1.13
webencodings==0.5.1
websocket-client==1.7.0
Werkzeug==3.0.2
widgetsnbextension==4.0.10
wrapt==1.16.0
xacro==2.0.9
xdg==5
xkit==0.0.0
zipp==1.0.0

_______________________________________________________________________________________________
jun 28 notes:
python3 main.py --system-id='double_integrator' --seed=0 --nb-cpus=15 --w-S=1e-2 --test-n=53 --GPU-flag
on gpu 2257s
on cpu