Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 transition #215

Closed
wants to merge 23 commits into from
Closed

Python 3 transition #215

wants to merge 23 commits into from

Conversation

sfalexrog
Copy link
Member

@sfalexrog sfalexrog commented Feb 13, 2020

(I am not comfortable even opening this P/R, but I guess we should do this eventually)

As of 2020, Python 2 is no longer maintained. Package registries are working for now, but new software should use Python 3 instead.
This is concerning for us, since Clever (Clover?) is an educational platform, and teaching how to use deprecated tools is not our goal. As such, we are transitioning to Python 3.

This transition is somewhat complicated:

  • ROS Melodic still uses Python 2;
  • ROS Noetic is not out yet (and will probably only be ready by summer);
  • not all packages support Python 3.

I've done some preparations for Python 3 (for example, I've built our dependencies against it, patching as required). There are still some issues to be resolved, though:

  • Butterfly should be running from a virtual environment (may be resolved by patching rosbridge_suite) (UPDATE: May be worked around by using an older tornado version) Butterfly runs like before, it just uses an older Tornado version;
  • rosbridge_suite does not work with Python 3 (UPDATE: Should be fixed by Python 3 updates/fixes RobotWebTools/rosbridge_suite#460) (UPDATE 2: We are using a patched build that should work);
  • cv_bridge seems to work after applying these patches: Python 3 compatibility (Melodic) ros-perception/vision_opencv#312;
  • python-opencv is not getting installed (since it's not required anywhere) python-opencv is installed explicitly;
  • native tests are failing because of #!/usr/bin/env python3 shebangs All tests now use ros_pytest, which selects the correct interpreter automatically;
  • documentation needs to be updated (mainly the usage of print statements in Python)

This is still a work-in-progress, but we should perform regular builds from this branch and try using them. This needs to be thoroughly tested before merge.

@sfalexrog sfalexrog requested a review from okalachev February 13, 2020 16:01
@sfalexrog
Copy link
Member Author

@okalachev, this P/R drops ROS support for Python 2 completely (for reasons stated elsewhere). python-opencv is just no longer a dependency of anything, we could re-add it manually though.

@@ -2,5 +2,5 @@
Description=Butterfly Terminal Server

[Service]
ExecStart=/usr/local/bin/butterfly.server.py --host="0.0.0.0" --unsecure
ExecStart=/bin/bash -c ". /root/butterfly_env/bin/activate; butterfly.server.py --host="0.0.0.0" --unsecure"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to just use Python 2 for Butterly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be, though I'd really like to keep as little of Python 2 as possible. In fact, we could probably just use a lower version of Tornado for Butterfly, it shouldn't cause any problems.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've restored system-wide butterfly installation, it should work with Tornado 4.2.1.

@@ -17,6 +17,8 @@
from clover.srv import GetTelemetry, Navigate, NavigateGlobal, SetPosition, SetVelocity, \
SetAttitude, SetRates, SetLEDEffect

get_telemetry = rospy.ServiceProxy('get_telemetry', GetTelemetry)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is diz?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Picked this from #154, which "contained useful code" (your own words!)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK :) Когда это в виде изменения, бросилось в глаза.

@@ -20,14 +20,16 @@ nmap --version
lsof -v
git --version
vim --version
pip --version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not to keep pip 2?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends on how much Pyhton2 we wish to keep. I could re-add pip for Python 2, but what would be the rationale? It's not like our users are going to be able to use Python 2 in any meaningful way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible to write and run normal programs (even without ROS), right? It looks like a meaningful way. For older programs, for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I've added installing pip for Python 2 back. pip will still default to pip3, users will still be able to use pip2 for Python 2 packages.

@sfalexrog
Copy link
Member Author

Regarding rosbridge_suite: I've sent them a pull request to add Python 3 compatibility. Might still be rough around the edges, but at the very least it works for our web visualizations: RobotWebTools/rosbridge_suite#460

@sfalexrog
Copy link
Member Author

Regarding our failing (native) tests: Looks like we need to use some hacks to be both Python2- and Python3-compatible. The ROS way is to use Python2 shebangs (#!/usr/bin/env python) and let catkin take care of everything (see http://wiki.ros.org/UsingPython3/SourceCodeChanges#Changing_shebangs), but that requires:

  1. adding catkin_install_python() for all of our Python scripts;
  2. actually using install space instead of devel space (if I understand correctly, of course).

Using install space is a no-go (at least in my opinion), at the very least this will make changing .launch files a pain.
One possible thing would be to invoke our Python scripts from shell scripts that would test for ROS_PYTHON_VERSION == 3 and select proper interpreter. Not a clean way to do anything, but if anyone has other ideas, they are very welcome to contribute them!

@okalachev
Copy link
Member

Work goes in #327.

@okalachev okalachev closed this May 6, 2021
@okalachev okalachev deleted the buster-python3 branch February 20, 2022 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants