[fetch_binary_drivers] new [fetch_drivers] package #27

moriarty · 2019-02-02T06:51:22Z

This is a new package providing the "fetch_drivers" which was previously
built internally and distributed as a binary only debian package.

We will officially announce when the upgrade from Indigo->Melodic is
ready. This is not yet the announcement.

18.04 and ROS Melodic on the Fetch Research Platforms is still being
tested.

The process isn't straight foward as there are calibration files
which you will want to back-up before the upgrade.

This fetch_drivers package also won't start like it use to because of
the change from upstart to systemd.
That requires fetch_system_config which will be open sourced soon.

This fetch_binary_drivers package is part of a plan to distribute
updates and upgrades to Fetch Research Platform customers faster in the future.

This relates to ZebraDevs/fetch_ros#63

An officiall announcement will be made when everything is ready.
We will update and document the process on:
https://docs.fetchrobotics.com/
We will also announce via our mailing list, ros discourse and post on:
https://opensource.fetchrobotics.com/

This is a new package providing the "fetch_drivers" which was previously built internally and distributed as a binary only debian package. We will officially announce when the upgrade from Indigo->Melodic is ready. 18.04 and ROS Melodic on the Fetch Research Platforms is still being tested. The process isn't straight foward as there are calibration files which you will want to back-up before the upgrade. This `fetch_drivers` package also won't start like it use to because of the change from `upstart` to `systemd`. That requires `fetch_system_config` which will be open sourced soon. This `fetch_binary_drivers` package is part of a plan to distribute updates and upgrades to Fetch Research Platform customers faster in the future. This relates to ZebraDevs/fetch_ros#63 1. An officiall announcement will be made when everything is ready. 2. We will update and document the process on: https://docs.fetchrobotics.com/ 3. We will also announce via our mailing list, ros discourse and post on: https://opensource.fetchrobotics.com/

Ported from internal Fetch Robotics code used for ROS Indigo/Ubuntu 14.04. Updated to support ROS Melodic/Ubuntu 18.04. Some key changes: - Change from using upstart to using systemd (expect version >= 236) - Correspondingly logs such as /var/log/upstart/robot.log are now available in /var/log/ros/robot.log (as well as via journalctl -u robot) - Manually creates udev rules and updates grub boot arguments to ensure consistent network device naming across different hardware. Uses netplan to set static IP for internal robot communications. - Sixad replaced with ps3joy (hopefully a better solution will be found) - Slightly better logrotate rules - Removed former joystick_monitor and soundplay upstarts

TODO: should this be catkinized

All these dependencies are exec_dependencies because this package doesn't compile anything. They're also in the CMakeLists.txt in the catkin_package and find_package to be safe.

moriarty

I will add additional dependencies just to be safe

fetch_binary_drivers/CMakeLists.txt

erelson

Couple typos I found

fetch_binary_drivers/package.xml

cjds

First pass. Will do a more thorough review soon

cjds · 2019-02-05T23:41:19Z

fetch_binary_drivers/README.md

+
+We will announce to our customers when we're officially ready and supporting melodic.
+
+See https://docs.fetchrobotics.com for more information.


cjds · 2019-02-05T23:42:04Z

fetch_binary_drivers/package.xml

+    This package, is a cmake/make only package which installs the binaries for the drivers and firmware.
+  </description>
+
+  <!-- One maintainer tag required, multiple allowed, one person per tag -->


Are these comments intentionally left in

They're just left over from the catkin generate package command. I think it was just reminding me to add the maintainers.

cjds · 2019-02-05T23:46:16Z

fetch_system_config/debian/compat

@@ -0,0 +1 @@
+9


You probably want 11?

Package: debhelper (9.20131227ubuntu1)

even for 14.04

My 14.04 desktop (still on 3.x kernel... nice and safe...) has 9.x debhelper installed. I don't think there's a reason to bump this requirement presently. And it sounds like the plan is we'll move to using bloom for this later anyways.

cjds · 2019-02-05T23:47:17Z

fetch_system_config/debian/control

+         openssh-server,
+         ros-melodic-ros
+Description: Configuration for Fetch on Melodic
+  This package will start ROS Melodic at bootup


What does this mean?

Very brief vague statement, but nominally accurate. It installs roscore + robot systemd units which run at computer power-on. I'll reword this probably.

fetch_system_config/root/lib/udev/rules.d/99-logitechF710.rules

fetch_system_config/debian/fetch-melodic-config.postinst

cjds · 2019-02-06T00:22:35Z

fetch_system_config/debian/fetch-melodic-config.postinst

+        # Modify default/grub to not use new-style network device naming
+        sed -i "s/GRUB_CMDLINE_LINUX=\"\"/GRUB_CMDLINE_LINUX=\"net.ifnames=0 biosdevname=0\"/" /etc/default/grub
+        update-grub
+        # Determine which device to map to eth0 and eth1 and update /etc/udev/rules.d/70-persistent-net.rules


Do eth0 and 1 apply on ubuntu 18.04

It is for our robots because I found the computer hardware we use is not 100% guaranteed to have the same e.g. ensp1s0 style names every time it boots. Therefore I implement a "standard" way to disable this newer naming behavior. See the commit message as well. I'll also being including notes about this networking decision in the docs for 14.04->18.04.

cjds · 2019-02-06T00:24:03Z

fetch_system_config/debian/fetch-melodic-config.postinst

+                cp -n /etc/udev/rules.d/70-persistent-net.rules /etc/udev/rules.d/70-persistent-net.rules.bak
+            fi
+            cp -f /etc/udev/rules.d/70-persistent-net.rules /tmp/70-persistent-net.rules
+            grep -v -E "eth0|eth1" /tmp/70-persistent-net.rules > /etc/udev/rules.d/70-persistent-net.rules | true


cjds · 2019-02-06T00:32:24Z

fetch_system_config/root/opt/ros/roscore_poststart.bash

@@ -0,0 +1,9 @@
+# This script is used by the roscore systemd service unit
+echo "waiting for roscore to come up"


Put the echo in the while so we know it hasn't locked up maybe?

I'm going to leave this the way it was for simplicity right now.

cjds · 2019-02-06T00:41:26Z

fetch_binary_drivers/package.xml

+  <exec_depend>fetch_auto_dock_msgs</exec_depend>
+  <exec_depend>nav_msgs</exec_depend>
+  <exec_depend>power_msgs</exec_depend>
+  <exec_depend>robot_calibration_msgs</exec_depend>


todo: ensure that these are the same as the ones we used when building the binary drivers.

you won't be able to do that -- the ROS buildfarm / repos only contain the latest debs - once a new version is synced, the old ones go away. Hardcoding the value would make your package uninstallable at best.

@mikeferguson: would a nightly build job of our binaries, ensuring we’ve always got the latest dependencies work here?

@cjds and I were hoping to open source the Dockerfile & Python script which builds the output.tar.gz- it won’t be helpful without pull access on our drivers but it would at least make the process transparent

@moriarty even having the released deb download a nightly binary build is a challenge -- when a sync to public happens, you'd have a ~24 hour window where the version built against public is no longer right. (and that sync happens every ~ 2 weeks, so that's about 7% downtime). Also how do you make it work on both shadow-fixed (where developers tend to test) and the main ros repo?

😞
@dbking77 & @chadrockey maybe long term we reopen the discussion about refactoring some things to better support public/private drivers to make releasing to research customers easier ❓

We might need a second version setup up and built against shadow-fixed, this still doesn't fix the original problem, and it also wouldn't be clear how to set that up so that the shadow-fixed pulled a fetch-drivers-shadow.tar.gz and the release pulled fetch-drivers.tar.gz especially since currently they're pulling: http://packages.fetchrobotics.com/binaries/fetch-drivers-0.8.0.tar.gz
... Fetch3 & my machine are currently setup against shadow-fixed, but these drivers were built against whatever is inside of

FROM ros:melodic-robot-bionic

(As mentioned @cjds will cleanup the creation process and we plan to put them:
https://github.com/fetchrobotics/fetch_firmware_repackager or rename that repository fetch_drivers_repackager)

And it works now, but that's likely because there hasn't been any API/ABI breaking changes, which is not always guaranteed.
We'll need to check which of the dependencies we have are core dependencies, and which are non-core dependencies.

For the most part, the 7% downtime is a worst case calculation coming from 24hr every 2 weeks. If the dependencies are part of the core ros repositories, I think the API/ABI is stable enough or at least suppose to be... For non-core, and not released by us then it's just a risk that we'll probably need to take.
I don't even know if the melodic sync is triggered every 2 weeks because most of the folks at open robots are developing towards ros2. @jacobperron & @wjwwood?

Our plan, was to setup a mirror of ROS, which we manually trigger a sync on.
So customers with Fetch Research Robots could point to our http://packages.fetchrobotics.com/ and get guaranteed stability. But have the option to get the packages from packages.ros.org either ros or ros-shadow-fixed if they really want to live on the edge and accept some instability.

By pushing this into the public build farm, our internal packages.fetchrobotics.com will become almost a pure mirror, which should have much maintainable for our DevOps team and @bluryi.

If we sync our mirror manually, after testing to know we're not in the 7% downtime, then we'll get stability and more frequent updates.

For the 7% of possible down time for the packages.ros.org I think @erelson will need to update the (not yet existing) documentation to include strong warnings about pointing to packages.fetchrobotics.com or packages.ros.org
The packages.fetchrobotics.com mirror for melodic does not yet exist, we were going to quietly get our packages into melodic, and then setup a mirror once they were in and synced.

@clalancette is the ROS Boss for Melodic, so it's up to him to decide when the syncs happen, though I think they've been trying to keep a regular cadence. Looking at the discourse posts, it's more like once a month, though maybe that will change.

Perhaps you can anticipate the syncs and be prepared with updated packages of your own to sync in order to minimize the period where your debs are out of date. We do usually announce an intention to sync a week or two before doing it, with the intention that people could test it for unintended regressions.

I don't know if this would be relevant as well, but there's this recent announcement by @nuclearsandwich:

https://discourse.ros.org/t/announcing-ros-snapshot-repositories/7705

My intention is to sync Melodic every two weeks, though I've basically never been able to achieve that due to regressions that I have to fix for every sync. So the cadence has indeed been more like once a month.

I intend to continue to attempt to sync every two weeks unless there is a compelling reason to change that. So if you have reasons you'd like it to change, please let me know and we can consider it.

@nickswalker and @mikeferguson I wanted to give you a warning: just in case you've been running melodic and testing with Fetch's using ros-shadow-fixed this is now there:

http://repositories.ros.org/status_page/ros_melodic_default.html?q=fetch_

@erelson will do a test on hardware using the ros-shadow-fixed before we make an official announcement with links to upgrade documentation. So: best case, everything works, but it's more likely that the official announcement to the fetch user mailing lists is going to be not the upcoming ros melodic sync, but the one after. If we're really unlucky we could be in one of those 7% downtime scenarios which was mentioned above.

Also: - Fix remaining indigo references - Don't worry about conflicting packages that never existed - String fixes

erelson · 2019-02-10T00:24:29Z

fetch_system_config/debian/fetch-melodic-config.postinst

+
+        # Disable wifi power management in 18.04
+        # https://gist.github.com/jcberthon/ea8cfe278998968ba7c5a95344bc8b55
+        sed -i "s/wifi.powersave = 3/wifi.powersave = 2/" /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf


fyi @cjds here's the updated approach 👍

cjds

LGTM. This works for a 0.1 release. We'll be improving it over time.

moriarty · 2019-02-13T23:53:14Z

Once ros/rosdistro#20301 is in the rosdep keys for fetch_description and fetch_teleop should be available and then I can bloom this.

moriarty · 2019-02-14T00:49:20Z

ros/rosdistro#20303

@cjds and @erelson we'll test this once on hardware before we merge the PR to the documentation website which officially announces melodic upgrade process.

moriarty and others added 6 commits February 1, 2019 22:21

[fetch_system_config] fix README

32f18f8

TODO: should this be catkinized

Fixup system config debian

5f022ca

Fixup .gitignore

529db79

[fetch_drivers] add more dependencies

ead867e

All these dependencies are exec_dependencies because this package doesn't compile anything. They're also in the CMakeLists.txt in the catkin_package and find_package to be safe.

moriarty commented Feb 5, 2019

View reviewed changes

fetch_binary_drivers/CMakeLists.txt Outdated Show resolved Hide resolved

fetch_binary_drivers/CMakeLists.txt Outdated Show resolved Hide resolved

moriarty requested review from cjds, dbking77, erelson and chadrockey February 5, 2019 20:23

erelson reviewed Feb 6, 2019

View reviewed changes

fetch_binary_drivers/package.xml Outdated Show resolved Hide resolved

fetch_binary_drivers/package.xml Outdated Show resolved Hide resolved

cjds suggested changes Feb 6, 2019

View reviewed changes

erelson added 2 commits February 6, 2019 19:10

Ensure robots power off safely when instructed to

b144529

Also: - Fix remaining indigo references - Don't worry about conflicting packages that never existed - String fixes

Update .gitignore

c5ee991

erelson mentioned this pull request Feb 7, 2019

First version of 18.04 upgrade documentation fetchrobotics/docs#60

Merged

erelson and others added 2 commits February 9, 2019 15:38

Fix typo in .gitignore

d305db0

Update wifi power management disabling

4432ece

erelson reviewed Feb 10, 2019

View reviewed changes

Copy updates to freight post-install script

68d33fa

cjds approved these changes Feb 13, 2019

View reviewed changes

cjds merged commit a8b971e into ZebraDevs:melodic-devel Feb 13, 2019

moriarty deleted the fetch-binary-drivers branch February 14, 2019 00:49

moriarty mentioned this pull request Feb 16, 2019

fetch_robots: 0.8.3-0 in 'melodic/distribution.yaml' [bloom] ros/rosdistro#20328

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fetch_binary_drivers] new [fetch_drivers] package #27

[fetch_binary_drivers] new [fetch_drivers] package #27

moriarty commented Feb 2, 2019

moriarty left a comment

erelson left a comment

cjds left a comment

cjds Feb 5, 2019

cjds Feb 5, 2019

moriarty Feb 11, 2019

cjds Feb 5, 2019

erelson Feb 7, 2019

cjds Feb 5, 2019

erelson Feb 6, 2019

cjds Feb 6, 2019

erelson Feb 6, 2019

cjds Feb 6, 2019

cjds Feb 6, 2019

erelson Feb 7, 2019

cjds Feb 6, 2019

mikeferguson Feb 6, 2019

moriarty Feb 7, 2019

mikeferguson Feb 7, 2019

moriarty Feb 7, 2019

wjwwood Feb 8, 2019

clalancette Feb 8, 2019

moriarty Feb 20, 2019

erelson Feb 10, 2019

cjds left a comment

moriarty commented Feb 13, 2019

moriarty commented Feb 14, 2019


		We will announce to our customers when we're officially ready and supporting melodic.

		See https://docs.fetchrobotics.com for more information.

		@@ -0,0 +1,9 @@
		# This script is used by the roscore systemd service unit
		echo "waiting for roscore to come up"

[fetch_binary_drivers] new [fetch_drivers] package #27

[fetch_binary_drivers] new [fetch_drivers] package #27

Conversation

moriarty commented Feb 2, 2019

moriarty left a comment

Choose a reason for hiding this comment

erelson left a comment

Choose a reason for hiding this comment

cjds left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cjds left a comment

Choose a reason for hiding this comment

moriarty commented Feb 13, 2019

moriarty commented Feb 14, 2019