Compare with python & C++ image folder publishing

[lucasw]

ROS_PACKAGE_PATH=/usr/share/sensor_msgs cargo build --release

cd ~/Pictures
~/own/src/rust/rosrust_image/rosrust_image/target/release/image_dir_pub

45% cpu

vs.

https://github.com/lucasw/image_manip/blob/master/image_manip/scripts/image_folder_publisher.py

rosrun image_manip image_folder_publisher.py _folder:=$HOME/Pictures _update_period:=0.2

25% cpu

The python script loads each image with imread fresh and converts to a ros image every single publish.

And now a C++ version:

https://github.com/lucasw/image_manip/blob/fix_zero_frame_rate/image_manip/src/image_folder_publisher_node.cpp

cd ~/Pictures
rosrun image_manip image_folder_publisher

It is also around 25% cpu (should see if it is lower latency later)

Why is it faster- and where is it faster? The image loading library and the pixels to Image message are places to look, but rosrust itself is the big one.

Zenoh docs mention wanting to refactor RT rosrust to speed it up

[lucasw]

https://github.com/flamegraph-rs/flamegraph?tab=readme-ov-file

[lucasw]

Make a version of the image_folder_pub that publishes using zenoh

[lucasw]

Update all the dependencies, especially pixels

->

That seemed to do the trick, now it takes about the same amount of cpu as the python and C++ nodes

[lucasw]

This converts a (2160, 3840, 3) image into 24883253 bytes, which is 53 header bytes beyond the 2160 * 3840 * 3:

https://github.com/lucasw/image_manip/blob/fix_zero_frame_rate/image_manip/scripts/zenoh_image_folder_publisher.py

It is received successfully by https://github.com/lucasw/ros_one2z/blob/main/ros1_example_pkg/scripts/zenoh_image_to_contour.py (too many zenoh experiments scattered across too many repos, I know)

But using rosrust msg.encode_vec() I get 24883249 bytes, which is 4 bytes short- this issue sounds familiar- those bytes are added back somewhere in the send instead of just coming out of the encode method.

---

I added the bytes and have the right length, but it's not working

[E] [1709913525.236 /zenoh_image_sub ...sr/lib/python3.11/threading.py:1002]: Characters replaced when decoding message sensor_msgs/Image (will print only once): 'utf-8' codec can't decode byte 0xbf in position 1: invalid start byte
Exception in thread Thread-5 (readqueue):
Traceback (most recent call last):
  File "/home/lucasw/install_base_catkin_ws/install/lib/python3/dist-packages/sensor_msgs/msg/_Image.py", line 169, in deserialize
    (_x.height, _x.width,) = _get_struct_2I().unpack(str[start:end])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: unpack requires a buffer of 8 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lucasw/other/install/lib/python3.11/site-packages/zenoh/closures.py", line 102, in readqueue
    adapted(*x)
  File "/home/lucasw/other/install/lib/python3.11/site-packages/zenoh/closures.py", line 95, in <lambda>
    adapted = lambda *args: _call_(type_adaptor(*args))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lucasw/ros/ros1_one/src/ros_one2z/ros1_example_pkg/scripts/zenoh_image_to_contour.py", line 47, in listener
    image_msg.deserialize(sample.payload)
  File "/home/lucasw/install_base_catkin_ws/install/lib/python3/dist-packages/sensor_msgs/msg/_Image.py", line 191, in deserialize
    raise genpy.DeserializationError(e)  # most likely buffer underfill
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
genpy.message.DeserializationError: unpack requires a buffer of 8 bytes

->

encode_vec may be completely wrong, look more in rosrust to see how messages are encoded

The message starts in api/raii.rs Publisher send() and gets passed into LossySender try_send

[lucasw]

header + rosrust encode_vec():

[0, 0, 0, 237, 233, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 114, 103, 98, 56, 0, 24, 0, 0, 0, 192, 0, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255]

An 8x8 white image encoded with ros python, secs set to 1 and nsecs to 2, frame_id is "":

[0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 98, 103, 114, 56, 0, 24, 0, 0, 0, 192, 0, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255]

the different parts:

0, 0, 0, 237, 233, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 114, 103, 98,

0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 8, 0, 0, 0, 4, 0, 0, 0, 98, 103, 114,

So the header is already there with the [233, 0, 0, 0], don't need to add one, but with reversed endianness- where in the rosrust code is the endianness reversed- probably in the network send, it looks like the

Also did I remove the need for the header elsewhere (e.g. in the zenoh to mcap scripts?)- no there's just no parity with the encode_vec vs. the python and c++ serialization- thoughs defer that header until the tcp send. Probably will just remove the header from encode_vec in my rosrust fork.