Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ente raspberry pi 5 camera test #133

Draft
wants to merge 4 commits into
base: ente
Choose a base branch
from

Conversation

liammcalpineduckietown
Copy link

Description

  • Please include a summary of the change and which issue is fixed.
  • Please also include relevant motivation and context.
  • List any dependencies that are required for this change.
  • Explicitly mention if the current changes can (or could) break any existing interface or functionality.
  • If this pull request results in from a fix to a GitHub issue, give a link to it.

Type of change

Please mark the relevant options with X:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change is constrained only to one package, any interfaces to nodes from other packages are maintained
  • This change is constrained only to packages in this repository, any interfaces to nodes from other repositories are maintained
  • This change required a documentation update
  • This change is only a documentation update
  • This change restructures the repository (at level higher than package level)
  • This change requires a change in the Duckiebook (if yes, please describe and, if possible, make a pull request for an edit in the book and link it here)

Tests

Please describe all tests ran and their outcome. It is highly recommended that you at least do the tests listed by default, even if you only changed the documentation.

  • Made the Duckiebot move with keyboard control
  • Visually checked the published camera stream
  • Verified that the lane following demo works
  • Verified that the indefinite navigation demo works

Style compliance

Please make sure that all your changes conform to the Duckietown (code style)[link] and code (documentation style)[link] guidelines:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I checked that this pull request is compliant with the Duckietown code style
  • I checked that this pull request is compliant with the Duckietown code documentation style

Next steps

Please describe if you have any questions, comments, or doubts towards the Duckietown software guardians regarding implementation, code style, running tests, documentation, etc.

FOR THE REVIEWERS

IMPORTANT: DO NOT CHANGE THIS SECTION!

  • Functionality (is the change necessary and is it implemented in the most appropriate way): @liampaull or @afdaniele
  • Integration testing (verify the tests and run additional if necessary): @?
  • Code style compliance: @gibernas
  • Documentation compliance: @AleksandarPetrov

To the reviewers: If you deem necessary, request code edits, more tests, documentation update, or additional reviewing. Do not review your own code!

PR Template, last edit 23 Aug 2019

Copy link

@Tuxliri Tuxliri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Launch the image using the command

dts devel run -L sensor-camera -H endeavour -- --device /dev:/dev -v /run/udev:/run/udev

Comment on lines +137 to +152
# libcamera
RUN git clone https://github.com/raspberrypi/libcamera.git
WORKDIR ${PROJECT_PATH}/libcamera
RUN meson setup build -Dpipelines=rpi/vc4,rpi/pisp -Dipas=rpi/vc4,rpi/pisp -Dv4l2=true -Dgstreamer=enabled -Dtest=false -Dlc-compliance=disabled -Dcam=disabled -Dqcam=disabled -Ddocumentation=disabled -Dpycamera=enabled
RUN ninja -C build install
WORKDIR ${PROJECT_PATH}
# kmsxx
RUN git clone https://github.com/tomba/kmsxx.git
WORKDIR ${PROJECT_PATH}/kmsxx
RUN meson setup build -Dpykms=enabled
RUN ninja -C build install
WORKDIR ${PROJECT_PATH}
RUN ldconfig
RUN python3 -m pip install picamera2
ENV PYTHONPATH=$PYTHONPATH:/usr/local/lib/aarch64-linux-gnu/python3.12/site-packages
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These steps are very time consuming, increasing build time from 2 to 6 minutes, are these packages available as precompiled binaries?

@Tuxliri
Copy link

Tuxliri commented Dec 5, 2024

image
There is a discrepancy in the aspect ratio between the image seen from dt-ros-interface and the one in the duckietown image viewer.

image

Let's verify that we are getting correctly sized images from the camera.

numpy_array = self._camera.capture_array() if self._camera else None
if self.configuration.rotation:
numpy_array = numpy.rot90(numpy_array)
success, jpeg_encoded_numpy_array = cv2.imencode('.jpeg', numpy_array)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add measurements of CPU usage on the Pi 5

"video_bitrate": 25000000,
self._camera = Picamera2()
main = {
"size": (self.configuration.res_w, self.configuration.res_h),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change with the configuration format, where we specified width and height after performing the rotation of the image.

@Tuxliri
Copy link

Tuxliri commented Dec 11, 2024

I have investigated the issue further.

ROS driver image

image

The node loaded the following configuration file:

allow_partial_fov: null
exposure: 100
exposure_mode: sports
fov: 160
framerate: 30
maker: Waveshare
model: OV5647
res_h: 480
res_w: 640
rotation: 270
use_hw_acceleration: null

Then I swapped res_h and res_w as per the default following configuration file:

maker: Waveshare
model: OV5647
framerate: 30
fov: 160
res_w: 480
res_h: 640
rotation: 270
exposure_mode: sports
exposure: 100

and we can see that the image is cropped

image

raspistill-image

I have confirmed with raspistill-image that when the image is correctly acquired at 640x480 the FoV is the same as the full sized image.

Captured image @ 640x480
capture_640x480

Captured image @ 2592x1944
capture_2592x1944

@Tuxliri Tuxliri marked this pull request as draft December 11, 2024 11:55
@Tuxliri Tuxliri added this to the raspberrypi5 support milestone Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants