video: emul: virtual driver for an imager and RX engine #79482

josuah · 2024-10-06T23:21:02Z

This takes inspiration on the other skeleton.c drivers through Zephyr tree:
https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/usb/udc/udc_skeleton.c
https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/display/display_dummy.c

It acts as a starting point and as a puppet to explain API changes affecting image sensors.

Be used for providing test cases for the entire API, as required for stable APIs.

Utilities are introduced in the process to reduce boilerplate.

Twister command (new with this PR):

west twister --inline-logs --platform native_sim --scenario tests/drivers/video/api/drivers.video.api

Fixes: #73867

josuah · 2024-10-06T23:23:02Z

I note that the frame interval API was merged, this will need to include it.

Video : Add frame interval APIs #72254

josuah · 2024-10-13T13:35:02Z

Aims to cover the most frequent use-cases and facilitate modifications on top
Assume that image sensor drivers are apply modes described by pixelformat+resolution+framerate
For every new sensor, require filling a few tables and let the ready-made functions hook them to the Zephyr video API

PR zephyrproject-rtos#79482 is where this commit would be added Signed-off-by: Josuah Demangeon <[email protected]>

josuah · 2024-11-27T23:33:57Z

In the process of implementing these emulated drivers, I introduced more video API helpers to reduce boilerplate.

If introducing helpers does not make sense in this same PR, I can integrate them back into the emulated drivers to reduce the amount of things this introduces.

ngphibang · 2024-11-28T09:33:54Z

I will try to review this whenever possible, thanks !

drivers/video/video_emul_imager.c

drivers/video/video_emul_rx.c

loicpoulain · 2024-12-01T20:02:27Z

include/zephyr/drivers/video.h

+ *
+ * @return The frame interval value in microseconds.
+ */
+uint64_t video_frmival_nsec(const struct video_frmival *frmival);


Could this be a simple inline helper?

Yes I agree. Done now.

josuah · 2024-12-02T07:16:39Z

force-push:

applied recommendations from review
fix the RGB565 pattern generator (wrong color order)
cfg seems more frequently used than conf
endpoint@0 (IN) first, endpoint@1 (OUT) after, in devicetree, to have the source upstream and sinks downstream in the dts files, which also matches the media-ctl --print-dot numbering

[EDIT: the above is this on Linux: media-ctl --print-dot | circo -Tpdf >media-ctl-pipeline.pdf]

PR zephyrproject-rtos#79482 is where this commit would be added Signed-off-by: Josuah Demangeon <[email protected]>

Introduce a video_get_format_index() utility to help finding a caps entry out of a given format. Introduce several utilities to seek and apply frame intervals. Signed-off-by: Josuah Demangeon <[email protected]>

Add a new implementation of a test pattern generator, with the same architecture as real drivers: split receiver core and I2C-controlled sub-device, with changes of video format in "zephyr,emul-imager" leads to different data produced by "zephyr,emul-rx". Signed-off-by: Josuah Demangeon <[email protected]>

loicpoulain

LGTM

ngphibang · 2024-12-06T09:22:06Z

include/zephyr/drivers/video.h

+ * @return 0 when a format is found.
+ * @return -ENOENT when no matching format is found.
+ */
+int video_format_caps_index(const struct video_format_cap *fmts, const struct video_format *fmt,


All the introduced functions here are supposed for drivers uses only ?. I think it should be put in an internal header rather in the public header video.h.

The users can access the format cap structure, so they could be interested in verifying: "for this particular width/height/pixelformat, what is the min/max/step?

Not sure this would ever get used that way though! Glad with putting it on a drivers/video/video_common.h header like often done for driver helper functions.

ngphibang · 2024-12-06T09:30:33Z

Sorry that I couldn't manage to review this before it gets merged (but I think this is merged a bit too fast though, seeing the number of new functions introduced in video.h).

ngphibang · 2024-12-06T09:35:14Z

drivers/video/video_emul_rx.c

+	return 0;
+}
+
+/* See #80649 */


This uses all the macros defined in #80649 by copying them here. I think the better way is to make this PR depending on #80649 or cherry-pick the commits.

As this was merged first, I will fix the conflict in #80649 then.

s/the better way/the correct way

This was opened 2 weeks before #80649, so got started with putting some custom macros, and then forgot to rebase it on #80649. Thank you for fixing these macros on #80649!

josuah · 2024-12-06T13:23:57Z

I had them on video_common.h at first, but noticed they could also be used from the application for some purpose... so I moved them to video.h.

Let me know which one would be a better fit in video_common.h.

I also wanted to introduce IMX219 and others following this template.
So reviews will help even post-merge!

josuah added area: Drivers area: Video Video subsystem labels Oct 6, 2024

zephyrbot requested review from loicpoulain and ngphibang October 6, 2024 23:21

josuah added DNM This PR should not be merged (Do Not Merge) and removed DNM This PR should not be merged (Do Not Merge) labels Oct 6, 2024

josuah marked this pull request as draft October 6, 2024 23:23

josuah force-pushed the pr-video-sensor-skeleton branch 5 times, most recently from eb4a7a4 to cfa3bfb Compare October 13, 2024 13:20

josuah marked this pull request as ready for review October 13, 2024 13:20

zephyrbot added the area: Base OS Base OS Library (lib/os) label Oct 13, 2024

zephyrbot requested review from andyross, dcpleung, nashif and peter-mitsis October 13, 2024 13:20

zephyrbot assigned andyross and nashif Oct 13, 2024

josuah requested review from pillo79 and uLipe October 13, 2024 13:36

josuah force-pushed the pr-video-sensor-skeleton branch 2 times, most recently from 64e0235 to 9a1a450 Compare October 14, 2024 12:02

zephyrbot added platform: STM32 ST Micro STM32 area: Watchdog Watchdog labels Oct 14, 2024

zephyrbot requested review from erwango and FRASTM October 14, 2024 12:03

josuah added a commit to tinyvision-ai-inc/zephyr that referenced this pull request Nov 21, 2024

drivers: video: add a virtual device pipeline

a5f6339

PR zephyrproject-rtos#79482 is where this commit would be added Signed-off-by: Josuah Demangeon <[email protected]>

josuah mentioned this pull request Nov 27, 2024

Implement video control framework #82158

Open

This was referenced Nov 29, 2024

drivers: video: API enhancement #72959

Open

drivers: video: sensors: add a sensor_skeleton.c to speed-up contribution of simple sensors #73867

Closed

josuah force-pushed the pr-video-sensor-skeleton branch from eb2d81e to 87e71dd Compare December 1, 2024 17:30

pdgendt reviewed Dec 1, 2024

View reviewed changes

drivers/video/video_emul_imager.c Outdated Show resolved Hide resolved

drivers/video/video_emul_rx.c Outdated Show resolved Hide resolved

loicpoulain reviewed Dec 1, 2024

View reviewed changes

josuah force-pushed the pr-video-sensor-skeleton branch 3 times, most recently from 8cef8e6 to b29dbdc Compare December 2, 2024 07:11

josuah mentioned this pull request Dec 2, 2024

drivers: video: shell: initial implementation #82393

Draft

josuah added a commit to tinyvision-ai-inc/zephyr that referenced this pull request Dec 3, 2024

drivers: video: add a virtual device pipeline

5d14b5f

PR zephyrproject-rtos#79482 is where this commit would be added Signed-off-by: Josuah Demangeon <[email protected]>

josuah added 2 commits December 3, 2024 02:08

drivers: video: common: Add utilities to seek frmival/caps structures

046e3e8

Introduce a video_get_format_index() utility to help finding a caps entry out of a given format. Introduce several utilities to seek and apply frame intervals. Signed-off-by: Josuah Demangeon <[email protected]>

josuah force-pushed the pr-video-sensor-skeleton branch from b29dbdc to fc94165 Compare December 3, 2024 01:09

loicpoulain approved these changes Dec 5, 2024

View reviewed changes

nashif removed the area: Base OS Base OS Library (lib/os) label Dec 5, 2024

nashif unassigned nashif and andyross Dec 5, 2024

nashif approved these changes Dec 5, 2024

View reviewed changes

nashif merged commit 9e908b1 into zephyrproject-rtos:main Dec 6, 2024
25 checks passed

ngphibang reviewed Dec 6, 2024

View reviewed changes

ngphibang mentioned this pull request Dec 6, 2024

include: devicetree: Add port and endpoint macros #80649

Merged

josuah deleted the pr-video-sensor-skeleton branch December 6, 2024 13:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

video: emul: virtual driver for an imager and RX engine #79482

video: emul: virtual driver for an imager and RX engine #79482

josuah commented Oct 6, 2024 •

edited

Loading

josuah commented Oct 6, 2024

josuah commented Oct 13, 2024

josuah commented Nov 27, 2024

ngphibang commented Nov 28, 2024

loicpoulain Dec 1, 2024

josuah Dec 1, 2024

josuah commented Dec 2, 2024 •

edited

Loading

loicpoulain left a comment

ngphibang Dec 6, 2024

josuah Dec 6, 2024

ngphibang commented Dec 6, 2024

ngphibang Dec 6, 2024 •

edited

Loading

ngphibang Dec 6, 2024

ngphibang Dec 6, 2024

josuah Dec 6, 2024

josuah commented Dec 6, 2024

video: emul: virtual driver for an imager and RX engine #79482

video: emul: virtual driver for an imager and RX engine #79482

Conversation

josuah commented Oct 6, 2024 • edited Loading

josuah commented Oct 6, 2024

josuah commented Oct 13, 2024

josuah commented Nov 27, 2024

ngphibang commented Nov 28, 2024

loicpoulain Dec 1, 2024

Choose a reason for hiding this comment

josuah Dec 1, 2024

Choose a reason for hiding this comment

josuah commented Dec 2, 2024 • edited Loading

loicpoulain left a comment

Choose a reason for hiding this comment

ngphibang Dec 6, 2024

Choose a reason for hiding this comment

josuah Dec 6, 2024

Choose a reason for hiding this comment

ngphibang commented Dec 6, 2024

ngphibang Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

ngphibang Dec 6, 2024

Choose a reason for hiding this comment

ngphibang Dec 6, 2024

Choose a reason for hiding this comment

josuah Dec 6, 2024

Choose a reason for hiding this comment

josuah commented Dec 6, 2024

josuah commented Oct 6, 2024 •

edited

Loading

josuah commented Dec 2, 2024 •

edited

Loading

ngphibang Dec 6, 2024 •

edited

Loading