image_view_node: support bayer images

[rursprung]

so far bayer images always failed with an error:

[ERROR] [..] [image_view_node]: Unable to convert 'bayer_rggb8' image for display: 'cv_bridge.cvtColorForDisplay() does not have an output encoding                that is color or mono, and has is bit in depth'

the stereo_view_node on the other hand already supports bayer images, however it always forcibly converts them to monochrome, even if they are colour images.

for now, this adds the same logic for the single-image viewer and thus only partially resolves #1045.

[mikeferguson]

Seems reasonable given what we do in stereo node

[rursprung]

i just realised that i was a bit tired when i thought yesterday that it's better to go with mono for now and then figure out colour later on... bayer is of course always colour, so i guess it'd be fine to just always use bgr (both here & in stereo)? if you concur, i can add a commit to this PR which changes it in both places

[mikeferguson]

bayer is of course always colour, so i guess it'd be fine to just always use bgr (both here & in stereo)

I'm pretty sure this won't show the expected colors you want. I think the trick here is that by casting to 'mono8' we treat the 4 pixels (2 green, 1 red, 1 blue) as a black and white image that is basically 2x the resolution of the actual image and so it looks roughly correct.

If you do test out 'bgr' and it looks OK - please post some screen shots here of the 'mono' and 'bgr' for review (I don't have any bayer cameras around to test with).

[rursprung]

ok, i'll do that either tonight or in the coming days.
it might be cleanest to merge this PR for now and then i'll post the information on the issue and raise a new PR?

it did look ok when i tried it (though the colours were off, but that's because i'd need to set white balance & co. in the camera for it to look correct)

[rursprung]

If you do test out 'bgr' and it looks OK - please post some screen shots here of the 'mono' and 'bgr' for review (I don't have any bayer cameras around to test with).

i've posted it on the issue: #1045 (comment)

[rursprung]

@mikeferguson: any chance that you could merge this?
unless you want me to use bgr8 instead of mono8 (see my screenshots on the issue).

either way it'd be great to get this merged & released soon so that i don't need a local build of the image pipeline to see raw images (while i do have the debayer node in place as well for further processing it's sometimes relevant to inspect the raw images)

[mikeferguson]

Let's stick with mono8 for now then.

[rursprung]

Let's stick with mono8 for now then.

thanks, @mikeferguson! should i create a backport PR for jazzy or do you do this automatically?

should i create a follow-up PR for bgr8 against rolling?

[mikeferguson]

https://github.com/Mergifyio backport jazzy

[mergify]

backport jazzy

✅ Backports have been created

• #1058 image_view_node: support bayer images (backport #1046) has been created for branch jazzy

[mikeferguson]

should i create a follow-up PR for bgr8 against rolling?

Are the colors actually accurate? I know you mentioned it wasn't color calibrated, but also, I imagine that having two green channels is maybe part of the issue (my understanding is that the data really isn't bgr8 under the hood, it seems wrong to try and interpret it that way - mono kinda makes sense because you're just casting what is basically BGGR to grayscale)

[rursprung]

Are the colors actually accurate? I know you mentioned it wasn't color calibrated, but also, I imagine that having two green channels is maybe part of the issue (my understanding is that the data really isn't bgr8 under the hood, it seems wrong to try and interpret it that way - mono kinda makes sense because you're just casting what is basically BGGR to grayscale)

OpenCV correctly debayers it if you use COLOR_BayerBG2BGR (or the appropriate Bayer.. version). the images looked just fine when i used a colour-corrected version (i just had to enable auto white balance for the camera)