Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extract Review: Support images/videos with only named channels #776

Open
2 tasks done
BigRoy opened this issue Jul 15, 2024 · 3 comments
Open
2 tasks done

Extract Review: Support images/videos with only named channels #776

BigRoy opened this issue Jul 15, 2024 · 3 comments

Comments

@BigRoy
Copy link
Collaborator

BigRoy commented Jul 15, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior:

Currently when a review is trying to get generated for a publish using e.g. a multilayer EXR with only named channels the ExtractReview logic fails:

For example, take this log:
error.log
From discussion here

Which shows it failing (in Deadline) on an EXR sequence that has named channels:

  • AO
  • CryptoAsset
  • CryptoMaterial
  • CryptoObject
  • Depth
  • DiffCol
  • DiffDir
  • Emit
  • Normal
  • UV
  • rgba

One could argue the rgba sensibly is the one to use here - maybe there's a way we can force ffmpeg to allow that as the main layer? If no dedicated R G B A channels are found.

I also wonder what makes it so that a lot of image viewers are capable of defaulting to viewing the RGBA by default instead of the other channels - if there's some sort of metadata or logic involved with knowing which of the named channels is the default viewable then we should adhere to that to in the hope FFMPEG can process it too.

Expected Behavior:

Allow these sequences to:

  1. Not error in the publishing process
  2. Generate correct reviewables.

Version

1.0.0

What platform you are running on?

Windows

Steps To Reproduce:

This can be reproduced from Blender according to this:

  1. Set up blender render scene with multilayer outputs

ayon_blender_render

  1. and from: ayon+settings://blender/RenderSettings/renderer?project=test

ayon_blender_render_settings

Are there any labels you wish to add?

  • I have added the relevant labels to the bug report.

Relevant log output:

No response

Additional context:

See relevant Ayon discord thread
@BigRoy BigRoy added the type: bug Something isn't working label Jul 15, 2024
@BigRoy
Copy link
Collaborator Author

BigRoy commented Jul 15, 2024 •
edited
Loading

According to this Blender Artists topic we should be able to pass a -layer argument to FFMPEG to have it force pick a named channel of our choice.

This may then be e.g. ffmpeg -layer rgba or whatever but I can't find much info about that -layer argument.

@BigRoy
Copy link
Collaborator Author

BigRoy commented Jul 15, 2024

Example EXR that fails here:

ch_carole_001_renderShadingMain.beauty.1001.zip

@BigRoy
Copy link
Collaborator Author

BigRoy commented Jul 15, 2024

For what it's worth, the -layer argument does work and needs to be passed before the -i (input) argument.

So this would work:

ffmpeg -layer rgba -i ch_carole_001_renderShadingMain.beauty.1001.exr test.mp4

Example log then would be:

ffmpeg version 6.1.1-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Depth.V.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.X.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.Y.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.Z.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.X.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.Y.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.Z.
[exr @ 000001cd9f021780] Channel match layer : rgba.A.
[exr @ 000001cd9f021780] Channel match layer : rgba.B.
[exr @ 000001cd9f021780] Channel match layer : rgba.G.
[exr @ 000001cd9f021780] Channel match layer : rgba.R.
Input #0, exr_pipe, from 'ch_carole_001_renderShadingMain.beauty.1001.exr':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: exr, gbrapf32le(unknown/unknown/linear), 432x324 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
File 'test.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (exr (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Depth.V.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.X.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.Y.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.Z.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.X.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.Y.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.Z.
[exr @ 000001cd9f014880] Channel match layer : rgba.A.
[exr @ 000001cd9f014880] Channel match layer : rgba.B.
[exr @ 000001cd9f014880] Channel match layer : rgba.G.
[exr @ 000001cd9f014880] Channel match layer : rgba.R.
[libx264 @ 000001cd9f066000] using SAR=1/1
[libx264 @ 000001cd9f066000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000001cd9f066000] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 10-bit
[libx264 @ 000001cd9f066000] 264 - core 164 r3172 c1c9931 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=10 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=81 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p10le(tv, unknown/unknown/linear, progressive), 432x324 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 12800 tbn
    Metadata:
      encoder         : Lavc60.31.102 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/mp4 @ 000001cd9f010ec0] video:3kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 29.601120%
frame=    1 fps=0.0 q=40.0 Lsize=       4kB time=00:00:00.00 bitrate=N/A speed=   0x
[libx264 @ 000001cd9f066000] frame I:1     Avg QP:34.75  size:  2169
[libx264 @ 000001cd9f066000] mb I  I16..4:  3.0% 82.4% 14.6%
[libx264 @ 000001cd9f066000] 8x8 transform intra:82.4%
[libx264 @ 000001cd9f066000] coded y,u,v intra: 17.9% 14.8% 6.3%
[libx264 @ 000001cd9f066000] i16 v,h,dc,p: 12% 41% 18% 29%
[libx264 @ 000001cd9f066000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 15% 48%  4%  3%  2%  3%  1%  2%
[libx264 @ 000001cd9f066000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 24% 21%  7%  8%  3%  6%  2%  6%
[libx264 @ 000001cd9f066000] kb/s:433.80

So, as mentioned here:

We could use the extracted metadata of the available channels to write some extra logic that if there are no main channels to allow fallback in our codebase by default.

So that e.g. we can have an ordered list as fallback like: rgba, depth or whatever. And furthermore we could even allow it to fallback on ANY named channel to avoid an error whatsoever, but we could log a warning that we're then just picking any named channel and that one may need to configure a certain setting to set a "preferred" channel?

@mkolar mkolar removed the type: bug Something isn't working label Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@BigRoy @mkolar