Question about aug_data.json metadata generation

[DY112]

Hello @mahmoudnafifi !
I have a question about your paper.
I think I found a mistake from your data augmentation process.
Specifically, this is about section D.2 of your supplementary material.
Here's what I did:

1. I extracted ColorMatrix1 & 2 from Canon1DsMkIII dng file
2. Then I calculated interpolation coefficient alpha using Eq.16 and interpolated C1 and C2 using Eq.15
3. I compared the manually interpolated matrix with the provided matrix that you support (xyz2raw matrix from aug_data.json)

Below is the result:

• Extract ColorMatrix1 and ColorMatrix2 from Canon1DsMkIII dng file
  C1 :
  [ 0.6705 -0.1154 -0.0458]
  [-0.6783 1.4191 0.2848]
  [-0.0853 0.1159 0.749 ]
  C2:
  [ 0.5858 -0.0211 -0.093 ]
  [-0.8254 1.6018 0.2354]
  [-0.1732 0.1888 0.7448]

• Calculate alpha using Eq.16
  (Here I used color temperature 3203 for Canon1DsMkIII_0001 from aug_data.json, and use 2856 and 6504 as CCT of calibration illuminant)
  alpha = (1/3203 - 1/2856) / (1/6504 - 1/2856) ‎ = 0.19315

• Interpolate ColorMatrix using Eq.15 and alpha=0.19315
  C = alpha*C2 + (1-alpha)*C1
  [ 1.30209954 -0.19146018 -0.11063935]
  [-0.69119251 1.42220026 0.26899225]
  [-0.13227646 0.16790859 0.96436787]

• Interpolate ColorMatrix using reversed coefficient, meaning 1-alpha is applied to C2 and alpha to C1.
  C = (1-alpha)*C2 + alpha*C1
  [ 1.2572173 -0.08208199 -0.17513531]
  [-0.78562298 1.54417397 0.24144901]
  [-0.20444999 0.22865764 0.97579235]

• xyz2raw matrix from aug_data.json
  [1.2677382126576495, -0.10772150067561889, -0.1600167119820306]
  [-0.76447729466669867, 1.5168605655771008, 0.247616729089598]
  [-0.18815360372693687, 0.21494084064240207, 0.97321276308453475]

Alpha has a value of 1 when q_l is q2 and 0 when q_l is q1, so Eq.15 is written correctly.
However, comparing the xyz2raw matrix in the json file with the values I calculated myself, even taking into account the slight numerical difference, I think you probably calculated it using inverted order coefficients - used alpha for C1, 1-alpha for C2.

The results were same for another camera subsets.
If you have time to check this calculation process, please revalidate it.
Thanks!

[DY112]

Hello, I see I was wrong.
Since ColorMatrix converts XYZ chromaticity to Illuminated Camera chromaticity,
I should rather interpolate ForwardMatrix (which converts WB Camera chromaticity to CIE XYZ) and invert that matrix to calculate xyz2raw matrix.

[mahmoudnafifi]

Hey, sorry for missing this. I was on vacation and hadn't been checking my email regularly. Glad you figured it out!

[EmilySmile]

Hello @mahmoudnafifi and @DY112 ：

I have questions about how to get Canon1DsMkIII DNG file for Canon1DsMkIII_0001 and how to calculate raw2xy matrix

(1) I download Canon1DsMkIII_RAW.zip from http://cvil.eecs.yorku.ca/projects/public_html/illuminant/illuminant.html
(2) Convert the CR2 raw file“Canon1DsMkIII_0001.CR2” from the Canon 1Ds Mark III camera to Adobe DNG files“Canon1DsMkIII_0001.dng” using the Adobe DNG Converter tool"AdobeDNGConverter_x64_17_1.exe ".
(3)Parse DNG info using exiftool-13.10_32.exe tool, and the details are attached inCanon1DsMkIII_0001.dng.txt,with some information listed below:
#Canon1DsMkIII_0001.dng.txt:
#WB RGGB Levels As Shot : 1950 1079 1079 2536
#Color Temp As Shot : 3592
#Forward Matrix 1 : 0.8098 0.0217 0.1328 0.4016 0.691 -0.0926 0.0599 -0.1224 0.8877
#Forward Matrix 2 : 0.8098 0.0217 0.1328 0.4016 0.691 -0.0926 0.0599 -0.1224 0.8877
#Color Matrix 1 : 0.6705 -0.1154 -0.0458 -0.6783 1.4191 0.2848 -0.0853 0.1159 0.749
#Color Matrix 2 : 0.5858 -0.0211 -0.093 -0.8254 1.6018 0.2354 -0.1732 0.1888 0.7448
(4) calculate raw2xy matrix C
alpha = (1/3203 - 1/2856) / (1/6504 - 1/2856) ‎ = 0.19315
C = alpha*Forward_Matrix_2 + (1-alpha)*Forward_Matrix_1
= 0.8098 0.0217 0.1328 0.4016 0.691 -0.0926 0.0599 -0.1224 0.8877
However C is different with raw2xyzmatrix from aug_data.json, which is
"raw2xyz":[[0.83241191941326331,0.041206179758279705,0.12638190082845693],[0.40796173206444919,0.70410873223358483,-0.11207046429803423],[0.070831032429659255,-0.14754084311174423,1.0767098106820852]],

Could you provide me with some guidance? I look forward to your reply.

[EmilySmile]

Canon1DsMkIII_0001.dng.txt

[mahmoudnafifi]

Please see my reply here. Thanks!