Phase One DF with P25+

[kidmarius]

Before filing any support requests, please do make sure that you have already
contributed the whole raw sample set for your camera under CC0 license to
https://raw.pixls.us/ !

Important note: to make sure that darktable can keep on supporting the raw

file format for your camera, please read
this post on
how/what raw samples you can contribute to ensure that we have the full raw
sample set for your camera under CC0 license!

[kidmarius]

Hi, I saw ther was an earlier post from 2021, but I just got this camera and Darktable is not reading the iiq file. Im on Ubuntu 23, latest version on Darktable (4.6.1), and although a page says IIQ files are supported, I get Darktable could not load, with the skull. Can anyone help on this issue?

[LebedevRI]

So what is the camera name? Have you contribute the requested samples as the blurb asked?

[kidmarius]

I did submit the sample. The camera name is Phase One DF, it's the successor of Mamyia. The digital back is now some years old, I think the most modern does 150MB, this one is 25 (as per the name).

[kidmarius]

I can submit as many files as needed

[LebedevRI]

I see no unverified samples with that camera name,
so either it didn't successfully upload or that is not the camera name.
Can you say which filename that was?

Also, please note that if that back can produce more than one raw type,
we need a whole set of samples with a sample per each raw mode.

[kidmarius]

uploading again, it oproduces IIQ L and IIQ S, I only have IIQ L today, can upload a S tomorrow. Also it gives an option for TIFF but I think it's only in name, and Darktable is showing a Tiff, its the only one it shows

[kidmarius]

CF028642.IIQ Phase One P25+, https://raw.pixls.us/upload.php stays in a blank page, if I try again it says already been uploaded

[LebedevRI]

it oproduces IIQ L and IIQ S, I only have IIQ L today, can upload a S tomorrow.

When doing so, please take both raws of the same subject and upload both of them.

CF028642.IIQ Phase One P25+, https://raw.pixls.us/upload.php stays in a blank page, if I try again it says already been uploaded

Huh, that's funny, RPU seems to be again having issues.
The file did upload: https://raw.pixls.us/edit-admin.php?id=7380
But somehow the metadata wasn't actually parsed from EXIF,
and doing so results in HTTP 500. @andabata

[kidmarius]

maybe I should try aganin with another one in an hiour or 2?

[andabata]

PHP Ran out of memory. Exiv2 decided to create a 74MB file with the exifdata. Gave php a bit more memory so it can parse the file.

[kmilos]

Exiv2 decided to create a 74MB file with the exifdata.

Huh. I'll check that out when the samples are available.

[kidmarius]

now it did

[kidmarius]

Ok, submitted files on all formats the caemra does. IIQ S and L with IIQ extension and Tiff extension

[kidmarius]

going to submit some canon raw as requested on website

[LebedevRI]

Ok, submitted files on all formats the caemra does. IIQ S and L with IIQ extension and Tiff extension

Thank you for contributing the full sample set!

[kidmarius]

Thank you guys for the great work!

[kmilos]

Exif.Photo.MakerNote                         Undefined 24692564

So exiv2 (even the latest 0.28.2) thinks there is a 24MB large MakerNote inside the IIQ L?! I'll take a closer look when I get a chance...

[kmilos]

@kidmarius Please feel free to test the linked PR.

P.S. Someone please also fix up the subject line and remove the placeholder text.

[LebedevRI]

I can't even verify crop in darktable-org/rawspeed#704,
there's constant glitchy border regardless of the raw crop i use and the demosaicer selected.
Oh, it helps to notice that the raw itself is one of those rare fun ones with portrait-oriented data.

(and i'm ignoring the jitterness of the re-draw after moving/zooming)
Also, does that camera lack an AA filter? The result looks so pixellated it almost looks like the CFA order is wrong.

[LebedevRI]

P.S. Someone please also fix up the subject line and remove the placeholder text.

@kmilos do you not have that permission on this repo?

[kmilos]

I can close, assign, change labels, projects and milestones, but not edit subject nor link issues to PRs...

[kidmarius]

@kidmarius Please feel free to test the linked PR.

Thanks Kmilos, I have no idea what to do with that. Is it supposed to go into teh Lua script part?

[LebedevRI]

I can close, assign, change labels, projects and milestones, but not edit subject nor link issues to PRs...

Hmm. Permissions for each role says:
Edit and delete anyone's comments on commits, pull requests, and issues (Read) NO (Triage) NO (Write) YES,
i though Triage allowed that, but clearly not... And custom roles aren't possible. Oh well.

[kmilos]

Thanks for checking, no biggie.

[kmilos]

So exiv2 (even the latest 0.28.2) thinks there is a 24MB large MakerNote inside the IIQ L?!

Update: Phase One actually put the raw data inside the MakerNote tag (also confirmed by exiftool), which, for better or worse, is dumped entirely by exiv2 in recent versions (the -b switch no longer has any effect, which maybe wasn't the best solution?).

[kmilos]

Looks like there's still some way to go on P25+: the white level is way off, and there is no automatic orientation...

Reopen? @LebedevRI

And I guess the IIQ S files are not supported for any of the Phase One models?

[kmilos]

the white level is way off

Note that both ADC and LibRaw's unprocessed_raw unpack this as really 16-bit values... Doesn't look like it's a simple shift left though, so a LUT maybe?

[LebedevRI]

IIQ S is not supported at all. Someone would need to take a look at dcraw for that.

I don't know what is going on with the white level on these backs, that is a very common issue.

I have a faint recollection of the similar orientation thing coming up recently,
and that, i think, is more on a darktable's side.

[kmilos]

I don't know what is going on with the white level on these backs, that is a very common issue.

So should we drop it to some empirical value in cameras.xml? 65535 is what was supplied after DNG conversion, which is using 16-bit unpacked values....

I have a faint recollection of the similar orientation thing coming up recently,
and that, i think, is more on a darktable's side.

Yes and no - dt handles it if it's supplied in the standard Exif tag via exiv2 (e.g. the converted DNG is rotated correctly; and I think it did work out on one of the old Hasselblad backs we enabled recently), which is not the case for these IIQ files... It's buried deep in the raw/"MakerNote" bytestream (exiftool seems to parse a bit more), but AFAIK RawSpeed doesn't parse nor expose orientation anyway...

The most transparent thing would be to have exiv2 parse that 25MB MakerNote of course, but that will also take a while to implement...

Edit: exiftool -v3 gives this hint:

  | | 14) MakerNotePhaseOne (SubDirectory) -->
  | |     - Tag 0x927c (24692564 bytes, undef[24692564]):
  | |         0008: 49 49 49 49 43 77 61 52 6c c4 78 01 00 10 00 00 [IIIICwaRl.x.....]
  | |         0018: 02 00 02 10 02 00 02 00 02 00 02 00 7f 41 00 00 [.............A..]
  | |         0028: bf f3 ff fb c6 4f f7 8f 3f ee 2f 7e 2f fd 62 18 [.....O..?./~/.b.]
  | |         0038: 0f ff e7 e7 f3 fe fa cc 80 02 00 e0 a2 02 a8 02 [................]
  | |         0048: 80 02 a8 02 aa 02 b2 02 2a 41 c4 02 4c 9d 68 2d [........*A..L.h-]
  | |         [snip 24692484 bytes]
  | | + [PhaseOne directory with 46 entries]
  | | | 0)  RawData = SCALAR(0x41231ec)
  | | |     - Tag 0x010f (23656036 bytes, undef[23656036]):
  | | |         0014: 00 10 00 00 02 00 02 10 02 00 02 00 02 00 02 00 [................]
  | | |         0024: 7f 41 00 00 bf f3 ff fb c6 4f f7 8f 3f ee 2f 7e [.A.......O..?./~]
  | | |         0034: 2f fd 62 18 0f ff e7 e7 f3 fe fa cc 80 02 00 e0 [/.b.............]
  | | |         0044: a2 02 a8 02 80 02 a8 02 aa 02 b2 02 2a 41 c4 02 [............*A..]
  | | |         0054: 4c 9d 68 2d 28 51 b5 4c bb 57 c5 2f 51 17 5f d4 [L.h-(Q.L.W./Q._.]
  | | |         [snip 23655956 bytes]
  | | | 1)  CameraOrientation = 1
  | | |     - Tag 0x0100 (4 bytes, int32s[1]):
  | | |      178c498: 01 00 00 00                                     [....]

[LebedevRI]

I don't know what is going on with the white level on these backs, that is a very common issue.

So should we drop it to some empirical value in cameras.xml? 65535 is what was supplied after DNG conversion, which is using 16-bit unpacked values....

I don't suppose there's whitelevel in the MakerNotePhaseOne somewhere?
Otherwise if we really want to change something, i guess the white level
is the bit depth of that digital back's ADC, i.e. 14 bit in most cases.
I guess we could hardcode that on case-by-case basis.

I have a faint recollection of the similar orientation thing coming up recently,
and that, i think, is more on a darktable's side.

Yes and no - dt handles it if it's supplied in the standard Exif tag via exiv2 (e.g. the converted DNG is rotated correctly; and I think it did work out on one of the old Hasselblad backs we enabled recently), which is not the case for these IIQ files... It's buried deep in the raw/"MakerNote" bytestream (exiftool seems to parse a bit more), but AFAIK RawSpeed doesn't parse nor expose orientation anyway...

The most transparent thing would be to have exiv2 parse that 25MB MakerNote of course, but that will also take a while to implement...

Edit: exiftool -v3 gives this hint:

  | | 14) MakerNotePhaseOne (SubDirectory) -->
  | |     - Tag 0x927c (24692564 bytes, undef[24692564]):
  | |         0008: 49 49 49 49 43 77 61 52 6c c4 78 01 00 10 00 00 [IIIICwaRl.x.....]
  | |         0018: 02 00 02 10 02 00 02 00 02 00 02 00 7f 41 00 00 [.............A..]
  | |         0028: bf f3 ff fb c6 4f f7 8f 3f ee 2f 7e 2f fd 62 18 [.....O..?./~/.b.]
  | |         0038: 0f ff e7 e7 f3 fe fa cc 80 02 00 e0 a2 02 a8 02 [................]
  | |         0048: 80 02 a8 02 aa 02 b2 02 2a 41 c4 02 4c 9d 68 2d [........*A..L.h-]
  | |         [snip 24692484 bytes]
  | | + [PhaseOne directory with 46 entries]
  | | | 0)  RawData = SCALAR(0x41231ec)
  | | |     - Tag 0x010f (23656036 bytes, undef[23656036]):
  | | |         0014: 00 10 00 00 02 00 02 10 02 00 02 00 02 00 02 00 [................]
  | | |         0024: 7f 41 00 00 bf f3 ff fb c6 4f f7 8f 3f ee 2f 7e [.A.......O..?./~]
  | | |         0034: 2f fd 62 18 0f ff e7 e7 f3 fe fa cc 80 02 00 e0 [/.b.............]
  | | |         0044: a2 02 a8 02 80 02 a8 02 aa 02 b2 02 2a 41 c4 02 [............*A..]
  | | |         0054: 4c 9d 68 2d 28 51 b5 4c bb 57 c5 2f 51 17 5f d4 [L.h-(Q.L.W./Q._.]
  | | |         [snip 23655956 bytes]
  | | | 1)  CameraOrientation = 1
  | | |     - Tag 0x0100 (4 bytes, int32s[1]):
  | | |      178c498: 01 00 00 00                                     [....]

Well, i suppose, the reasoning to add it to rs isn't too dissimilar to the reasoning used to add white balance.