Add method to expose CICP in image decoder

[mstoeckl]

This adds a method which exposes the image color space information as CICP values when possible. This is immediately useful for decoders for formats like PNG, which directly supports CICP; but also for specialized formats like HDR (uses linear transfer function, although the official default color primaries are weird) and QOI (has a linear colorspace option that I expect is used for non-color data, assuming it gets used at all).

Edit: the current revision also updates the way ICC color profiles are communicated.

This PR is somewhat minimal; I've not added support for CICP in AVIF as that requires dependency upgrades; or tried to map the full set of PNG color chunks (gAMA, cHRM, sRGB) to CICP when possible; or looked at to what extent various other formats (BMP, JPEG, WEBP, Farbfeld, OpenEXR, TIFF) could have color space information implementable as CICP. But this (along with synthesizing ICC profiles as needed, similar to the way the BMP decoder does it) is something I'd rather defer if there is no reason to expect it to break the core design.

This would close #2985.

[197g]

How does this handle different CICP priorities vs. other color information? For PNG the block is by specification supposed to supersede other color information so that an ICC profile becomes a mere auxiliary hint whereas in other formats the role is reversed. I'm also not sure how to deal with errors in these cases. The ICC can itself contain a CICP which must be consistent with the profile itself. While that is consistently specified the situation of encountering unsupported by well-formed data here is odd. We don't want to error but the ImageReader could conceivably:

• fallback to the complicated ICC data and indicate our own Cicp as undefined
• attempt to further fallback to deprioritized CICP information.

[mstoeckl]

How does this handle different CICP priorities vs. other color information?

I've updated the ImageDecoder documentation to make it clear that if both cicp_values() and icc_profile() return Ok(Some(...)), they should match.

The decoder implementation should ensure this is the case; if not, then users of the trait (like ImageReader) risk choosing arbitrarily.

For PNG the block is by specification supposed to supersede other color information so that an ICC profile becomes a mere auxiliary hint whereas in other formats the role is reversed.

The PNG decoder returns Ok(None) for the ICC profile when a cICP tag is present; it could also return a synthetic ICC profile derived from the cICP tag. (And for cicp_values(), if there is no cICP, and the ICC profile has a cicp tag, the decoder could return the contents of that tag.)

For a format where the ICC profile takes priority, if any ICC profile is present, then the decoder could just return an UnsupportedError in cicp_values because it cannot convert the color space of the image (determined by the ICC profile) to CICP.

[197g]

For a format where the ICC profile takes priority, if any ICC profile is present, then the decoder could just return an UnsupportedError

I find that a bit odd. A major incentive for the structural change of ImageDecoder to me was resolving that decoders could say what factually exists based on the file instead of having to cover something up for the sake of some policy of ImageReader. Imo such erasure of information should happen as part of the algorithm consuming the available information instead.

[mstoeckl]

A major incentive for the structural change of ImageDecoder to me was resolving that decoders could say what factually exists based on the file instead of having to cover something up for the sake of some policy of ImageReader.

1. For AVIF and PNG have different policies on which of ICC vs CICP-derived color spaces is used when both chunks are available. (https://github.com/AOMediaCodec/libavif/wiki/CICP#aside-icc-profiles, vs https://www.w3.org/TR/png-3/#color-chunk-precendence)
2. Many formats support color space information that is not directly encoded using ICC or CICP; mostly these specify color primaries and maybe also custom gamma (see BMP's CalibratedRgb, PRIMARIES for HDR, gAMA+cHRM for PNG, the chromaticities for OpenEXR), but there is also HDR metadata (mDCV, cLLI for PNG) and other parameters (adoptedNeutral for OpenEXR). Mapping these to CICP and ICC when possible could make it is easier to establish a uniform interface.

That being said, I'm certainly not certain that this PR is the right way to handle the issue. One alternative I've been thinking of is a colorspace() -> ColorSpace method for ImageDecoder which provides the decoder's best estimate at the color space information of the image, where ColorSpace is an enum containing any of (CICP, ICC, gamma + primaries) plus whatever else proves necessary int the future. Then icc_profile() could be left as a largely decorative function to be used if someone needs the exact bytes or ICC profile metadata (author, etc.) contained in the image, and cicp_values would likely not be needed.

Imo such erasure of information should happen as part of the algorithm consuming the available information instead.

This is tricky, because image format specs may conflict with metadata specs. DPX explicitly permits external metadata but recommends that the main file metadata (like orientation or CICP codes) override any present in external metadata (which presumably includes XMP and EXIF), except when left undefined; JXL files always ignore EXIF orientation, as should AVIF.

[197g]

It is indeed tricky, which is why we should be doing it right. Regarding non-icc/cicp color space information in combination with standard ICC data we could have a separate method for any profile synthesized by a decoder. At least that would fit with ImageReader choosing between available options and hopefully clarify the role for a re-encoding loop. (We could also hack it as an Abstract ICC Profile with specific copyright data, a mandatory tag, if we must have in-band information). Regarding application of metadata we should improve the compliance and add flags if there's a difference between formats.

I continue to be weirded out by JXL and heif directions, this time for explicitly refusing to push complexity through dedicated standards as if that removes said complexity from reality, but reading through the ''justification'' given in the AVIF thread I seem to at least understand what collective mind parasite got us there—so now we have more complexity, yay. Since: "Metadata of type Exif as specified by JEITA CP-3451 may be present in the file and shall be output by the MIAF reader" but we must always heed "irot and imir" and it's definitely wrong to apply twice. Requiring "no image transformations […] indicated in the Exif metadata" as opposed to presuming equivalence is absolutely asinine. So now re-encoding such an image into another format is no longer simple, requiring processing of of metadata to scrub orientation; and suddenly a re-encoder must deeply understand every kind of metadata to modify it accordingly but of course it never does, evidently imagmagick does not, and then you get broken images. (Sidenote that CSS's image-orientation: from-image also doesn't add to clarity by yet another need for information; is imir a from-image orientation or an inherent part of the pixel matrix? That kind of switch, if well-defined at all, would be easier to support if it were placed in ImageReaderOptions than something for every decoder to do).

[mstoeckl]

Regarding non-icc/cicp color space information in combination with standard ICC data we could have a separate method for any profile synthesized by a decoder.

I've updated the design of the MR to expose ICC profiles and CICP value through new color_profile_to_icc and color_profile_to_cicp functions that make it clear the results may be synthetic. I also extended color_profile_to_cicp to return a struct through which additional HDR metadata can be passed; it looks like PQ's tone mapping rules may require additional maxCLL/maxFALL parameters to implement, and future video color standards might get even more complicated.

(I've made the changes somewhat quickly to get the core ideas out, so the decoder implementations may need a bit more work.)

[mstoeckl]

(I've made the changes somewhat quickly to get the core ideas out, so the decoder implementations may need a bit more work.)

I was indeed missing a few implementations; now every format which could expose an ICC profile should now have the color_profile_to_* methods implemented. I've also updated color_profile_to_cicp to return Result<Option<>> like color_profile_to_icc.

(Edit: I may tweak decoder implementations over the next few days to help make it clearer how this might play out, but the current change to the ImageDecoder API is certainly ready to review.)

[197g]

I do like the new sketch better than the previous for sure, just the duplication in conversion between different color profile representations is rather annoying. It's of course plausible for (many older) formats with much more special representations but will just happen to be a lot of close-but-not-duplicate code otherwise especially for image-ext. Unless we make it accessible to downstream but that's a bigger API suggestion.

[RunDevelopment]

Just some thoughts. I don't know a lot about color, so please take what I say with a grain of salt.

[awxkee]

It seems to me microsoft followed its own convention where the sign is declared as part of the fixed point format, so instead of Q1.30 this is Q2.30, as these seem to be signed types in microsoft SDK. That's actually makes sense, since those coordinates might be negative.

[mstoeckl]

The CIE XYZ color space is constructed to ensure X,Y,Z are always nonnegative, see https://en.wikipedia.org/wiki/CIE_1931_color_space#CIE_XYZ_color_space, so I don't think this is likely. The definition of xyY also ensures x+y+z=1, so I expect x and y to only be valid if they are in the range [0,1]. Within this range, the high bit of 2.30 fixed point will never be set, so for valid color primaries it shouldn't make a difference whether FXPT2DOT30 is signed or not. So I think it is safe to make this change, and have done so.

[197g]

Out-of-gamut colors are well-defined, algebraically at least, even though they may have no physical meaning. Afterall, some "subtractive" part is pretty much how the experimental setup for color matching CIE XYZ was defined in the first place and why the observer functions are in part negative. It's not even true that all colors lie in 1931's XYZ given that the blue and red monochromatic light sources were rather narrower than appropriate. (Should I be buying CIE 170:2-2015 for 190€? Idk).

At the end of the day, primaries are just base points so that the linear interpolation region covers more the physically possible color space sensibly. If you want to reduce clipping of computational artifacts it's not absurd to have negative x or y coordinates. And clearly the winsdk considers out-of-gamut definitions to be possible.