rgb to yuv limitations, lossless avif for storage?

[fredgido]

I used
colorist convert 1.png 2.avif --quantizer 0,0
and converted back and there seems to be rounding errors as the image isn't the same?

I suspect it is the limitations of rgb24 to yuv444p. I was planning to use only for archival... is it possible to convert to YCoCg24 so that yuv444p -> rgb24 conversion can be lossless?...

[arun54321]

I have tried converting png to ycocg avif using ffmpeg and rav1e. Still there is some color difference.

$ffprobe test.ivf   
                                                                                                                                      
[libdav1d @ 0x55f2f4e8d180] libdav1d 0.6.0
Input #0, ivf, from 'test.ivf':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: av1 (High) (AV01 / 0x31305641), yuv444p(tv, ycgco/unknown/unknown), 1920x1080, 25 tbr, 25 tbn, 25 tbc
[libdav1d @ 0x55f2f4e913c0] libdav1d 0.6.0

[joedrago]

Hello! There are multiple things at play here, for sure.

First, rav1e doesn't support lossless mode yet. From my understanding, there is a different transform that is supposed to be used in AV1 encoding when lossless is requested, and rav1e hasn't plumbed that through. Here's the relevant rav1e issue:

xiph/rav1e#151

So for this first issue, using --codec aom for now should dodge this one.

The second issue (as arun54321 hints at) is YUV conversion. Normal YUV conversion has a drift of ~1 codepoint when roundtripping, so lossless is out the window for it. However, by signaling a different conversion method, you can avoid this. I'm currently in the process of supporting more "interesting" CICP matrixCoefficients (see my identity branch), which should hopefully alleviate this.

As arun54321 suggests, one possible avenue for this is to YCgCo (CICP 8), which I haven't implemented yet, but I'd like to get added this week, and another (much worse but technically "valid") method is Identity (CICP 0), where RGB pixels are just directly jammed into the YUV planes instead of actually converting them.

If I run these commands in my identity branch (starting with a source SRGB PNG):

avifenc src.png yuv_aom.avif --codec aom --min 0 --max 0 --nclx 1/13/1
avifdec yuv_aom.avif yuv_aom.png
avifenc src.png yuv_rav1e.avif --codec rav1e --min 0 --max 0 --nclx 1/13/1
avifdec yuv_rav1e.avif yuv_rav1e.png
avifenc src.png gbr_aom.avif --codec aom --min 0 --max 0 --nclx 1/13/0
avifdec gbr_aom.avif gbr_aom.png
avifenc src.png gbr_rav1e.avif --codec rav1e --min 0 --max 0 --nclx 1/13/0
avifdec gbr_rav1e.avif gbr_rav1e.png

As expected, yuv_rav1e.png suffers the worst codepoint drift of 7, as we're converting to YUV and we're not getting a lossless AV1 encode. By doing awful things and jamming GBR directly into YUV (gbr_rav1e.png), we only drift 2 codepoints.

If we switch to AOM (which supports the lossless transform), yuv_aom.png only suffers from the YUV conversion, so we see a drift of 1 CP. And finally, if we do the naughty GBR trick and ask AOM to be lossless, we see that gbr_aom.png is flawless.

I think using CICP0 (Identity) is as far from "best practices" as you can go, as AV1 encoders are definitely going to make all kinds of assumptions about the Y channel actually being luma and the other channels making up the chroma, but YCgCo (CICP 8) might end up being the sweet spot for this. I could even see emitting helpful warnings in avifenc in the future if I see that you're mostly asking for lossless, but you're about to be burnt by your selection of codec or matrixCoeffs.

You should see some new matrixCoeffs code coming in at some point this week to libavif, and perhaps we can finally pin down some nice lossless recipes for people. I'll leave this GitHub Issue open so we have a place to chat about it.

lossless.zip

[joedrago]

Oops, I didn't realize this was the colorist repo; my comments about the identity branch and whatnot are referring to libavif. Most everything else in the comment applies the same to colorist, however.

[arun54321]

avifenc src.png gbr_aom.avif --codec aom --min 0 --max 0 --nclx 1/13/0

Thanks. This works very well.

[joedrago]

I hope you read my whole message; this will make very inefficient encodes as you are not actually handing YUV data to the AV1 encoders. I would wait for other implemented matrix coefficients before deciding on a recipe. we don't really know what assumptions other AV1 decoders are going to have.

[fredgido]

How much is the size difference when you encode GBR?
Yes I think these encoders compress a lot better when color is correlated to luma so maybe even YCgCo causes a little compression loss.

[arun54321]

$ls -l normal.avif gbr.avif   
415.1 KB   gbr.avif   
415.1 KB   normal.avif

EDIT: please ignore this comment. I was using master branch. The compression ratio is not good on identity branch.

[joedrago]

They certainly shouldn't be identical in file size. Are you sure you are running the correct branch of code?

[gitoss]

You should see some new matrixCoeffs code coming in at some point this week to libavif, and perhaps we can finally pin down some nice lossless recipes for people. I'll leave this GitHub Issue open so we have a place to chat about it.

With avif now supported by Firefox and Chrome, the format becomes more relevant.

Are there any news on how to do actual lossless conversion?

This is the one point webp is currently better at (and might be still better at, even after solvign the yuv/rgb problem on av1's encoder sides).

[joedrago]

Lossless SRGB can be easily done now with --lossless in avifenc, but if colorist is built with a recent enough libavif, that can be simulated with:

-q 100 --codec aom --nclx 1/13/0

If you aren't trying to do wild ICC-abusing HDR tricks with AVIF, I recommend just using avifenc over colorist. It will be much more supported in the future.

[joedrago]

As a note, lossless is easy to do now and correct, but don't expect large gains over other lossless formats. AV1 encoders aren't currently optimizing for lossless, unfortunately. Maybe we will see gains in the future of the encoder authors make it a priority.

[radum]

@joedrago Out of curiosity (sorry if this is off topic) but what is the main difference between colorist and avifenc in terms of conversion to avif?

From how I understand the tools there is none.

[joedrago]

It is a question of scope and standards, honestly. colorist was borne out of a need to jam HDR images into semi-intolerant image formats, and perform HDR color conversion / analysis. The fact that it can convert SDR images between image formats is more of a side effect than an explicit goal, and many ICC profiles it generates are minor abuses of the ICC standard (as the colorist Introduction page states). It is very opinionated on how it does conversion, and is willing to pack whatever it needs into the file to retain the information it needs; standards are quite secondary. Imagine it more like a bleeding edge R&D tool, not an every day image conversion tool (please use ImageMagick for that; also said in the colorist homepage).

avifenc is designed to make standard AVIFs, and only has "opinions" on default values. Everything is quite tunable, and other than the necessary conversion to YUV in many cases, it tries to leave your pixels intact. It doesn't interpret ICC profiles, let alone generate or alter one ICC profile. It also can generate AVIF image sequences now, which I'm never going to add to colorist. To be fair, if you use the AVIF-specific colorist cmdlines like --nclx, --yuv, --speed, --codec, --quantizer, you're likely to make a very similar file to what avifenc makes.

Maybe the simplest answer is one of support. avifenc is an official tool bundled with libavif and is likely to have more eyes on it, more features, more maintenance, and be more updated. colorist is a great test bed for stretching the limits of HDR image processing, but the endgame for it was always to take the learnings from it and apply it to other tools.