Varying quality within the frame

[kornelski]

When encoding AVIF images with an alpha channel, I know which areas of the image are transparent, and therefore their YUV channels are meaningless (or proportionally less important, in case of semi-transparency).

I would like to pass that information to the encoder, so that it can avoid wasting bits on invisible pixels.

[dwbuiten]

Another pitfall of decoupled alpha, I suppose. I'm not sure how high ROI encoding is on anyone's list of things to do... or how it would interact with all of our current RDO, etc.

[tdaede]

If their YUV channels are zero, zero bits will be spent, and there's even a fast path in RDO for handling the zero-distortion case (which should usually be true for a zero channel). So forcing YUV channel values to zero in completely transparent areas is a good way to get this behavior.
It's not ideal of course, but also won't hurt even if we add some extra sideband information later.

[kornelski]

The problem is, I shouldn't have 0 values near the edges of opaque pixels. Black color will bleed into visible pixels, and a sharp edges will waste bits.

[vi]

Even without the transparency, sometimes center of the picture is more important than the borders.

Can QP be set on per-tile basis, so that central tiles have lower QP?

[kornelski]

So far I've worked around that by setting solid color under transparent pixels and adding blur/bloom around the edges with opaque pixels, so that the color channel can have hopefully-cheaper-to-encode soft edges when alpha gives it sharp edges.

https://github.com/kornelski/cavif-rs/blob/ff0e85bb67c6bb193fd496e195fa92ae6e93fb83/ravif/src/dirtyalpha.rs

However, it's all still very speculative and costs some bits. Since I can't know how the image will be split into tiles, I can't clear transparent colors to zero completely (I'd like to clear to zero only rectangular areas exactly matching tiles).