grain/detail is retained on every other frame

[plonk264]

simple settings, but multiple quant and speed settings
rav1e.exe --metrics --threads 8 --speed 6 --quantizer 80 fightclubsnippet.y4m -o fightclubsnippet-rav1e-q80-speed6.ivf

also when i tried quant 72, 64, 56, and speeds down to 1

every other frame seems to have detail: https://cdn.discordapp.com/attachments/587033245061873759/736876069537316864/mpv-shot0002.png
and every other frame seems to wipe it out:
https://cdn.discordapp.com/attachments/587033245061873759/736876046753988634/mpv-shot0001.png

source: http://www.mediafire.com/file/55j542f7ws24ygo/fightclubsnippet.mkv/file

encodes: http://www.mediafire.com/file/q10922pyrepzhop/fightclubsnippet-rav1e-q72-speed6.ivf/file
http://www.mediafire.com/file/dn55dfvvlfc1gig/fightclubsnippet-rav1e-q64-speed6.ivf/file
http://www.mediafire.com/file/f5jl6ugeamjdu3h/fightclubsnippet-rav1e-q64-speed1.ivf/file
http://www.mediafire.com/file/kxzgzza2x323kby/fightclubsnippet-rav1e-q56-speed6.ivf/file

[rzumer]

I can't find a related open issue, but I remember this having been reported before. Will look into it when I have some time.

[plonk264]

hugely appreciated! i've been bashing my head into a wall trying to get aomenc to retain grain and/or detail at reasonable bitrates and this is my first hope so far!

[tdaede]

Does it reproduce with --low-latency true?

[quietvoid]

At first I thought this was something about key frames vs inter frames, but there really is significant smoothing every other frame.
@tdaede --low-latency makes the inter frame avg QP correct (same as --quantizer) while the default behavior doesn't.

I've uploaded my samples and logs here: https://mega.nz/folder/xMsHXTqD#4BYnbCiUVHM5-z4Fy3_Rng

I used the CLI from OP, these are the stats:
Default: Inter frame: 91 | avg QP: 119.00 | avg size: 182092 B
Low latency: Inter frame: 93 | avg QP: 80.00 | avg size: 302079 B

Interestingly, the metrics don't change much but the inter frame avg size is much smaller.
This is using rav1e built from master, 9770a0a

[plonk264]

--low-latency seemed to improve it (tho resulting in more than a 100% size increase), but it still has less detail/less prominent grain every other frame https://www.mediafire.com/file/wevkq9dufjgfwdp/fightclubsnippet-rav1e033-q72-speed6-lowlatency.ivf/file

edit: oops, in spite of naming it 033 (for 0.3.3), i actually didn't use it. i used 0.3.0

[tdaede]

--quantizer does not guarantee the frame average QP, it only guarantees the "base pyramid" frame QP. By default depth 2 pyramid coding is used (3 levels) so average QP will be higher. Pyramid coding is also better for compression.

Thanks, I also see it on the file you provided.

[tdaede]

FWIW nothing obvious on the analyzer. Either incorrect reference frames are being used, or somehow lambda or distortion scale is getting set wrong every other frame.

[master-of-zen]

Another Sample:

Frames

Encoding settings are ` --tiles 16 --quantizer 90 -s 5 --matrix BT2020NCL --trasnfer BT20202_10Bit `

Video:
vid.tar.gz

[tdaede]

Is this a regression for 0.5? I'm not sure whether to mark it as a blocker.

[shssoichiro]

I was looking into this a bit, and there's only one particular pattern that I found when going through the analyzer, which is present even in the low latency encode.

The even numbered frames are more likely to choose larger, DCT based blocks.

However, I haven't yet determined if this is the cause of the issue or a result of it.

[BlueSwordM]

Furthermore, this is an instance where limited SGR in CPU-6 improves subjective quality a bit in some blocks.

I also encounter the issue with even numbered frames choosing bigger partitions.
Currently investigating that last one.

[shssoichiro]

It looks like there was a bit of this effect present in the source, which let to some red herrings in investigating.

The end result we came to is that this is a natural effect of using frame pyramids, and all encoders are prone to it. aomenc definitely exhibits the same effect.

I do have a couple of ideas I want to try to see if I can improve this, however, so I'll leave this ticket open but remove the "bug" tag since it's functioning "as expected". (But we want it to function better than expected.)