Skip to content

[BUG] Runtime issues encountered with FFmpeg's VSR integration with software-based encoders #81

Open
@Brainiarc7

Description

@Brainiarc7

Describe the Bug
Hello there,

I've built Intel's VSR from the reisr4nas branch of ffmpeg cartwheel, and can likewise confirm that it works as expected; at least in GPU modes where OpenCL is available.

However, the same cannot be said of the out-of-the-box ASM modes with software-based encoding via libx264 and libx265, as documented on the wiki.

Of note is that this was built against Intel's Integrated Performance Primitives (IPP) standalone installation on Ubuntu 22.04LTS environment.

These trivial examples simply fail/exit with no output(s) on the console:

With `libx264:

ffmpeg -init_hw_device qsv=hw \
-init_hw_device opencl=cl@hw \
-filter_hw_device cl \
-noautorotate -i "bbb.mp4" \
-vf "raisr=filterfolder=/filters_2x/filters_highres:asm=opencl" \
-c:v libx264 -pix_fmt yuv420p -x264-params "scenecut=0" -maxrate:v 5000k -bufsize:v 1250k \
-keyint_min 120 -g:v 120 -preset:v faster -tune:v fastdecode -bf:v 0 \
-flags +global_header+cgop -f mp4 -y libx264_highres.mp4

With `libx265:

ffmpeg -init_hw_device qsv=hw \
-init_hw_device opencl=cl@hw \
-filter_hw_device cl \
-noautorotate -i "bbb.mp4" \
-vf "raisr=filterfolder=/filters_2x/filters_highres:asm=opencl" \
-c:v libx265 -pix_fmt yuv420p -x265-params "scenecut=0" -maxrate:v 5000k -bufsize:v 1250k \
-keyint_min 120 -g:v 120 -preset:v faster -tune:v fastdecode -bf:v 0 \
-flags +global_header+cgop -f mp4 -y libx265_highres.mp4

Now, if an OpenCL hardware context is provided as with the trivial example below for libx264, and asm=opencl is set, the filter runs/executes as expected:

ffmpeg -init_hw_device qsv=hw \
-init_hw_device opencl=cl@hw \
-filter_hw_device cl \
-noautorotate -i "bbb.mp4" \
-vf "raisr=filterfolder=/filters_2x/filters_highres:asm=opencl" \
-c:v libx264 -pix_fmt yuv420p -x264-params "scenecut=0" -maxrate:v 5000k -bufsize:v 1250k \
-keyint_min 120 -g:v 120 -preset:v faster -tune:v fastdecode -bf:v 0 \
-flags +global_header+cgop -f mp4 -y libx264_highres.mp4

However, attempting the same with libx265 fails with the error:

[libx265 @ 0x556433417c80] Unknown picture type encountered.
[vost#0:0/libx265 @ 0x5564333c5d40] Error submitting video frame to the encoder
[vost#0:0/libx265 @ 0x5564333c5d40] Error encoding a frame: Generic error in an external library
[vost#0:0/libx265 @ 0x5564333c5d40] Task finished with error code: -542398533 (Generic error in an external library)
[vost#0:0/libx265 @ 0x5564333c5d40] Terminating thread with return code -542398533 (Generic error in an external library)
[out#0/mp4 @ 0x5564333fb8c0] video:0KiB audio:135KiB subtitle:0KiB other streams:0KiB global headers:3KiB muxing overhead: 1.210572%
frame=    0 fps=0.0 q=0.0 Lsize=     137KiB time=N/A bitrate=N/A dup=2 drop=0 speed=N/A    
x265 [info]: frame I:      1, Avg QP:35.82  kb/s: 923.52  

encoded 1 frames in 2.38s (0.42 fps), 923.52 kb/s, Avg QP:35.82

Regardless of the OpenCL device type/context created.

In brief: The filter is virtually unusable with the software-based libx264 and libx265 encoders, as-is, out of the box.

Expected Behavior
The raisr filter should work in software mode as advertised, and with the native ASM modes, as compiled.
Only the OpenCL modes work, and again, only in GPU mode.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: Ubuntu 22.04LTS
  • Version: Tested against the master git tip and on the last release point of Intel's VSR.

Additional Context
As described above in the runtime issues.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions