HWAccelIntro
QuickSync
“Intel Quick Sync Video” is the marketing name for a set of hardware features available inside many Intel GPUs.

Hardware Support

(Each new platform supports a superset of the capabilities of the previous platform.)

API Support

The hardware can be accessed through a number of different APIs:

DXVA2 / D3D11VA

These are standard Windows APIs, which are implemented by the Intel graphics driver to support video decode.

libmfx on Linux

This is a library from Intel which can be installed as part of the Intel Media SDK, and supports a subset of encode and decode cases.

libmfx on Windows

This is a library supplied with Intel’s graphics drivers which supports all encode and decode cases.

Media Foundation

Another Windows API which supports some encode and decode cases via the Intel graphics drivers. Not supported in ffmpeg.

VAAPI with i965 driver

This is a mostly-free (but see below) driver for the libva / VAAPI instructure. Most Linux distributions package it.

VAAPI with iHD driver

The back-end of libmfx on Linux uses a modified libva and VAAPI driver; this can also be used directly by the user.

Linux

A whole open source media stack is provided with much wider HW platforms and Linux distributions supported.
(You can also download the installation binary Media Server Studio from ​https://software.intel.com/en-us/intel-media-server-studio/.
However, only limited HW platforms and Linux distributions are supported by Media Server Studio.)

Intel open source media stack

VAAPI VS libmfx

VAAPI / i965

• Packaged as standard in most Linux distributions.
• Runs on all usable hardware, including older and cheaper devices.
• Wider codec support.
• Common API for applications which may also use AMD / Nvidia hardware with Mesa.
• Interopable with standard APIs (EGL/OpenGL, OpenCL).

libmfx / iHD

• May give better encode quality in some cases (such as look_ahead).
• May give higher encode throughput in some cases (such as MFE, particularly on Iris graphics).
• Common API for applications which may also run on Windows.
• Interoperable with Intel OpenCL implementation.

Windows

(TODO)

Windows version MSDK can be got from ​https://software.intel.com/media-sdk

Running

VAAPI

See Hardware/VAAPI.

DXVA2 / D3D11VA

libmfx

The library has a large number of options to set, the possible valid values of are dependent on the version and hardware. libavcodec attempts to map common options sensibly to the libmfx options, but the mapping is crude and has holes, especially around rate control.

Installing the Media SDK on Linux

Install with open source MSDK stack

Install MSDK and iHD driver:

• Follow the guide of ​https://github.com/Intel-Media-SDK/MediaSDK to build from source code.
• Starting from Ubuntu 19.04, it is possible to install via apt (see: ​https://github.com/Intel-Media-SDK/MediaSDK/wiki/Intel-media-stack-on-Ubuntu):

sudo apt-get install libva-dev libmfx-dev

Install with MediaServerStudio?

Note that the kernel patches and modified system libraries are all required. It is recommended not to install this on any machine also used for other purposes, as it does not use normal distribution mechanisms and may break / be broken by other packages unexpectedly.

Build machine:

• Build and install the packaged dispatcher: <​https://github.com/lu-zero/mfx_dispatch>. (It is also possible to extract the necessary files from the Media SDK installation as described in the install manual – this is not recommended, just use the package instead.)

Target machine:

• Ensure the target machine has a supported CPU. Current versions only support gen8/gen9 graphics on expensive CPUs (“Xeon”/”Core i” branding). The same graphics cores on cheaper CPUs (“Pentium”/”Celeron”/”Atom” branding) are explicitly disabled, presumably for commercial reasons.
• Get a clean version of the supported kernel version (currently 4.4: <​https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.4.tar.xz>) and apply the supplied patches. Build and install.
• Build and install the supplied libva and libdrm trees.
• Run the Media SDK install script to install the proprietary components.
• Reboot.

Build FFmpeg with MSDK

 ./configure --enable-libmfx && make -j8 && make install

Licence status of i965 VAAPI driver

All of the user source code is available, but it includes proprietary blobs of compiled GPU code. Since the complete human-readable source is not available, this certainly renders it GPL-incompatible and is likely to cause issues with other copyleft licences. Using the libva dynamic-loading shim mostly sidesteps this, and therefore is encouraged.

Full Examples

Decode-only

Check supported qsv decoder list

ffmpeg -decoders|grep qsv

H264 video decode and download as yuv420p raw file

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p output.yuv

H264 video decode and display with sdl

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p -f sdl -

H264 video decode without output (this can be used as a crude benchmark of the decoder)

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -f null -

HEVC 10bit video decode and download as p010le yuv file

ffmpeg -hwaccel qsv -c:v hevc_qsv -load_plugin hevc_hw -i input.mp4 -vf hwdownload,format=p010 -pix_fmt p010le output.yuv

Encode-only

Check supported qsv decoder list

ffmpeg -encoders|grep qsv

Check private option list of h264 encoder

ffmpeg -h encoder=h264_qsv

Encode a 1080p raw yuv input as H264 with 5Mbps using VBR mode

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -i input.yuv -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -b:v 5M output.mp4

Encode a 1080p p010le raw yuv input as HEVC main10 profile (supported since Kaby Lake platform)

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -v verbose -f rawvideo -video_size 1920x1080 -pix_fmt p010le -i input.yuv -an -vf 'hwupload=extra_hw_frames=64,format=qsv' -c:v hevc_qsv -profile:v main10 output.mp4

Transcode

Software decode + h264 qsv encode with 5Mbps using CBR mode

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -b:v 5M -maxrate 5M output.mp4

Software decode + h264 qsv encode with CQP mode

ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -i input.mp4 -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -q 25 output.mp4

H264 qsv decode + h264 qsv encode with 5Mbps using VBR && Look_ahead mode (look_ahead may not be supported on some platforms)

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -b:v 5M -look_ahead 1 output.mp4

As previous, but use ICQ mode (which is similar to crf mode of x264)

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -global_quality 25 output.mp4

As previous, but use ICQ && Look_ahead mode

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -c:v h264_qsv -global_quality 25 -look_ahead 1 output.mp4

Hevc qsv decode + qsv scaling to 1080p@60fps + h264 qsv encode

ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080' -c:v h264_qsv output.mp4

Hevc qsv decode + qsv scaling to 1080p@60fps and color space convert from nv12 to rgb32 + output as sdl

ffmpeg -hwaccel qsv -c:v hevc_qsv -i input.mp4 -vf 'vpp_qsv=framerate=60,scale_qsv=w=1920:h=1080:format=rgb32,hwdownload,format=rgb32' -f sdl -

1:N transcoding: (MFE will be enabled by default if MSDK can support it)

   ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 \
        -filter_complex "split=2[s1][s2]; [s1]scale_qsv=1280:720[o1];[s2]scale_qsv=960:540[o2]" \
        -map [o1] -c:v h264_qsv -b:v 3200k 3200a.mp4 \
        -map [o2] -c:v h264_qsv -b:v 1750k 1750a.264

M:N transcoding

   ffmpeg -hwaccel qsv -c:v h264_qsv -i input1.mp4 -hwaccel qsv -c:v h264_qsv -i input2.mp4 \
     -filter_complex '[0:v]split=2[out1][out2],[1:v]split=2[out3][out4]' \
     -map '[out1]' -c:v h264_qsv output1.mp4 -map '[out2]' -c:v h264_qsv output2.mp4 \
     -map '[out3]' -c:v h264_qsv output3.mp4 -map '[out4]' -c:v h264_qsv output4.mp4

-qsv_device is an qsv customized option can be used to specify a hardware device and avoid the default device initialization failure when multiple devices usable (eg: an Intel integrated GPU and an AMD/Nvidia discrete graphics card). One example on Linux (more details please see ​https://trac.ffmpeg.org/ticket/7649)

ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i input.mp4 -c:v h264_qsv output.mp4

Hybrid transcode

It is also possible to use “vaapi decode + vaapi scaling + qsv encode” (available on Linux platform)

ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i input.mp4 -vf 'scale_vaapi=1280:720,hwmap=derive_device=qsv,format=qsv' -c:v h264_qsv output.mp4

Or use “dxva decode + qsv scaling + qsv encode” (available on Windows)

ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i input.mp4 -vf 'hwmap=derive_device=qsv,format=qsv,scale_qsv=w=1280:h=720' -c:v h264_qsv output.mp4