Skip to content

feat: decode with Vulkan Video on Linux in the media player - #345

Merged
devopvoid merged 1 commit into
mainfrom
feat/media-linux-vulkan
Oct 11, 2026
Merged

devopvoid merged 1 commit into
mainfrom
feat/media-linux-vulkan

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Summary

On Linux the media player decoded in hardware on NVIDIA GPUs only, through NVDEC. This adds Vulkan Video, so the GPUs of Intel and AMD decode H.264, HEVC, VP9 and AV1 in hardware too. It is the first step towards hardware H.264, HEVC, VP9 and AV1 on Windows, macOS and Linux; Windows and macOS already have all four in the media player.

Follows #331 (hardware HEVC/AV1, HEVC in hardware only).

  • FFmpeg is built with --enable-vulkan and the h264_vulkan, hevc_vulkan, vp9_vulkan and av1_vulkan hwaccels on Linux x86-64 and ARM64, the targets that have NVDEC.
  • FFmpeg loads the Vulkan loader (libvulkan.so.1) with dlopen when a device is created, so the libraries keep loading on a machine without Vulkan, as they do without the NVIDIA driver. VA-API was the alternative, but FFmpeg links libva into libavutil, which would make the Linux jar fail to load wherever libva is not installed.
  • The build needs Vulkan headers newer than distributions ship (VP9 needs 1.4.317; Ubuntu 24.04 has 1.3.275), so the C headers of Khronos Vulkan-Headers v1.4.317 are vendored in dependencies/vulkan, unchanged, with a pkg-config file for configure, the way the NVDEC headers are. Their licenses (Apache-2.0 or MIT) go into the Linux platform jars under META-INF/licenses/vulkan.
  • VideoDecoder tries CUDA (NVDEC) first and Vulkan second on Linux, so NVIDIA machines keep the path they had.
  • HEVC stays hardware only: on an Intel or AMD GPU whose driver has no Vulkan Video decoding for it, HEVC does not play, as before.
  • The docs, the MediaPlayer Javadoc and the HEVC disclaimer in the NOTICE files name Vulkan Video. The disclaimer paragraph is also rewrapped: one of its lines ran past the wrap in feat: decode HEVC and AV1 in hardware in the media player #331.

Testing

  • Linux, in WSL (Debian 12, GCC 12): FFmpeg n8.1 configured and built with this module's Linux component list, NVDEC and Vulkan, against the vendored headers (with --disable-x86asm and without dav1d, which this check did not need). All four Vulkan hwaccels and the four NVDEC ones are enabled (CONFIG_VULKAN_1_4 is 1, so VP9 is in), and libavutil.so and libavcodec.so need only libc and libm, not libvulkan.
  • Windows x64: on top of current main, the media module builds and all 88 of its tests pass with hardware decoding required; the Windows FFmpeg build is unchanged.

Not verified

Decoding through Vulkan Video has not run on an Intel or AMD GPU: there is no Linux machine with one here, and CI runners have no GPU, so HardwareDecodingTest skips its hardware tests there. Which codecs work depends on the Mesa (ANV, RADV) or NVIDIA driver version; AV1 and VP9 Vulkan decoding need recent ones. The full Linux build of the module (with dav1d and x86 assembly) runs first in CI.

On Linux the player decoded in hardware on NVIDIA GPUs only, through
NVDEC. FFmpeg is now built with Vulkan Video as well, with the H.264,
HEVC, VP9 and AV1 hwaccels, so the GPUs of Intel and AMD decode all four
in hardware too. The player tries NVDEC first and Vulkan after it.

FFmpeg loads libvulkan.so.1 with dlopen when a device is created, so the
libraries load on a machine without Vulkan, as they do without the
NVIDIA driver. VA-API, the alternative, would have linked libva into
libavutil, and the Linux jar would not load where libva is missing.

The build needs Vulkan headers newer than distributions ship (VP9 needs
1.4.317), so the C headers of Khronos Vulkan-Headers v1.4.317 are
vendored, unchanged, with a pkg-config file for FFmpeg's configure, as
the NVDEC headers are. Their licenses go into the Linux platform jars.

The docs and the HEVC disclaimer in the NOTICE files name Vulkan Video;
the disclaimer paragraph is rewrapped, as one of its lines ran long.
@devopvoid
devopvoid merged commit 4ae21f7 into main Oct 11, 2026
24 checks passed
@devopvoid
devopvoid deleted the feat/media-linux-vulkan branch October 11, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant