feat: list the hardware codecs of the hardware video codec factories

docs/guide/advanced/video-codecs.md

@@ -73,6 +73,8 @@ PeerConnectionFactory factory = PeerConnectionFactory.builder()
 
 On Linux, NVDEC needs an NVIDIA driver, which provides `libcuda.so.1` and `libnvcuvid.so.1`; both are loaded when the factory is made, so a machine without the driver is not affected. Streams with spatial layers are decoded in software.
 
+On Windows and Linux, H.264 is decoded in hardware in its 4:2:0 profiles. High 4:4:4 Predictive, which GPU decoders do not take, is decoded in software.
+
 AV1 on Windows needs the *AV1 Video Extension*, which Windows 11 includes, and VP9 the *VP9 Video Extensions*, a free component of the Microsoft Store that a system may not have; without it, VP9 is decoded in software. H.265 needs the *HEVC Video Extensions* of the Microsoft Store, or an HEVC decoder of the GPU vendor; without one, H.265 is not offered, since it has no software decoder. Streams with spatial layers (SVC) are decoded in software too, since their layers reach a decoder without the index that says where each ends. Decoded frames are copied from GPU memory back to system memory, where WebRTC's frames are, so hardware decoding pays off mostly at high resolutions and with many streams; at low resolutions WebRTC's software decoders are about as cheap. A hardware decoder that fails, or turns out to decode in software, is replaced by the software decoder, which starts with the next key frame.
 
 On macOS, VP9 decoding in hardware saves CPU: in a measurement on an Apple M2 it took a fraction of the processor time libvpx needs, but multi-threaded libvpx was about as fast on the clock. VideoToolbox decodes VP9 in hardware only where the Mac has a decoder for it, which Apple silicon does; a Mac without one decodes VP9 with libvpx as before. Profile 2 (10 bit), sizes below 64x64 or above 4096x4096, and streams with spatial layers (SVC) are decoded with libvpx too.
@@ -81,6 +83,22 @@ AV1 is decoded by VideoToolbox on Macs with an AV1 decoder, which are those with
 
 Which decoder a stream uses shows in the `decoderImplementation` statistic of its `inbound-rtp` stats, e.g. `MediaFoundation (Microsoft H264 Video Decoder MFT)`, `MediaFoundation (AV1VideoExtension)`, `VideoToolbox (VP9)`, `VideoToolbox (H.265)`, `VideoToolbox (AV1)` or `NVDEC (NVIDIA GeForce RTX 4070)`.
 
+### Listing Hardware Codecs
+
+Both hardware factories tell which of their codecs the GPU of the machine handles. `getHardwareCodecs()` returns those of `getSupportedCodecs()`, in the same order, that a hardware encoder or decoder takes; on macOS that includes H.264 through VideoToolbox. The other supported codecs are encoded or decoded in software. The hardware is probed once per process, which every factory then shares, and again only when one of the system properties above is set differently; a GPU, driver or codec extension installed while the process runs is seen by the next one:
+
+```java
+HardwareVideoEncoderFactory encoders = new HardwareVideoEncoderFactory();
+List<VideoCodecInfo> hardware = encoders.getHardwareCodecs();
+
+for (VideoCodecInfo codec : encoders.getSupportedCodecs()) {
+    String where = hardware.contains(codec) ? "hardware" : "software";
+    System.out.println(codec.getName() + " " + codec.getParameters() + ": " + where);
+}
+```
+
+The list is per format: on Windows, for example, Media Foundation encoders take H.264 Constrained Baseline and Baseline with packetization mode 1, so the Main profile and packetization mode 0 are listed as software. A hardware codec is the one tried first, not a guarantee: a stream that falls back to software, as described above, or asks for what the hardware does not do, such as SVC, is encoded or decoded in software. The `powerEfficientEncoder` and `powerEfficientDecoder` statistics of the `outbound-rtp` and `inbound-rtp` stats tell what a running stream uses.
+
 ### Native Codecs
 
 The encoders and decoders these factories create are `NativeVideoEncoder`s and `NativeVideoDecoder`s. They are placeholders that make WebRTC create the built-in codec, which then runs entirely inside WebRTC, so their methods are not to be called from Java.
@@ -248,7 +266,7 @@ Whatever an encoder, decoder or factory method throws is logged and treated as `
 - `VideoEncoderFactory`, `VideoDecoderFactory` — create the codecs of a factory.
 - `VideoEncoder`, `VideoDecoder`, `EncodedImage` — codecs implemented in Java.
 - `DefaultVideoEncoderFactory`, `DefaultVideoDecoderFactory` — the built-in codecs.
-- `HardwareVideoEncoderFactory`, `HardwareVideoDecoderFactory` — the built-in codecs, on the GPU where possible.
+- `HardwareVideoEncoderFactory`, `HardwareVideoDecoderFactory` — the built-in codecs, on the GPU where possible; `getHardwareCodecs()` lists those the GPU handles.
 - `RTCRtpTransceiver.setCodecPreferences()` — chooses among the negotiated codecs.
 
 For the full API, see the JavaDoc of the `dev.onvoid.webrtc.media.video.codec` package.

docs/guide/examples.md

@@ -32,9 +32,10 @@ This example is useful for applications that need to receive media streams from
 The [`CodecListExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/CodecListExample.java) demonstrates how to list all supported codecs with the WebRTC peer-connection-factory.
 
 **Key features demonstrated:**
-- Creating a `PeerConnectionFactory`
+- Creating a `PeerConnectionFactory` with the `HardwareVideoEncoderFactory` and `HardwareVideoDecoderFactory`
 - Getting the supported codecs for both sending and receiving audio and video
 - Displaying detailed information about each codec
+- Listing the video codecs of the hardware factories and marking those the GPU encodes or decodes, with `getHardwareCodecs()`
 
 This example is useful for understanding what codecs are available on the current system, which can help with debugging compatibility issues or optimizing media quality.
 

webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/CodecListExample.java

@@ -23,25 +23,51 @@
 import dev.onvoid.webrtc.RTCRtpCapabilities;
 import dev.onvoid.webrtc.RTCRtpCodecCapability;
 import dev.onvoid.webrtc.media.MediaType;
+import dev.onvoid.webrtc.media.video.codec.HardwareVideoDecoderFactory;
+import dev.onvoid.webrtc.media.video.codec.HardwareVideoEncoderFactory;
+import dev.onvoid.webrtc.media.video.codec.VideoCodecInfo;
 
 /**
  * Example demonstrating how to list all supported codecs with the WebRTC peer-connection-factory.
  * <p>
  * This example shows how to:
  * <ul>
- *   <li>Create a PeerConnectionFactory</li>
+ *   <li>Create a PeerConnectionFactory that encodes and decodes video on the GPU where it can</li>
+ *   <li>List the video codecs of the hardware factories, marking those the GPU handles</li>
  *   <li>Get the supported codecs for both sending and receiving audio and video</li>
  *   <li>Display detailed information about each codec</li>
  * </ul>
- * 
+ * <p>
+ * A hardware codec is tried first, not guaranteed: a stream falls back to software when the
+ * hardware fails or cannot do what the stream asks for. The stats of a running connection tell
+ * which one a stream uses ({@code powerEfficientEncoder} and {@code powerEfficientDecoder}).
+ *
  * @author Alex Andres
  */
 public class CodecListExample {
 
     public static void main(String[] args) {
-        PeerConnectionFactory factory = new PeerConnectionFactory();
+        HardwareVideoEncoderFactory hardwareEncoderFactory = new HardwareVideoEncoderFactory();
+        HardwareVideoDecoderFactory hardwareDecoderFactory = new HardwareVideoDecoderFactory();
+
+        PeerConnectionFactory factory = PeerConnectionFactory.builder()
+                .setVideoEncoderFactory(hardwareEncoderFactory)
+                .setVideoDecoderFactory(hardwareDecoderFactory)
+                .build();
 
         try {
+            // List the video codecs of the hardware factories and which of them the GPU handles.
+            System.out.println("\nVIDEO CODEC FACTORIES:");
+            System.out.println("---------------------");
+
+            System.out.println("\nVideo Encoders:");
+            printFactoryCodecs(hardwareEncoderFactory.getSupportedCodecs(),
+                    hardwareEncoderFactory.getHardwareCodecs());
+
+            System.out.println("\nVideo Decoders:");
+            printFactoryCodecs(hardwareDecoderFactory.getSupportedCodecs(),
+                    hardwareDecoderFactory.getHardwareCodecs());
+
             // Get receiver capabilities for audio and video.
             System.out.println("\nRECEIVER CAPABILITIES:");
             System.out.println("---------------------");
@@ -72,6 +98,43 @@ public static void main(String[] args) {
         }
     }
 
+    /**
+     * Prints the codecs of a hardware video codec factory, marking each as encoded or decoded in
+     * hardware or in software.
+     *
+     * @param supportedCodecs The codecs the factory supports.
+     * @param hardwareCodecs  The supported codecs the GPU handles.
+     */
+    private static void printFactoryCodecs(List<VideoCodecInfo> supportedCodecs,
+                                           List<VideoCodecInfo> hardwareCodecs) {
+        if (supportedCodecs.isEmpty()) {
+            System.out.println("  No codecs found.");
+            return;
+        }
+
+        for (int i = 0; i < supportedCodecs.size(); i++) {
+            VideoCodecInfo codec = supportedCodecs.get(i);
+            StringBuilder sb = new StringBuilder();
+
+            sb.append("  Codec #").append(i + 1).append(": ");
+            sb.append("Name: ").append(codec.getName());
+
+            Map<String, String> parameters = codec.getParameters();
+            if (!parameters.isEmpty()) {
+                sb.append(" | Params: ").append(parameters);
+            }
+
+            List<String> scalabilityModes = codec.getScalabilityModes();
+            if (!scalabilityModes.isEmpty()) {
+                sb.append(" | Scalability Modes: ").append(scalabilityModes.size());
+            }
+
+            sb.append(" | ").append(hardwareCodecs.contains(codec) ? "HARDWARE" : "SOFTWARE");
+
+            System.out.println(sb);
+        }
+    }
+
     /**
      * Prints detailed information about each codec in the provided list.
      * All information for a codec is printed in a single line for conciseness.

webrtc-jni/src/main/cpp/include/JNI_HardwareVideoDecoderFactory.h

@@ -24,10 +24,10 @@ extern "C" {
 #endif
 	/*
 	 * Class:     dev_onvoid_webrtc_media_video_codec_HardwareVideoDecoderFactory
-	 * Method:    getSupportedCodecsInternal
-	 * Signature: ()[Ldev/onvoid/webrtc/media/video/codec/VideoCodecInfo;
+	 * Method:    getCodecsInternal
+	 * Signature: ()[[Ldev/onvoid/webrtc/media/video/codec/VideoCodecInfo;
 	 */
-	JNIEXPORT jobjectArray JNICALL Java_dev_onvoid_webrtc_media_video_codec_HardwareVideoDecoderFactory_getSupportedCodecsInternal
+	JNIEXPORT jobjectArray JNICALL Java_dev_onvoid_webrtc_media_video_codec_HardwareVideoDecoderFactory_getCodecsInternal
 	(JNIEnv *, jclass);
 
 #ifdef __cplusplus

webrtc-jni/src/main/cpp/include/JNI_HardwareVideoEncoderFactory.h

@@ -24,10 +24,10 @@ extern "C" {
 #endif
 	/*
 	 * Class:     dev_onvoid_webrtc_media_video_codec_HardwareVideoEncoderFactory
-	 * Method:    getSupportedCodecsInternal
-	 * Signature: ()[Ldev/onvoid/webrtc/media/video/codec/VideoCodecInfo;
+	 * Method:    getCodecsInternal
+	 * Signature: ()[[Ldev/onvoid/webrtc/media/video/codec/VideoCodecInfo;
 	 */
-	JNIEXPORT jobjectArray JNICALL Java_dev_onvoid_webrtc_media_video_codec_HardwareVideoEncoderFactory_getSupportedCodecsInternal
+	JNIEXPORT jobjectArray JNICALL Java_dev_onvoid_webrtc_media_video_codec_HardwareVideoEncoderFactory_getCodecsInternal
 	(JNIEnv *, jclass);
 
 #ifdef __cplusplus

webrtc-jni/src/main/cpp/include/media/video/codec/DefaultVideoCodecFactories.h

@@ -17,10 +17,12 @@
 #ifndef JNI_WEBRTC_MEDIA_VIDEO_CODEC_DEFAULT_VIDEO_CODEC_FACTORIES_H_
 #define JNI_WEBRTC_MEDIA_VIDEO_CODEC_DEFAULT_VIDEO_CODEC_FACTORIES_H_
 
+#include "api/video_codecs/sdp_video_format.h"
 #include "api/video_codecs/video_decoder_factory.h"
 #include "api/video_codecs/video_encoder_factory.h"
 
 #include <memory>
+#include <vector>
 
 namespace jni
 {
@@ -31,14 +33,26 @@ namespace jni
 
 	// The default encoders, with the encoders of the GPU in front of them
 	// where the platform has them, falling back to the software ones. On
-	// macOS the default encoders already use VideoToolbox.
+	// macOS the default encoders already use VideoToolbox. The hardware is
+	// probed once per process and setting of the hardware codec options.
 	std::unique_ptr<webrtc::VideoEncoderFactory> CreateHardwareVideoEncoderFactory();
 	std::unique_ptr<webrtc::VideoDecoderFactory> CreateDefaultVideoDecoderFactory();
 
 	// The default decoders, with the decoders of the GPU in front of them
 	// where the platform has them, falling back to the software ones. On
-	// macOS the default decoders already use VideoToolbox.
+	// macOS the default decoders already use VideoToolbox. The hardware is
+	// probed once per process and setting of the hardware codec options.
 	std::unique_ptr<webrtc::VideoDecoderFactory> CreateHardwareVideoDecoderFactory();
+
+	// The supported formats of the factory, in their order, that it encodes
+	// on the GPU first: those it reports as power efficient, and on macOS
+	// H.264, which the default encoders encode with VideoToolbox.
+	std::vector<webrtc::SdpVideoFormat> GetHardwareVideoEncoderFormats(const webrtc::VideoEncoderFactory & factory);
+
+	// The supported formats of the factory, in their order, that it decodes
+	// on the GPU first, picked the way GetHardwareVideoEncoderFormats picks
+	// them.
+	std::vector<webrtc::SdpVideoFormat> GetHardwareVideoDecoderFormats(const webrtc::VideoDecoderFactory & factory);
 }
 
 #endif

webrtc-jni/src/main/cpp/include/media/video/codec/H264Format.h

@@ -0,0 +1,33 @@
+/*
+ * Copyright 2026 Alex Andres
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef JNI_WEBRTC_MEDIA_VIDEO_CODEC_H264_FORMAT_H_
+#define JNI_WEBRTC_MEDIA_VIDEO_CODEC_H264_FORMAT_H_
+
+#include "api/video_codecs/sdp_video_format.h"
+
+#include <vector>
+
+namespace jni
+{
+	// The H.264 formats the hardware decoders offer: those WebRTC decodes,
+	// without High 4:4:4 Predictive. GPU decoders take 4:2:0 only, so a 4:4:4
+	// stream goes to the software decoder from its start, instead of failing
+	// on the GPU and waiting for a key frame.
+	std::vector<webrtc::SdpVideoFormat> CreateH264HardwareDecoderFormats();
+}
+
+#endif

webrtc-jni/src/main/cpp/include/media/video/codec/HardwareVideoDecoderFactory.h

@@ -18,11 +18,13 @@
 #define JNI_WEBRTC_MEDIA_VIDEO_CODEC_HARDWARE_VIDEO_DECODER_FACTORY_H_
 
 #include "api/environment/environment.h"
+#include "api/video/resolution.h"
 #include "api/video_codecs/sdp_video_format.h"
 #include "api/video_codecs/video_decoder.h"
 #include "api/video_codecs/video_decoder_factory.h"
 
 #include <memory>
+#include <optional>
 #include <vector>
 
 namespace jni
@@ -34,17 +36,28 @@ namespace jni
 	class HardwareVideoDecoderFactory : public webrtc::VideoDecoderFactory
 	{
 		public:
+			// The hardware factories. They hold nothing but what probing for
+			// them found, so several factories, on any thread, can share them.
+			using HardwareFactories = std::vector<std::unique_ptr<webrtc::VideoDecoderFactory>>;
+
 			// The hardware factories are in order of preference.
-			HardwareVideoDecoderFactory(std::vector<std::unique_ptr<webrtc::VideoDecoderFactory>> hardware,
+			HardwareVideoDecoderFactory(std::shared_ptr<const HardwareFactories> hardware,
 				std::unique_ptr<webrtc::VideoDecoderFactory> software);
 			~HardwareVideoDecoderFactory() override = default;
 
 			std::vector<webrtc::SdpVideoFormat> GetSupportedFormats() const override;
 			std::unique_ptr<webrtc::VideoDecoder> Create(const webrtc::Environment & env,
 				const webrtc::SdpVideoFormat & format) override;
 
+			using webrtc::VideoDecoderFactory::QueryCodecSupport;
+
+			// A supported format is power efficient when a hardware factory
+			// supports it, since that factory's decoder is the one tried first.
+			CodecSupport QueryCodecSupport(const webrtc::SdpVideoFormat & format, bool reference_scaling,
+				std::optional<webrtc::Resolution> resolution) const override;
+
 		private:
-			const std::vector<std::unique_ptr<webrtc::VideoDecoderFactory>> hardware;
+			const std::shared_ptr<const HardwareFactories> hardware;
 			const std::unique_ptr<webrtc::VideoDecoderFactory> software;
 	};
 

webrtc-jni/src/main/cpp/include/media/video/codec/HardwareVideoEncoderFactory.h

@@ -18,11 +18,14 @@
 #define JNI_WEBRTC_MEDIA_VIDEO_CODEC_HARDWARE_VIDEO_ENCODER_FACTORY_H_
 
 #include "api/environment/environment.h"
+#include "api/video/resolution.h"
 #include "api/video_codecs/sdp_video_format.h"
 #include "api/video_codecs/video_encoder.h"
 #include "api/video_codecs/video_encoder_factory.h"
 
 #include <memory>
+#include <optional>
+#include <string>
 #include <vector>
 
 namespace jni
@@ -36,17 +39,29 @@ namespace jni
 	class HardwareVideoEncoderFactory : public webrtc::VideoEncoderFactory
 	{
 		public:
+			// The hardware factories. They hold nothing but what probing for
+			// them found, so several factories, on any thread, can share them.
+			using HardwareFactories = std::vector<std::unique_ptr<webrtc::VideoEncoderFactory>>;
+
 			// The hardware factories are in order of preference.
-			HardwareVideoEncoderFactory(std::vector<std::unique_ptr<webrtc::VideoEncoderFactory>> hardware,
+			HardwareVideoEncoderFactory(std::shared_ptr<const HardwareFactories> hardware,
 				std::unique_ptr<webrtc::VideoEncoderFactory> software);
 			~HardwareVideoEncoderFactory() override = default;
 
 			std::vector<webrtc::SdpVideoFormat> GetSupportedFormats() const override;
 			std::unique_ptr<webrtc::VideoEncoder> Create(const webrtc::Environment & env,
 				const webrtc::SdpVideoFormat & format) override;
 
+			using webrtc::VideoEncoderFactory::QueryCodecSupport;
+
+			// A supported format is power efficient when a hardware factory
+			// supports it, since that factory's encoder is the one tried first.
+			CodecSupport QueryCodecSupport(const webrtc::SdpVideoFormat & format,
+				std::optional<std::string> scalability_mode,
+				std::optional<webrtc::Resolution> resolution) const override;
+
 		private:
-			const std::vector<std::unique_ptr<webrtc::VideoEncoderFactory>> hardware;
+			const std::shared_ptr<const HardwareFactories> hardware;
 			const std::unique_ptr<webrtc::VideoEncoderFactory> software;
 	};