Skip to content

test: check that hardware decoders crop padded frames to the picture - #322

Merged
devopvoid merged 1 commit into
mainfrom
test/padded-frame-crop
Oct 4, 2026
Merged

devopvoid merged 1 commit into
mainfrom
test/padded-frame-crop

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Tests that the hardware decoders crop a padded frame to the picture. This was a known gap of #315: the decoder tests sent 320x240, which needs no padding, so the readback of a picture from a larger decoded surface never ran in a test.

Why it matters

A decoder puts a picture whose size is not a multiple of its block size into a larger surface: 360 rows sit in a texture of 368 or more, 1080 in 1088. MFVideoDecoder copies the picture out of that surface. It takes the chroma plane from below the padded height (RowPitch * desc.Height) and crops to the display aperture. A mistake there, such as the chroma offset taken from the picture height, gives frames of the right size with the wrong colours, and the existing checks (frame size, and the first row of luma) would not notice.

Tests

HardwareVideoDecoderIntegrationTest has three new tests, hardwareCropsPaddedH264, hardwareCropsPaddedAv1 and hardwareCropsPaddedVp9. Each sends 640x360 and checks every decoded frame:

  • Shape: the size that was sent, or a scaled-down one of the same shape. Padding that came along would change the shape.
  • Luma: the gradient in the first and the last row.
  • Chroma: U and V within 16 of what was sent.

TestMediaCall now sends flat chroma (U 90, V 160) instead of whatever the allocated buffer held, so chroma read from the wrong offset, which would be luma or padding, shows. hasPicture reuses the new ramp check.

Each test runs where its codec is decoded in hardware, and follows the properties that require a hardware decoder (webrtc.test.hardwareDecoder, webrtc.test.hardwareAv1Decoder, webrtc.test.hardwareVp9Decoder), as the other tests of the class do. On macOS the H.264 and VP9 tests run against VideoToolbox; AV1 is skipped there.

Testing

On Windows 11 with an AMD Radeon RX 9070 XT:

mvn -pl webrtc-jni,webrtc verify -Dtest=HardwareVideoDecoderIntegrationTest,HardwareVideoEncoderIntegrationTest,RTCEncodedFrameTransformTests,RTCRtpEncodingParametersTests,VideoCodecFactoryTests -Dsurefire.failIfNoSpecifiedTests=false -Dwebrtc.test.hardwareDecoder=true -Dwebrtc.test.hardwareAv1Decoder=true -Dwebrtc.test.hardwareVp9Decoder=true
  • The three new tests pass, decoded by MediaFoundation (Microsoft H264 Video Decoder MFT), MediaFoundation (AV1VideoExtension) and MediaFoundation (VP9VideoExtensionDecoder).
  • Negative control: with the chroma plane taken from the picture height instead of the texture's, all three fail with U not at 90. The decoded textures are padded for all three codecs, and the tests see the difference. The decoder was restored afterwards; this PR changes tests only.
  • The decoder class and every other class that uses TestMediaCall: 46 tests, 0 failures, 2 skipped (macDecodesH264WithVideoToolbox, and a hardware encoder test without its GPU).

Not verified: macOS (the VideoToolbox paths these tests now run there), and Windows on Intel and NVIDIA GPUs. CI runners have no GPU decoder, so the tests are skipped there.

A decoder puts a picture whose size is not a multiple of its block size
into a larger surface: 360 rows, for one, sit in a texture of 368 or more.
MFVideoDecoder copies the picture out of that surface, taking the chroma
plane from below the padded height and cropping to the display aperture,
but the tests sent 320x240, which needs no padding, and checked only the
size and the first row of luma, so none of that was exercised.

The new tests send 640x360 in H.264, AV1 and VP9 and check every decoded
frame: the shape that was sent, the luma gradient in the first and the
last row, and the chroma. The test call now sends flat chroma (U 90,
V 160), so chroma read from the wrong offset, which would be luma or
padding, shows. Each test runs where the codec is decoded in hardware and
follows the properties that require a hardware decoder.

On an AMD Radeon RX 9070 XT the three tests pass. With the chroma plane
taken from the picture height instead of the texture's, all three fail,
so the decoded textures are padded there and the tests see the
difference.
@devopvoid
devopvoid merged commit d927436 into main Oct 4, 2026
22 checks passed
@devopvoid
devopvoid deleted the test/padded-frame-crop branch October 4, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant