Repository navigation
test: check that hardware decoders crop padded frames to the picture - #322
Merged
Merged
Conversation
A decoder puts a picture whose size is not a multiple of its block size into a larger surface: 360 rows, for one, sit in a texture of 368 or more. MFVideoDecoder copies the picture out of that surface, taking the chroma plane from below the padded height and cropping to the display aperture, but the tests sent 320x240, which needs no padding, and checked only the size and the first row of luma, so none of that was exercised. The new tests send 640x360 in H.264, AV1 and VP9 and check every decoded frame: the shape that was sent, the luma gradient in the first and the last row, and the chroma. The test call now sends flat chroma (U 90, V 160), so chroma read from the wrong offset, which would be luma or padding, shows. Each test runs where the codec is decoded in hardware and follows the properties that require a hardware decoder. On an AMD Radeon RX 9070 XT the three tests pass. With the chroma plane taken from the picture height instead of the texture's, all three fail, so the decoded textures are padded there and the tests see the difference.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests that the hardware decoders crop a padded frame to the picture. This was a known gap of #315: the decoder tests sent 320x240, which needs no padding, so the readback of a picture from a larger decoded surface never ran in a test.
Why it matters
A decoder puts a picture whose size is not a multiple of its block size into a larger surface: 360 rows sit in a texture of 368 or more, 1080 in 1088.
MFVideoDecodercopies the picture out of that surface. It takes the chroma plane from below the padded height (RowPitch * desc.Height) and crops to the display aperture. A mistake there, such as the chroma offset taken from the picture height, gives frames of the right size with the wrong colours, and the existing checks (frame size, and the first row of luma) would not notice.Tests
HardwareVideoDecoderIntegrationTesthas three new tests,hardwareCropsPaddedH264,hardwareCropsPaddedAv1andhardwareCropsPaddedVp9. Each sends 640x360 and checks every decoded frame:TestMediaCallnow sends flat chroma (U 90, V 160) instead of whatever the allocated buffer held, so chroma read from the wrong offset, which would be luma or padding, shows.hasPicturereuses the new ramp check.Each test runs where its codec is decoded in hardware, and follows the properties that require a hardware decoder (
webrtc.test.hardwareDecoder,webrtc.test.hardwareAv1Decoder,webrtc.test.hardwareVp9Decoder), as the other tests of the class do. On macOS the H.264 and VP9 tests run against VideoToolbox; AV1 is skipped there.Testing
On Windows 11 with an AMD Radeon RX 9070 XT:
MediaFoundation (Microsoft H264 Video Decoder MFT),MediaFoundation (AV1VideoExtension)andMediaFoundation (VP9VideoExtensionDecoder).U not at 90. The decoded textures are padded for all three codecs, and the tests see the difference. The decoder was restored afterwards; this PR changes tests only.TestMediaCall: 46 tests, 0 failures, 2 skipped (macDecodesH264WithVideoToolbox, and a hardware encoder test without its GPU).Not verified: macOS (the VideoToolbox paths these tests now run there), and Windows on Intel and NVIDIA GPUs. CI runners have no GPU decoder, so the tests are skipped there.