Repository navigation
fix: keep hardware encoders for sources whose first frame has an odd size - #324
Merged
Merged
Conversation
WebRTC asks a source for the encoder's requested_resolution_alignment only after the encoder has been set up, so the first frame of a window capture or a camera with an odd size reaches InitEncode as it is. NvencVideoEncoder refused it with WEBRTC_VIDEO_CODEC_ERR_PARAMETER, and the fallback encoder then left the whole session to OpenH264. The encoder now takes the even size below an odd one and drops the last row and column of such frames, which needs no copy. New tests send 1080p, an odd 641x361, a change of resolution, a bitrate cap that is lifted again, fifteen calls one after the other (a leaked session would run the GPU out of them) and four calls at once, and require NVENC for all of them. The odd size failed before the change. Run on an NVIDIA GeForce RTX 3080 on Windows, also with -Xcheck:jni.
The same as for NVENC: WebRTC asks a source for the encoder's requested_resolution_alignment only after the encoder has been set up, so an odd first frame reached MFVideoEncoder::InitEncode, which refused it with WEBRTC_VIDEO_CODEC_ERR_PARAMETER, and the fallback encoder left the whole session to OpenH264. The encoder now takes the even size below an odd one and drops the last row and column of such frames, which needs no copy. The input type and the encoded size follow the even size. Compiled, and the NVENC path next to it is unchanged (all tests of HardwareVideoEncoderIntegrationTest pass), but not run with a Media Foundation encoder: the only H.264 one on the machine it was written on is a leftover AMD driver component that cannot start without the GPU. hardwareEncodesOddSize covers it wherever Media Foundation is the first hardware encoder.
devopvoid
changed the title
The same as for NVENC and Media Foundation: WebRTC asks a source for the encoder's requested_resolution_alignment only after the encoder has been set up, so an odd first frame reached VaapiH264Encoder::InitEncode, which refused it with WEBRTC_VIDEO_CODEC_ERR_PARAMETER, and the fallback encoder left the whole session to OpenH264. The encoder now takes the even size below an odd one and drops the last row and column of such frames, which needs no copy. The macroblock counts, the cropping in the sequence parameters and the encoded size follow the even size, which the cropping (in units of two samples) needs anyway. Neither compiled nor run: this machine is Windows without a Linux toolchain or WSL, and has no VA-API device. The change is the one made to NVENC, which was built and tested, and CI compiles this file on the Linux builds. hardwareEncodesOddSize covers it on a machine where VA-API is the first hardware encoder.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WebRTC asks a source for the encoder's
requested_resolution_alignmentonly after the encoder has been set up, so the first frame of a window capture or a camera with an odd size reachesInitEncodeas it is.NvencVideoEncoder,MFVideoEncoderandVaapiH264Encoderrefused it withWEBRTC_VIDEO_CODEC_ERR_PARAMETER, and the fallback encoder then left the whole session to OpenH264 (Hardware encoder failed to initialize (-4), switching to software).All three encoders now take the even size below an odd one and drop the last row and column of such frames, which needs no copy. The input type and the encoded size follow the even size.
Tests (
HardwareVideoEncoderIntegrationTest), each requiring a hardware encoder wherewebrtc.test.hardwareEncoderis set:hardwareEncodesOddSize: 641x361. This failed before the NVENC change.hardwareEncodes1080phardwareFollowsResolutionChange: 320x240 to 640x480 and back.hardwareFollowsBitrateCap: the encoder is reconfigured in place and keeps sending; the cap is lifted again.hardwareRestartsCleanly: 15 calls one after the other, so a leaked session would run the GPU out of them.hardwareEncodesSeveralStreams: four calls at once.NVENC was run on an NVIDIA GeForce RTX 3080 on Windows: all 11 tests of the class pass, also under
-Xcheck:jni(noFATAL ERROR).The Media Foundation change is compiled but has not been run with a Media Foundation encoder. The only H.264 one on the machine it was written on is a leftover AMD driver component that fails to start without the GPU (
hr=0x8007000E), so with NVENC out of the way even an even-size call fell back to OpenH264.hardwareEncodesOddSizecovers it on a machine where Media Foundation is the first hardware encoder, for example an AMD or Intel GPU on Windows; it would be good to run it on one before merging. CI runners have no GPU, so they skip it.The VA-API change was neither compiled nor run on the machine it was written on (Windows, no Linux toolchain, no VA-API device). It is the change made to NVENC, which was built and tested, and the macroblock counts, the cropping in the sequence parameters and the encoded size follow the even size. CI compiles it on the Linux builds.
hardwareEncodesOddSizecovers it on a machine where VA-API is the first hardware encoder (an Intel or AMD GPU on Linux); please run it on one before merging.