Skip to content

Pass encoded frames through in the media player instead of decoding and re-encoding #368

Open

Description

@devopvoid

MediaRecorder already writes encoded frames to a file without decoding or re-encoding. Playback does not: MediaPlayer / MediaFileSource decode with FFmpeg and push raw frames into a CustomVideoSource / CustomAudioSource, which WebRTC then encodes again.

When the source codec matches the one the sender negotiated, the player could hand FFmpeg's demuxed packets straight to the encoded frame injectors from #365 instead, keeping decoding as the fallback. For example, an H.264 RTSP camera would be forwarded without a decode and an encode, saving CPU and avoiding generation loss.

Things to handle:

  • H.264/H.265 from MP4 need Annex B (h264_mp4toannexb / hevc_mp4toannexb).
  • Streams with B-frames cannot pass through, as WebRTC does not reorder frames.
  • Key frame requests can only be met at the source's next key frame (seek for files, wait for live streams).
  • A fixed source cannot follow the bitrate the sender allows; at most it can drop frames.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions