Skip to content

feat: add first-packet observers, sender and receiver IDs, and track volume - #338

Merged
devopvoid merged 2 commits into
mainfrom
feat/rtp-sender-receiver-extras
Oct 10, 2026
Merged

devopvoid merged 2 commits into
mainfrom
feat/rtp-sender-receiver-extras

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Stacked on #335: the new observers end with its ReportPendingException helper. The base becomes main once #335 is merged.

More of the Android SDK's RtpReceiver, RtpSender and AudioTrack API.

First-packet observers

  • RTCRtpReceiver.setObserver(RTCRtpReceiverObserver): onFirstPacketReceived(MediaType) when the receiver gets its first RTP packet, for example to show a remote video only once there is something to show.
  • RTCRtpSender.setObserver(RTCRtpSenderObserver): onFirstPacketSent(MediaType) for the first packet a sender sends.

An observer set after the packet already went through is called right away, before setObserver returns, as WebRTC does. null removes it. Both run on the signaling thread, and an exception they throw goes to the thread's uncaught exception handler.

Lifetime: WebRTC requires an observer to be removed before it is destroyed. The native observer is held by the Java object it was set through (an observerHandle field, as RTCDataChannel does) and freed when it is replaced, set to null or that object is disposed, each time after SetObserver has handed it back. That call runs on the signaling thread, the thread that calls the observer, so no call is in progress once it returns. A receiver has one observer, so setting one through a second Java object of the same receiver replaces it, and disposing the first object then removes it; the javadoc says to set it through one object, such as the one onTrack provides.

IDs and streams

  • RTCRtpReceiver.getId() and RTCRtpSender.getId().
  • RTCRtpSender.getStreams(), the counterpart of setStreams(). A track added without streams has none, as Unified Plan does.

Remote track volume

AudioTrack.setVolume(double) sets the playout volume of a remote track, from 0 (silent) to 10, where 1 plays it as received. Values outside that range, and NaN, throw IllegalArgumentException: WebRTC only checks the range in debug builds. The gain is applied after the audio is handed to an AudioTrackSink, so sinks are unaffected, and a local track's source ignores it.

Docs

get-started.md gains a "Receiving Tracks" section with the receiver observer and the volume.

Tests

RTCRtpSenderReceiverTests:

  • firstPacketEvents: in a real call with audio and video, the sender and both receivers report their first packet with the right media type, observers set afterwards are called during setObserver, and removing and disposing work.
  • senderIdAndStreams, streamsOfSenderWithoutStreams, receiverId: IDs are the same through every Java object of a sender or receiver; stream IDs follow addTrack and setStreams.
  • remoteTrackVolume, localTrackIgnoresVolume: valid volumes apply, invalid ones throw.

The full webrtc suite passes on Windows (259 run, 0 failures, 2 skipped), also under -Pjni-check with no JNI errors.

The callbacks WebRTC invokes ended with ExceptionCheck(), which turns a
Java exception thrown by the application into a C++ exception. It then
unwound through WebRTC's frames, which are built without unwinding
support, leaving WebRTC's locks held or ending the process, depending on
the platform. The same held for C++ exceptions thrown while converting a
callback's arguments.

Every such callback now catches its own C++ exceptions, turns them into
Java ones, and ends with ReportPendingException(), which clears the
pending exception and hands it to the thread's uncaught exception
handler: the peer connection, session description, data channel, DTLS,
DTMF and stats observers, the audio and video track sinks, the track
observer, the audio device's recorded-data sink and the desktop capture
callback. The add-ICE-candidate and data channel send observers, which
already reported exceptions this way, now share the helper.

While in these callbacks:
- the audio sinks hold their byte array in a JavaLocalRef, so it is
  freed when the callback throws, and pass jint values to the Java
  callback instead of size_t through varargs;
- the track observer no longer leaks a local reference per event;
- CustomVideoSource.pushFrame catches conversion errors instead of
  letting a C++ exception leave the JNI call.
…volume

- RTCRtpReceiver.setObserver(RTCRtpReceiverObserver) reports the first
  RTP packet a receiver receives, and RTCRtpSender.setObserver(
  RTCRtpSenderObserver) the first one a sender sends. An observer set
  after that is called right away. It belongs to the Java object it was
  set through, which removes it from WebRTC before freeing it, when it
  is replaced, set to null or disposed.
- RTCRtpReceiver.getId(), RTCRtpSender.getId() and
  RTCRtpSender.getStreams(), the counterpart of setStreams.
- AudioTrack.setVolume(double) sets the playout volume of a remote
  track, from 0 to 10, rejecting values outside that range, which
  WebRTC only checks in debug builds. A local track ignores it.

The observers report a Java exception to the thread's uncaught
exception handler, like the other callbacks WebRTC invokes.
@devopvoid
devopvoid force-pushed the feat/rtp-sender-receiver-extras branch from 0a44d4b to 9caf8aa Compare October 9, 2026 23:04
Base automatically changed from fix/observer-exceptions to main October 9, 2026 23:10
@devopvoid
devopvoid merged commit cbf4b81 into main Oct 10, 2026
1 check passed
@devopvoid
devopvoid deleted the feat/rtp-sender-receiver-extras branch October 10, 2026 12:25
SendableMetatype added a commit to EduGeyser/webrtc-java that referenced this pull request Oct 10, 2026
Brings in the work upstream added on October 10: the missing peer
connection signaling and control calls (devopvoid#334), callback exceptions kept
from unwinding through WebRTC (devopvoid#335), first-packet observers, sender and
receiver IDs and track volume (devopvoid#338), video processors for camera and
desktop frames (devopvoid#342), and fixes to audio device changes, camera
hotplug and media stream views (devopvoid#333, devopvoid#337, devopvoid#341, devopvoid#343).

Fork-side adaptations:
- setAudioPlayout and setAudioRecording reach the voice engine, so the
  data channels variant leaves them out, like the other audio natives.
- The new tests that need audio or video carry the media tag.
- The Maven publish action stays deleted.
SendableMetatype added a commit to EduGeyser/webrtc-java that referenced this pull request Oct 10, 2026
Brings in the work upstream added on October 10: the missing peer
connection signaling and control calls (devopvoid#334), callback exceptions kept
from unwinding through WebRTC (devopvoid#335), first-packet observers, sender and
receiver IDs and track volume (devopvoid#338), video processors for camera and
desktop frames (devopvoid#342), and fixes to audio device changes, camera
hotplug and media stream views (devopvoid#333, devopvoid#337, devopvoid#341, devopvoid#343).

Fork-side adaptations:
- setBitrate, setAudioPlayout and setAudioRecording reach the call and
  the voice engine, which a factory without a media engine does not
  create, so the data channels variant leaves them out, like the other
  media natives.
- The new tests that need audio or video carry the media tag.
- The Maven publish action stays deleted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant