Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/.vitepress/sidebar.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ function sidebarGuide(): DefaultTheme.SidebarItem[] {
{ text: 'Field Trials', link: '/advanced/field-trials' },
{ text: 'Encoded Transforms', link: '/advanced/encoded-transforms' },
{ text: 'Video Codecs', link: '/advanced/video-codecs' },
{ text: 'Audio Codecs', link: '/advanced/audio-codecs' },
],
},
]
Expand Down
50 changes: 50 additions & 0 deletions docs/guide/advanced/audio-codecs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Audio Codecs

A `PeerConnectionFactory` decides which audio codecs it can send and receive through an audio encoder factory and an audio decoder factory. Unless told otherwise it uses all codecs built into WebRTC. This guide explains how to limit them, for example to offer only Opus, or to put a codec first that a remote endpoint expects.

The API lives in the `dev.onvoid.webrtc.media.audio.codec` package.

## Built-in Codecs

`BuiltinAudioEncoderFactory` and `BuiltinAudioDecoderFactory` provide WebRTC's audio codecs: Opus, G722, PCMU and PCMA. Ask a factory for the exact list:

```java
for (AudioCodecInfo codec : new BuiltinAudioEncoderFactory().getSupportedCodecs()) {
System.out.println(codec.getName() + "/" + codec.getClockRate() + "/" + codec.getChannels()
+ " " + codec.getParameters());
}
```

## Limiting and Ordering the Codecs

Pass codec names to a factory to keep only those codecs, in the given order of preference. Names are compared ignoring case, and a name that is not a built-in codec throws an `IllegalArgumentException` that lists the valid ones:

```java
PeerConnectionFactory factory = PeerConnectionFactory.builder()
.setAudioEncoderFactory(new BuiltinAudioEncoderFactory("opus"))
.setAudioDecoderFactory(new BuiltinAudioDecoderFactory("opus", "PCMU"))
.build();
```

The encoder factory decides what the factory can send and in which order its offers list the codecs; the decoder factory decides what it can receive. Without names, or without a factory set, all built-in codecs are used.

WebRTC adds the auxiliary codecs by itself, depending on the selected ones:

| Codec | Added when |
|---|---|
| `telephone-event` | A selected codec has a clock rate of 8000 or 48000 Hz (Opus, G722, PCMU, PCMA all do), so DTMF keeps working |
| `CN` | A selected codec runs at 8000 Hz and allows comfort noise |
| `red` | Opus is selected |

`getSupportedCodecs()` does not list them; `PeerConnectionFactory.getRtpSenderCapabilities(MediaType.AUDIO)` does.

::: tip
To change the codec order of a single connection rather than of the whole factory, use `RTCRtpTransceiver.setCodecPreferences()`. A factory's codec list is the set those preferences choose from.
:::

## Related API

- `PeerConnectionFactory.Builder.setAudioEncoderFactory()`, `setAudioDecoderFactory()` — set the factories.
- `BuiltinAudioEncoderFactory`, `BuiltinAudioDecoderFactory` — WebRTC's codecs, optionally limited.
- `AudioCodecInfo` — a codec's name, clock rate, channels and format parameters.
- [Video Codecs](/guide/advanced/video-codecs) — the same for video, including codecs written in Java.
1 change: 1 addition & 0 deletions docs/guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ This section provides detailed guides for various features of the webrtc-java li
- [Field Trials](/guide/advanced/field-trials) - Enabling experimental features and tuning WebRTC internals
- [Encoded Transforms](/guide/advanced/encoded-transforms) - Reading and changing encoded frames, e.g. for end-to-end encryption
- [Video Codecs](/guide/advanced/video-codecs) - Adding video codecs implemented in Java, or limiting the built-in ones
- [Audio Codecs](/guide/advanced/audio-codecs) - Limiting and ordering the built-in audio codecs

## Additional Resources

Expand Down
2 changes: 2 additions & 0 deletions webrtc-jni/src/main/cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ file(GLOB SOURCES_API_AUDIO "src/api/audio/*.cpp")
file(GLOB SOURCES_MEDIA "src/media/*.cpp")
file(GLOB SOURCES_MEDIA_AUDIO "src/media/audio/*.cpp")
file(GLOB SOURCES_MEDIA_AUDIO_OS "src/media/audio/${SOURCE_TARGET}/*.cpp")
file(GLOB SOURCES_MEDIA_AUDIO_CODEC "src/media/audio/codec/*.cpp")
file(GLOB SOURCES_MEDIA_VIDEO "src/media/video/*.cpp")
file(GLOB SOURCES_MEDIA_VIDEO_CODEC "src/media/video/codec/*.cpp")
file(GLOB SOURCES_MEDIA_VIDEO_CODEC_OS "src/media/video/codec/${SOURCE_TARGET}/*.cpp")
Expand Down Expand Up @@ -81,6 +82,7 @@ list(APPEND SOURCES
${SOURCES_MEDIA}
${SOURCES_MEDIA_AUDIO}
${SOURCES_MEDIA_AUDIO_OS}
${SOURCES_MEDIA_AUDIO_CODEC}
${SOURCES_MEDIA_VIDEO}
${SOURCES_MEDIA_VIDEO_CODEC}
${SOURCES_MEDIA_VIDEO_CODEC_OS}
Expand Down
21 changes: 21 additions & 0 deletions webrtc-jni/src/main/cpp/include/JNI_BuiltinAudioDecoderFactory.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions webrtc-jni/src/main/cpp/include/JNI_BuiltinAudioEncoderFactory.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions webrtc-jni/src/main/cpp/include/JNI_PeerConnectionFactory.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions webrtc-jni/src/main/cpp/include/JNI_WebRTC.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#define PKG_LOG "dev/onvoid/webrtc/logging/"
#define PKG_MEDIA "dev/onvoid/webrtc/media/"
#define PKG_AUDIO "dev/onvoid/webrtc/media/audio/"
#define PKG_AUDIO_CODEC "dev/onvoid/webrtc/media/audio/codec/"
#define PKG_VIDEO "dev/onvoid/webrtc/media/video/"
#define PKG_CODEC "dev/onvoid/webrtc/media/video/codec/"
#define PKG_DESKTOP "dev/onvoid/webrtc/media/video/desktop/"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_FACTORIES_H_
#define JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_FACTORIES_H_

#include "JavaRef.h"

#include "api/audio_codecs/audio_decoder_factory.h"
#include "api/audio_codecs/audio_encoder_factory.h"
#include "api/scoped_refptr.h"

#include <jni.h>

namespace jni
{
// Creates the native factory for a Java BuiltinAudioEncoderFactory: the
// built-in factory, limited to the codecs the Java object selected. A
// null object stands for all built-in codecs.
webrtc::scoped_refptr<webrtc::AudioEncoderFactory> CreateAudioEncoderFactory(JNIEnv * env,
const JavaRef<jobject> & factory);

// The same for a Java BuiltinAudioDecoderFactory.
webrtc::scoped_refptr<webrtc::AudioDecoderFactory> CreateAudioDecoderFactory(JNIEnv * env,
const JavaRef<jobject> & factory);
}

#endif
48 changes: 48 additions & 0 deletions webrtc-jni/src/main/cpp/include/media/audio/codec/AudioCodecInfo.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_INFO_H_
#define JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_INFO_H_

#include "JavaClass.h"
#include "JavaRef.h"

#include "api/audio_codecs/audio_format.h"

#include <jni.h>

#include <vector>

namespace jni
{
namespace AudioCodecInfo
{
JavaLocalRef<jobject> toJava(JNIEnv * env, const webrtc::SdpAudioFormat & format);

JavaLocalRef<jobjectArray> toJavaArray(JNIEnv * env, const std::vector<webrtc::AudioCodecSpec> & specs);

class JavaAudioCodecInfoClass : public JavaClass
{
public:
explicit JavaAudioCodecInfoClass(JNIEnv * env);

jclass cls;
jmethodID ctor;
};
}
}

#endif
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_SELECTION_H_
#define JNI_WEBRTC_MEDIA_AUDIO_CODEC_AUDIO_CODEC_SELECTION_H_

#include "api/audio_codecs/audio_format.h"

#include <string>
#include <vector>

namespace jni
{
// Whether a codec name is one of the selected names, ignoring case as SDP
// does.
bool IsSelectedAudioCodec(const std::string & name, const std::vector<std::string> & names);

// Returns the codecs with the selected names, ordered by the names and,
// for one name, by their order in `specs`. The Java side,
// AudioCodecSelection.select, orders the same way.
std::vector<webrtc::AudioCodecSpec> SelectAudioCodecs(const std::vector<webrtc::AudioCodecSpec> & specs,
const std::vector<std::string> & names);
}

#endif
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_MEDIA_AUDIO_CODEC_FILTERED_AUDIO_DECODER_FACTORY_H_
#define JNI_WEBRTC_MEDIA_AUDIO_CODEC_FILTERED_AUDIO_DECODER_FACTORY_H_

#include "api/audio_codecs/audio_decoder_factory.h"
#include "api/scoped_refptr.h"

#include <memory>
#include <string>
#include <vector>

namespace jni
{
// Offers and creates only the decoders of another factory that have one
// of the given names, in the order of the names.
class FilteredAudioDecoderFactory : public webrtc::AudioDecoderFactory
{
public:
FilteredAudioDecoderFactory(webrtc::scoped_refptr<webrtc::AudioDecoderFactory> factory,
std::vector<std::string> names);

std::vector<webrtc::AudioCodecSpec> GetSupportedDecoders() override;

bool IsSupportedDecoder(const webrtc::SdpAudioFormat & format) override;

// The variant with a codec pair ID calls this one.
using webrtc::AudioDecoderFactory::Create;

std::unique_ptr<webrtc::AudioDecoder> Create(const webrtc::Environment & env,
const webrtc::SdpAudioFormat & format) override;

private:
const webrtc::scoped_refptr<webrtc::AudioDecoderFactory> factory;
const std::vector<std::string> names;
};
}

#endif
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Copyright 2026 Alex Andres
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#ifndef JNI_WEBRTC_MEDIA_AUDIO_CODEC_FILTERED_AUDIO_ENCODER_FACTORY_H_
#define JNI_WEBRTC_MEDIA_AUDIO_CODEC_FILTERED_AUDIO_ENCODER_FACTORY_H_

#include "api/audio_codecs/audio_encoder_factory.h"
#include "api/scoped_refptr.h"

#include <memory>
#include <optional>
#include <string>
#include <vector>

namespace jni
{
// Offers and creates only the codecs of another factory that have one of
// the given names, in the order of the names.
class FilteredAudioEncoderFactory : public webrtc::AudioEncoderFactory
{
public:
FilteredAudioEncoderFactory(webrtc::scoped_refptr<webrtc::AudioEncoderFactory> factory,
std::vector<std::string> names);

std::vector<webrtc::AudioCodecSpec> GetSupportedEncoders() override;

std::optional<webrtc::AudioCodecInfo> QueryAudioEncoder(const webrtc::SdpAudioFormat & format) override;

std::unique_ptr<webrtc::AudioEncoder> Create(const webrtc::Environment & env,
const webrtc::SdpAudioFormat & format, Options options) override;

private:
const webrtc::scoped_refptr<webrtc::AudioEncoderFactory> factory;
const std::vector<std::string> names;
};
}

#endif
Loading
Loading