Get Started

This guide will help you get started with webrtc-java quickly. We'll cover installation, a first call with the client API, and the same steps with the core API.

Supported Platforms

Maven Central artifacts contain native libraries that can be loaded on the following platforms:

	x64	arm	arm64
Linux	✔	✔	✔
macOS	✔	-	✔
Windows	✔	-	✔

INFO

Linux runtime dependencies

The Linux native library links the X11 client libraries and the GLib, GBM and libdrm libraries that libwebrtc's desktop capturers use. A minimal installation, for example a container image, needs them installed before the library can load. On Debian and Ubuntu:

apt-get install libx11-6 libxext6 libxfixes3 libxdamage1 libxtst6 libxrandr2 libxcomposite1 libglib2.0-0 libgbm1 libdrm2

Since 0.17.0, PulseAudio, udev and D-Bus are opened at runtime only when audio devices, cameras or the screen saver inhibition are used, so the library loads without libpulse0, libudev1 and libdbus-1-3. Earlier versions link them and need those three packages as well.

Installation

Maven

Add the following dependency to your pom.xml:

<dependency>
    <groupId>dev.onvoid.webrtc</groupId>
    <artifactId>webrtc-java</artifactId>
    <version>1.0.0</version>
</dependency>

INFO

Native library classifiers

The webrtc-java artifact will automatically add the dependency for the native library artifact matching the currently running operating system and architecture at runtime.

If you encounter issues loading the native library "webrtc-java", you can declare the platform-specific classifier explicitly. For example, for Windows 64-Bit:

<dependency>
    <groupId>dev.onvoid.webrtc</groupId>
    <artifactId>webrtc-java</artifactId>
    <version>1.0.0</version>
    <classifier>windows-x86_64</classifier>
</dependency>

Complete list of available classifiers:

• windows-x86_64
• windows-aarch64
• macos-x86_64
• macos-aarch64
• linux-x86_64
• linux-aarch64
• linux-aarch32

Using SNAPSHOT Versions

If you want to use the latest development version, you can use a SNAPSHOT release:

<dependency>
    <groupId>dev.onvoid.webrtc</groupId>
    <artifactId>webrtc-java</artifactId>
    <version>1.0.1-SNAPSHOT</version>
</dependency>

To use SNAPSHOT versions, you need to add the following repository configuration to your pom.xml:

<repositories>
    <repository>
        <name>Central Portal Snapshots</name>
        <id>central-portal-snapshots</id>
        <url>https://central.sonatype.com/repository/maven-snapshots/</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Gradle

Add the following to your build.gradle:

implementation "dev.onvoid.webrtc:webrtc-java:1.0.0"

For specific platforms, add the appropriate classifier:

implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "windows-x86_64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "windows-aarch64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "macos-x86_64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "macos-aarch64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "linux-x86_64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "linux-aarch64"
implementation group: "dev.onvoid.webrtc", name: "webrtc-java", version: "1.0.0", classifier: "linux-aarch32"

Where the Native Library Is Loaded From

The native library is looked for in this order:

1. In the directory named by the system property webrtc.java.library.dir, if the library is there. It is loaded in place, nothing is extracted. Use this when the library is installed with the application, for example by a jpackage installer or a distribution package.
2. In the native JAR on the class path. The library is extracted into the directory named by webrtc.java.extract.dir, or into the temporary directory (java.io.tmpdir) if that is not set, and loaded from there. Set webrtc.java.extract.dir when the temporary directory does not allow running code, as when /tmp is mounted with noexec.
3. Where System.loadLibrary looks: the libraries of the Java runtime, which is where a jlink image keeps those of its modules, then java.library.path.

java -Dwebrtc.java.library.dir=/opt/myapp/lib -jar myapp.jar
java -Dwebrtc.java.extract.dir=/var/lib/myapp/natives -jar myapp.jar

A library loaded from a directory keeps the file name it has in the native JAR, for example libwebrtc-java-linux-x86_64.so. The media module's library is found the same way; its FFmpeg libraries go next to it under their names in its native JAR (libavcodec.so.62 and so on). If the library is found nowhere, the UnsatisfiedLinkError lists every place that was tried.

The Linux libraries are built against glibc. They do not load on a system with the musl C library, such as Alpine Linux; the error says so when it finds one.

Your First Call

webrtc-java has two APIs. The Client API in dev.onvoid.webrtc.client negotiates by itself, keeps your code off WebRTC's threads, and closes everything it created with one call. The core API in dev.onvoid.webrtc mirrors WebRTC's own classes one to one, for full control. Start with the client API; Using the Core API below shows the same steps with the core API.

Create the Runtime

A WebRtc runtime owns everything the client API creates. Closing it, here at the end of the try-with-resources block, closes it all:

import dev.onvoid.webrtc.client.*;

try (WebRtc rtc = WebRtc.create()) {
    // Peers, tracks and channels go here.
}

WebRtc.create() captures the microphone and plays to the speakers. WebRtc.builder().audio(AudioMode.HEADLESS).build() needs no audio hardware at all, which suits servers and tests.

Connect to a Peer

Two peers find each other through signaling: they exchange offers, answers and ICE candidates over a channel you provide, such as a WebSocket. The library has no transport of its own; you hand it a function that sends, and pass it what the other side sent:

Peer peer = rtc.createPeer(PeerOptions.builder()
        .iceServer("stun:stun.l.google.com:19302")
        .polite(true)
        .signaling(message -> socket.send(message.toJson()))
        .build());

socket.onText(text -> peer.signal(SignalMessage.fromJson(text)));

The peer negotiates by itself whenever its tracks or channels change. When both sides send an offer at the same moment, the polite one gives way, so set polite(true) on exactly one of the two peers. The JSON messages are the ones browsers use, so the other side can be a web page.

Send and Receive Media

peer.addTrack(rtc.media().microphone());
peer.addTrack(rtc.media().camera());

peer.onTrack(track -> {
    System.out.println("Receiving " + track.kind());

    if (track.isVideo()) {
        track.asVideo().onFrame(frame -> render(frame));
    }
});

peer.onStateChange(state -> System.out.println("Call is " + state));

Handlers run on a thread of the peer's own, one at a time and in order. They may take their time and may call any method, so a slow handler never stalls WebRTC.

Exchange Messages

DataChannel chat = peer.createChannel("chat");

chat.onMessage(message -> System.out.println("Peer says: " + message.text()));
chat.send("Hello");

Messages sent before the channel opens are sent once it does.

Hang Up

peer.close() ends the call; closing the runtime ends every call and releases the camera, the microphone and the native resources. Operations return CompletableFutures, which a thread such as main may wait for:

peer.whenConnected().get(10, TimeUnit.SECONDS);
// ...
peer.close();

The Client API guide covers everything else: pushing your own audio and video, manual negotiation for WHIP and WHEP, statistics and errors. The ClientApiExample runs a whole call between two peers in one process, without any hardware.

Using the Core API

The core API leaves negotiation, threads and the lifetime of native objects to you. The steps below make the same call with it.

Initialize the WebRTC Library

First, create a PeerConnectionFactory which is the entry point for most WebRTC operations:

import dev.onvoid.webrtc.PeerConnectionFactory;

// Create a peer connection factory
PeerConnectionFactory factory = new PeerConnectionFactory();

Create a Peer Connection

To establish communication with another peer, you need to create a RTCPeerConnection:

import dev.onvoid.webrtc.RTCConfiguration;
import dev.onvoid.webrtc.RTCIceCandidate;
import dev.onvoid.webrtc.RTCIceServer;
import dev.onvoid.webrtc.RTCPeerConnection;
import dev.onvoid.webrtc.PeerConnectionObserver;

// Configure ICE servers (STUN/TURN)
RTCConfiguration config = new RTCConfiguration();
RTCIceServer iceServer = new RTCIceServer();
iceServer.urls.add("stun:stun.l.google.com:19302");
config.iceServers.add(iceServer);

// Create a peer connection with an observer to handle events
RTCPeerConnection peerConnection = factory.createPeerConnection(config, new PeerConnectionObserver() {
    @Override
    public void onIceCandidate(RTCIceCandidate candidate) {
        // Send the candidate to the remote peer, see below. The other
        // methods are optional.
    }
});

Signaling

WebRTC requires a signaling mechanism to exchange connection information between peers. The library doesn't provide this, so you'll need to implement it using your preferred method (WebSockets, HTTP, etc.).

Here's a simplified example of the signaling process:

// Create an offer
RTCOfferOptions options = new RTCOfferOptions();

peerConnection.createOffer(options, new CreateSessionDescriptionObserver() {
    @Override
    public void onSuccess(RTCSessionDescription description) {
        // Set local description
        peerConnection.setLocalDescription(description, new SetSessionDescriptionObserver() {
            @Override
            public void onSuccess() {
                // Send the offer to the remote peer via your signaling channel
                signalingChannel.send(description);
            }
            
            @Override
            public void onFailure(String error) {
                System.err.println("Failed to set local description: " + error);
            }
        });
    }
    
    @Override
    public void onFailure(String error) {
        System.err.println("Failed to create offer: " + error);
    }
});

// When you receive an answer from the remote peer via your signaling channel
signalingChannel.onMessage(message -> {
    RTCSessionDescription remoteDescription = parseSessionDescription(message);
    peerConnection.setRemoteDescription(remoteDescription, new SetSessionDescriptionObserver() {
        @Override
        public void onSuccess() {
            System.out.println("Remote description set successfully");
        }
        
        @Override
        public void onFailure(String error) {
            System.err.println("Failed to set remote description: " + error);
        }
    });
});

// Handle ICE candidates
PeerConnectionObserver connectionObserver = new PeerConnectionObserver() {
    @Override
    public void onIceCandidate(RTCIceCandidate candidate) {
        // Send the ICE candidate to the remote peer via your signaling channel
        signalingChannel.send(candidate);
    }
};

// When you receive an ICE candidate from the remote peer
signalingChannel.onIceCandidate(candidateMessage -> {
    RTCIceCandidate candidate = parseIceCandidate(candidateMessage);
    peerConnection.addIceCandidate(candidate);
});

Letting the Peer Connection Create the Description

setLocalDescription can also be called with only an observer. It then creates the offer or the answer, whichever the signaling state calls for, and applies it in one step, so nothing can change in between. That makes it the simpler way to implement "perfect negotiation". Read the result with getLocalDescription() once the observer reports success:

peerConnection.setLocalDescription(new SetSessionDescriptionObserver() {
    @Override
    public void onSuccess() {
        signalingChannel.send(peerConnection.getLocalDescription());
    }

    @Override
    public void onFailure(String error) {
        System.err.println("Failed to set local description: " + error);
    }
});

Adding Candidates and Knowing the Outcome

addIceCandidate(candidate) reports nothing and fails silently when the remote description is not set yet. The form that takes an AddIceCandidateObserver is queued behind any pending offer/answer operation, as the standard requires, so a candidate can be added right after setRemoteDescription() without waiting for it, and the observer says whether it was accepted:

peerConnection.addIceCandidate(candidate, new AddIceCandidateObserver() {
    @Override
    public void onSuccess() {
    }

    @Override
    public void onFailure(String error) {
        // e.g. "[INVALID_STATE] The remote description was null"
        System.err.println("Candidate rejected: " + error);
    }
});

A local candidate can also go away, for example when its network interface does. The observer's onIceCandidateRemoved(RTCIceCandidate) reports it; send it to the remote peer, which passes it to removeIceCandidate(candidate).

Once ICE has picked the candidate pair it uses, onSelectedCandidatePairChanged(RTCCandidatePairChangeEvent) reports both candidates of the pair and why it was chosen. It fires again whenever ICE switches pairs:

@Override
public void onSelectedCandidatePairChanged(RTCCandidatePairChangeEvent event) {
    System.out.println("Using " + event.getLocal().sdp + " <-> " + event.getRemote().sdp
            + " (" + event.getReason() + ")");
}

Media Streams

Accessing Media Devices

To query media devices (cameras and microphones):

import dev.onvoid.webrtc.media.MediaDevices;
import dev.onvoid.webrtc.media.video.VideoDevice;
import dev.onvoid.webrtc.media.audio.AudioDevice;

// Get available video devices
List<VideoDevice> videoDevices = MediaDevices.getVideoCaptureDevices();
for (VideoDevice device : videoDevices) {
    System.out.println("Video device: " + device.getName());
}

// Get available audio devices
List<AudioDevice> audioDevices = MediaDevices.getAudioCaptureDevices();
for (AudioDevice device : audioDevices) {
    System.out.println("Audio device: " + device.getName());
}

Creating Media Tracks

To create audio and video tracks:

import dev.onvoid.webrtc.media.video.VideoTrack;
import dev.onvoid.webrtc.media.audio.AudioTrack;
import dev.onvoid.webrtc.media.video.VideoDeviceSource;
import dev.onvoid.webrtc.media.audio.AudioOptions;
import dev.onvoid.webrtc.media.audio.AudioTrackSource;

// Create a video source and track
VideoDeviceSource videoSource = new VideoDeviceSource();
videoSource.setVideoCaptureDevice(videoDevices.get(0)); // Use the first available camera
VideoTrack videoTrack = factory.createVideoTrack("video0", videoSource);

// Create an audio source and track
AudioOptions audioOptions = new AudioOptions();
audioOptions.echoCancellation = true;
audioOptions.autoGainControl = true;
audioOptions.noiseSuppression = true;

// Create an audio source using the default audio device
AudioTrackSource audioSource = factory.createAudioSource(audioOptions);
AudioTrack audioTrack = factory.createAudioTrack("audio0", audioSource);

INFO

Audio devices can be specified via the AudioDeviceModule, see the guide on Audio Devices for more details.

Adding Tracks to Peer Connection

import java.util.ArrayList;
import java.util.List;
import dev.onvoid.webrtc.RTCRtpSender;

// Add tracks to the peer connection
List<String> streamIds = new ArrayList<>();
streamIds.add("stream1");
RTCRtpSender videoSender = peerConnection.addTrack(videoTrack, streamIds);
RTCRtpSender audioSender = peerConnection.addTrack(audioTrack, streamIds);

Receiving Tracks

A remote track arrives in onTrack with the transceiver that carries it. To learn when its media actually starts flowing, for example to show a video only once there is something to show, set an RTCRtpReceiverObserver; an RTCRtpSenderObserver does the same for the first packet a sender sends. Either is called right away when that has already happened:

@Override
public void onTrack(RTCRtpTransceiver transceiver) {
    RTCRtpReceiver receiver = transceiver.getReceiver();

    receiver.setObserver(mediaType -> System.out.println("First " + mediaType + " packet received"));

    MediaStreamTrack track = receiver.getTrack();

    if (track instanceof AudioTrack) {
        // Playout volume of this remote track, 0 to 10; 1 plays it as received.
        ((AudioTrack) track).setVolume(0.5);
    }
}

An observer belongs to the RTCRtpReceiver or RTCRtpSender object it was set through, and is removed when that object is disposed. getId() identifies a sender or receiver, and RTCRtpSender.getStreams() returns the stream IDs its track was added with.

Cleanup

Always properly dispose of resources when you're done:

// Dispose of senders (not owned by the peer connection)
videoSender.dispose();
audioSender.dispose();

// Dispose of tracks
videoTrack.dispose();
audioTrack.dispose();

// Dispose of sources
videoSource.dispose();
audioSource.dispose();

// Close peer connection and release resources
peerConnection.close();

// Dispose of factory
factory.dispose();

A track may be disposed, and a listener removed, from within one of its listeners: the listeners the notification has not reached yet are then not called, and what goes is freed once the notification is over. An observer registered on one thread while the object is disposed on another is freed once and never left registered. A factory disposed while a closed connection is still being freed is released once that connection is.

dispose() and close() may be called from any thread and more than once: only the first call frees anything, later ones do nothing. Once an object is disposed, its methods throw an IllegalStateException (a closed connection still reports its state). A method that another thread is running at that moment keeps the native object until it returns, and what the dispose frees is freed then.

Apart from the media sources, which only have dispose(), objects with a dispose() method also have a close() that does the same. A try-with-resources statement therefore releases them, also when an exception leaves the block:

try (AudioConverter converter = new AudioConverter(48000, 1, 16000, 1)) {
    converter.convert(input, output);
}

RTCDataChannel is the exception: its close() closes the channel, which then still has to be disposed.

Finding Objects That Were Never Disposed

An object that becomes unreachable without being disposed or closed leaks its native object. To find such objects, start the JVM with -Dwebrtc.java.leak.detection=true: each one is then reported, with its class, once the garbage collector finds it unreachable. Add -Dwebrtc.java.leak.trace=true to have the report tell where the object was created, at the cost of a stack trace per object.

java -Dwebrtc.java.leak.detection=true -Dwebrtc.java.leak.trace=true -jar app.jar

The reports are warnings through Logging, which writes nothing unless the application asks for it:

Logging.logToDebug(Logging.Severity.WARNING);

Only objects that own a native object and have a dispose() or close() method are watched: not views such as the track of RTCRtpSender.getTrack(), which own nothing, and not reference-counted objects such as video frame buffers. The detector only reports; it never frees anything itself, since a native object may still be in use on a WebRTC thread. While it is off, which is the default, nothing is watched.

Next Steps

Now that you have a basic understanding of webrtc-java, you can explore more advanced features:

• Client API - Everything the client API does, and how its threads work
• Data Channels - Learn more about data channels
• RTC Stats - Monitor connection quality
• Desktop Capture - Share screens and windows
• All Guides - Complete list of guides

For a complete API reference, check the JavaDoc.