Streamers

Introduction

The Streamer is a class that streams audio and video from a source to an endpoint. It is responsible for controlling the audio and video sources, encoders, and endpoint.

Multiple streamers are available depending on the number of independent outputs you want to have:

• SingleStreamer: for a single output (example: live or record)
• DualStreamer: for 2 independent outputs (example: live stream and record)
• for multiple outputs, you can use the StreamerPipeline class that allows to create more complex pipeline with multiple independent outputs (example: audio in one file, video in another file)

Single streamers

The SingleStreamer is a Streamer that streams to a single output. The implementation is the SingleStreamer. Underneath, it is StreamerPipeline with a single EncodingOutput.

The single streamers data flow is as follows:

flowchart LR
    subgraph SingleStreamer
        subgraph StreamerPipeline
            VideoSource[Video source]
            AudioSource[Audio source]

            subgraph EncodingOutput
                video((video))
                audio((audio))
                VideoEncoder[Video encoder]
                AudioEncoder[Audio encoder]
                Endpoint[Endpoint]

                audio --> AudioEncoder
                video --> VideoEncoder
                AudioEncoder --> Endpoint
                VideoEncoder --> Endpoint
            end

            VideoSource --> video
            AudioSource --> audio
        end
    end

• AudioOnlySingleStreamer: A streamer that streams from an audio source (microphone by default).
• VideoOnlySingleStreamer: A streamer that streams from a video source (microphone by default).
• cameraSingleStreamer: A factory to create a streamer with a camera source.
• videoMediaProjectionSingleStreamer: A factory to create a streamer with a media projection video source. You need to set activity result
• audioVideoMediaProjectionSingleStreamer: A factory to create a streamer with a media projection video source and a media projection audio source. You need to set activity result

By default the Streamer endpoint is the DynamicEndpoint which made the Streamer agnostic of the protocol. The DynamicEndpoint infers from the MediaDescriptor object passed to the Streamer by open or startStream methods.

Dual streamers

The DualStreamer is a Streamer that streams to 2 independent outputs. The implementation is the DualStreamer. Underneath, it is StreamerPipeline with 2 EncodingOutput.

Streamer pipeline

The StreamerPipeline offers a way to create a custom pipeline with multiple independent outputs. Add an EncodingOutput to the pipeline with createOutput method.

There are currently no limitations on the number of outputs you can create but be careful with the number of encoders you create. Each encoder will use CPU and memory resources.

// In this example, we create a streamer pipeline with 2 outputs: one for audio and one for video
val streamerPipeline = StreamerPipeline(context, withAudio = true, withVideo = true)

// Add sources
streamerPipeline.setAudioSource(MicrophoneSourceFactory())
streamerPipeline.setVideoSource(CameraSourceFactory())

// Add outputs
val audioOnlyOutput =
    streamerPipeline.createEncodingOutput(withVideo = false) as IConfigurableAudioEncodingPipelineOutput
val videoOnlyOutput =
    streamerPipeline.createEncodingOutput(withAudio = false) as IConfigurableVideoEncodingPipelineOutput

// Configure outputs
val audioConfig = AudioCodecConfig(mimeType = MediaFormat.MIMETYPE_AUDIO_OPUS)
val videoConfig = VideoCodecConfig(
    mimeType = MediaFormat.MIMETYPE_VIDEO_AVC,
    resolution = Size(VIDEO_WIDTH, VIDEO_HEIGHT)
)

audioOnlyOutput.setAudioCodecConfig(audioConfig)
videoOnlyOutput.setVideoCodecConfig(videoConfig)

// Run stream
val audioOnlyDescriptor = UriMediaDescriptor(FileUtils.createCacheFile("audio.ogg").toUri())
val videoOnlyDescriptor = UriMediaDescriptor(FileUtils.createCacheFile("video.mp4").toUri())
audioOnlyOutput.startStream(audioOnlyDescriptor)
videoOnlyOutput.startStream(videoOnlyDescriptor)