Video Streaming

The ASG Client supports live video streaming via three protocols: SRT, RTMP/RTMPS, and WHIP (WebRTC). Protocol selection is automatic based on the stream URL provided by the cloud.

Overview

Video streaming enables:

Live video broadcasting from glasses
Real-time video analysis by apps
Remote assistance scenarios
Video recording to cloud servers

Supported Protocols

Protocol	URL Scheme	Service Class	Use Case
SRT	`srt://`	`SrtStreamingService`	Low-latency, resilient streaming (used by managed streams)
RTMP/RTMPS	`rtmp://`, `rtmps://`	`RtmpStreamingService`	Platform compatibility (YouTube, Twitch)
WHIP	`https://`, `http://`	`WhipStreamingService`	Ultra-low latency WebRTC ingest

Protocol detection is handled by StreamCommandHandler.detectProtocol() based on the URL scheme.

Stream Commands

The system uses four commands for stream control. These are sent from the phone to the glasses over BLE.

1. start_stream

Initiates a stream to the specified URL. The protocol is auto-detected from the URL scheme. Command Structure:

{
  "type": "start_stream",
  "streamUrl": "srt://server.com:4201?streamid=key&passphrase=secret",
  "streamId": "unique-stream-id",
  "video": {
    "width": 1280,
    "height": 720,
    "bitrate": 2000000
  },
  "audio": {
    "bitrate": 128000,
    "sampleRate": 44100
  }
}

Requirements:

Active WiFi connection
Valid stream URL (srt://, rtmp://, rtmps://, http://, or https://)
Sufficient battery level
Sufficient bandwidth

Response:

{
  "type": "stream_status",
  "status": "initializing",
  "streamId": "unique-stream-id",
  "resolvedConfig": {
    "transport": "rtmp",
    "video": {
      "width": 854,
      "height": 480,
      "captureWidth": 1920,
      "captureHeight": 1080,
      "bitrate": 1000000,
      "fps": 15
    },
    "audio": {
      "bitrate": 64000,
      "sampleRate": 44100,
      "echoCancellation": false,
      "noiseSuppression": false
    }
  }
}

resolvedConfig reports the effective stream settings after the glasses apply defaults, clamps, and camera-mode selection. video.width and video.height are the encoded output dimensions sent to the stream endpoint. video.captureWidth and video.captureHeight are the native camera buffer dimensions selected on the glasses before crop or downscale; when no separate capture mode is needed, they match width and height. video.bitrate is the encoded video bitrate in bits per second, and video.fps is the resolved capture/encode frame rate. The audio object reports the resolved audio bitrate, sample rate, echo cancellation, and noise suppression settings when audio is active.

2. stop_stream

Terminates the active stream (whichever protocol is running). Command Structure:

{
  "type": "stop_stream"
}

Response:

{
  "type": "stream_status",
  "status": "stopped",
  "streamId": "unique-stream-id"
}

3. keep_stream_alive

Keep-alive mechanism to prevent stream timeout. Must be sent at least every 60 seconds. Command Structure:

{
  "type": "keep_stream_alive",
  "streamId": "unique-stream-id",
  "ackId": "unique-ack-id"
}

ACK Response (from glasses):

{
  "type": "keep_alive_ack",
  "streamId": "unique-stream-id",
  "ackId": "unique-ack-id",
  "timestamp": 1234567890
}

4. get_stream_status

Queries the current streaming status. Command Structure:

{
  "type": "get_stream_status"
}

Response:

{
  "type": "stream_status",
  "streaming": true,
  "reconnecting": false,
  "resolvedConfig": {
    "transport": "rtmp",
    "video": {
      "width": 854,
      "height": 480,
      "captureWidth": 1920,
      "captureHeight": 1080,
      "bitrate": 1000000,
      "fps": 15
    },
    "audio": {
      "bitrate": 64000,
      "sampleRate": 44100,
      "echoCancellation": false,
      "noiseSuppression": false
    }
  }
}

The get_stream_status response uses the same resolvedConfig schema as start_stream: encoded output size in video.width / video.height, native camera capture size in video.captureWidth / video.captureHeight, video bitrate and fps, plus resolved audio settings.

Stream Lifecycle

Starting a Stream

Request received: Phone sends start_stream via BLE
Battery check: Verify sufficient battery level
WiFi check: Verify WiFi connection is active
Stop existing: Any running stream is stopped first
Protocol detection: URL scheme determines which service to use
Stream init: Initialize encoder and connection for the detected protocol
Start streaming: Begin sending video data
Status updates: Send status back to phone/cloud

Keep-Alive Mechanism

The stream has a 60-second timeout that requires periodic keep-alive messages:

Cloud sends keep-alive every 15 seconds with unique ackId
Glasses reset timeout and respond with ACK containing same ackId
If no keep-alive for 60 seconds, stream automatically stops
If 3 ACKs are missed, cloud marks connection as degraded

Cloud -> Glasses: keep_stream_alive (every 15s)
Glasses -> Cloud: keep_alive_ack (immediate response)

Stopping a Stream

Streams can stop in three ways:

Explicit stop: Via stop_stream command
Timeout: No keep-alive received for 60 seconds
Error: Network failure, encoder error, etc.

Implementation Details

StreamCommandHandler

Routes commands to the appropriate streaming service based on URL protocol:

// Protocol detection
private static Protocol detectProtocol(String url) {
    if (url.startsWith("srt://")) return Protocol.SRT;
    if (url.startsWith("rtmp://") || url.startsWith("rtmps://")) return Protocol.RTMP;
    if (url.startsWith("https://") || url.startsWith("http://")) return Protocol.WHIP;
    return Protocol.UNKNOWN;
}

Streaming Services

All three services share the same StreamingStatusCallback interface:

// SRT
SrtStreamingService.startStreaming(context, streamUrl, streamId, flash, sound, config);

// RTMP
RtmpStreamingService.startStreaming(context, streamUrl, streamId, flash, sound, config);

// WHIP (WebRTC)
WhipStreamingService.startStreaming(context, streamUrl, streamId, flash, sound, config);

Each service supports:

isStreaming() / isReconnecting() — state queries
stopStreaming(context) — stop and clean up
resetStreamTimeout(streamId) — keep-alive timeout reset
setStreamingStatusCallback(callback) — status event reporting

Status Messages

All services report status through the shared callback:

initializing — Stream setup in progress
streaming / active — Streaming successfully
reconnecting — Attempting to reconnect after failure
reconnected — Successfully reconnected
reconnect_failed — All reconnection attempts exhausted
error — Stream failed with error details
stopped — Stream terminated
timeout — Stream stopped due to keep-alive timeout

Network Requirements

Bandwidth

Minimum: 1 Mbps upload
Recommended: 2-3 Mbps upload
Adapts bitrate based on connection

WiFi Stability

Requires stable WiFi connection
Automatic reconnection on brief disconnects
Stops on extended network loss

Debugging

Log Filters

# All streaming logs
adb logcat | grep -E "StreamCommandHandler|RtmpStreaming|SrtStreaming|WhipStreaming"

# Keep-alive activity
adb logcat | grep "keep_stream_alive\|keep_alive_ack"

# Stream status
adb logcat | grep "stream_status"

Common Issues

Stream stops after ~60 seconds: Check keep-alive implementation
ACKs not received: Verify BLE communication both ways
Poor quality: Check WiFi signal strength and bandwidth
Can’t start stream: Ensure only one stream active at a time
Unknown protocol: Verify URL scheme is srt://, rtmp://, rtmps://, http://, or https://

​Video Streaming

​Overview

​Supported Protocols

​Stream Commands

​1. start_stream

​2. stop_stream

​3. keep_stream_alive

​4. get_stream_status

​Stream Lifecycle

​Starting a Stream

​Keep-Alive Mechanism

​Stopping a Stream

​Implementation Details

​StreamCommandHandler

​Streaming Services

​Status Messages

​Network Requirements

​Bandwidth

​WiFi Stability

​Debugging

​Log Filters

​Common Issues

Video Streaming

Overview

Supported Protocols

Stream Commands

1. start_stream

2. stop_stream

3. keep_stream_alive

4. get_stream_status

Stream Lifecycle

Starting a Stream

Keep-Alive Mechanism

Stopping a Stream

Implementation Details

StreamCommandHandler

Streaming Services

Status Messages

Network Requirements

Bandwidth

WiFi Stability

Debugging

Log Filters

Common Issues