App Streaming

What It Is

App streaming runs a heavy application on a powerful org host and streams individual windows to a thin client via WebRTC. The user sees a unified desktop where some windows are local (file manager, terminal, browser) and some are remote (Blender, CAD software, IDE with large codebase). They don't perceive the difference except for slight latency on streamed content.

This is per-app, not per-desktop. The desktop compositor, taskbar, and system tray run locally. Only individual application windows are streamed.

Why It Matters

A family with one powerful desktop and several laptops shouldn't need to buy GPU hardware for every device. A business with thin clients at desks and a GPU rack in the server room shouldn't install applications on every machine. A roaming user who closes their laptop and opens a tablet should see the same Blender session, still rendering.

FortrOS handles this at the hypervisor level: the user's client VM runs on whatever hardware the org assigns (based on what the app needs), and streaming is transparent.

How It Works

The Streaming Pipeline

Remote host:
  App renders to Wayland surface (GPU)
    -> Capture DMA-BUF (zero-copy, GPU-native buffer)
    -> Hardware encode (NVENC/VAAPI, < 1ms, near-zero CPU)
    -> WebRTC video stream (adaptive bitrate)
    -> WebRTC data channel (input events: keyboard, mouse)

Thin client:
  WebRTC video stream received
    -> Hardware decode (VAAPI/VideoToolbox, GPU-native)
    -> Native Wayland surface (real OS window, not a browser tab)
    -> Input captured on this surface
    -> WebRTC data channel sends input back to remote host

Why Per-App, Not Per-Desktop

Whole-desktop streaming (RDP, VNC, Spice) sends the entire screen as one video stream. This means:

The compositor runs remotely (input lag on every interaction)
Window management is remote (dragging windows has network latency)
Local hardware is wasted (the local GPU does nothing)

Per-app streaming sends only the heavy windows remotely. The desktop runs locally, so window management, file browsing, and text editing are instant. Only the GPU-heavy or memory-heavy windows go through the network.

Why Native, Not Browser

The streaming client is a native Wayland application, not a browser tab.

Aspect	Browser	Native client
Decode	JavaScript + soft decode	Hardware GPU decode
Display	Canvas in a browser window (double composited)	Real Wayland surface (single compositor)
Input	JavaScript event handlers	Direct Wayland input protocol
Latency	5-15ms overhead	< 1ms overhead
Power	High (browser + JS + soft decode)	Low (hardware path)

A remote Blender window should feel like a local Blender window. Browser-based streaming adds enough latency and overhead to break that illusion.

Adaptive Bitrate

WebRTC's congestion control adjusts encoding parameters to network conditions:

Static content (IDE, document): Tiny P-frames, minimal bandwidth
Partial motion (scrolling, UI interaction): Proportional bandwidth
Full motion (3D viewport, video playback): Full bitrate
Network degradation: Reduce resolution before dropping frames; prioritize framerate over visual fidelity for interactive content

App Placement

When a user launches an application, the maintainer decides where to run it:

Check local hardware capabilities (GPU, RAM, CPU) against app requirements
If local hardware is sufficient: run locally (no streaming, best latency)
If not: query org for capable hosts (GPU-rich, lowest utilization)
Start the app on the selected host, stream windows back to the client

The user doesn't choose where apps run. FortrOS does, based on hardware. Users can override ("always run this locally" or "always stream this") via client settings.

Game Streaming

Games need lower latency and higher framerates than productivity apps:

60+ fps (vs 30 fps acceptable for productivity)
Controller passthrough (USB passthrough over overlay)
Adaptive quality: framerate > visual fidelity under network stress

FortrOS integrates with Sunshine/Moonlight for game streaming as a proven path. The per-app streaming infrastructure handles productivity apps; games use the dedicated game streaming protocol for optimal latency.

LAN Steam transfers work without special configuration: FortrOS ensures port access on the overlay, Steam's built-in mechanism handles the rest.

How FortrOS Uses It

Transparent to the user: The desktop is unified. Local windows and streamed windows are indistinguishable in the taskbar.
Hardware-aware placement: The reconciler decides local vs remote based on hardware probing and workload requirements.
Persistent remote apps: A Blender session rendering on Host-C keeps running even if the user closes their laptop. Reopening reconnects to the same process. The app doesn't know the user disconnected.
Audio: Separate WebRTC Opus stream, mixed locally by PipeWire. Not bundled with video encoding.
Clipboard: Remote copy -> data channel -> local clipboard. Paste flows the opposite direction. Standard pattern from RDP/Spice.