How It Works — ControlFloor

Architecture Overview

ControlFloor is composed of five components that work together seamlessly. The Server coordinates everything, Providers manage the physical connection to devices, and on-device agents handle automation and video.

+---------------------+ | Web Dashboard / | | Python API Client | +----------+----------+ | HTTPS / WebSocket | +----------v----------+ | ControlFloor Server | | (Go, SQLite, Auth) | +----------+----------+ | WebSocket + REST API | +---------------+--------------+ | | +---------v---------+ +---------v---------+ | Provider A | | Provider B | | (Mac Mini #1) | | (Mac Mini #2) | +---------+---------+ +---------+---------+ | | USB / Tunnels USB / Tunnels | | +-------+-------+ +-------+-------+ | | | | | | +--v-+ +--v-+ +--v-+ +--v-+ +--v-+ +--v-+ | D1 | | D2 | | D3 | | D4 | | D5 | | D6 | +----+ +----+ +----+ +----+ +----+ +----+ Each device runs: - CFA Agent (XCTest-based automation) - VidStream (ReplayKit broadcast extension) - UIInspector (element tree inspection)

The Components

🏗

ControlFloor Server

The central coordination layer. Written in Go, it manages authentication, device inventory, session state, and serves the web dashboard. It stores all data in a local SQLite database — no external database server required. Providers register with the Server and relay device commands through it.

🖥

ControlFloor Provider

Runs on each Mac that has iOS devices physically connected. The Provider detects USB-connected devices, manages their lifecycle (launching agents, handling disconnections), and relays commands between the Server and the on-device agents. Supports both direct USB and iOS 17+ RSD tunnel connections.

📱

CFA Agent

An XCTest-based automation agent that runs directly on each iOS device. It receives commands from the Provider over TCP-over-USB (or tunnel) and executes them using Apple's XCTest and XCUITest frameworks. Handles taps, swipes, text input, element inspection, screenshots, and more.

🎥

VidStream

A ReplayKit Broadcast Extension that captures the device screen at high quality and streams it to the Provider. Supports both video and audio streaming. Multiple clients can view the same device simultaneously.

🔍

UIInspector

A companion iOS app that provides rich element tree inspection. Queries the accessibility hierarchy to return detailed information about every UI element on screen — types, labels, values, and positions — enabling precise automation targeting.

🔑

ujsonin Parser

A custom high-performance JSON parser used throughout the ControlFloor stack. Available in C, Go, and Perl implementations. Optimized for the message-passing patterns used in device communication.

Communication Flow

All communication uses encrypted channels. The Server and Provider communicate over HTTPS and WebSocket. Providers communicate with devices over USB or RSD tunnels.

🔒

Client to Server

HTTPS for API calls and page loads. WebSocket for real-time communication (video streaming, live device interaction). All traffic is TLS-encrypted. Authentication via session cookies or bearer tokens.

🔄

Server to Provider

The Provider maintains a persistent WebSocket connection to the Server. Commands are relayed from the Server to the Provider and responses flow back. This design means Providers don't need to be directly accessible from the internet.

🔌

Provider to Device

TCP-over-USB via Apple's usbmuxd protocol for iOS 16 and earlier. For iOS 17+, ControlFloor supports multiple QUIC-based RSD tunnel methods: its own internal tunnel implementation, pymobiledevice3 individual tunnels, and pymobiledevice3's tunneld daemon mode.

Deployment Overview

ControlFloor supports simple single-Mac setups and multi-provider distributed deployments.

1

Download & Install

Run the install script on each Mac. It sets up the Server (on one Mac) and Provider (on each Mac with devices).

2

Sign & Deploy iOS Apps

Re-sign the CFA Agent, VidStream, and UIInspector with your Apple Developer certificate and deploy to devices.

3

Register & Run

Register the Provider with the Server, start the Provider, and your devices appear in the web dashboard ready for use.

Full Setup Guide

See It in Action

Download ControlFloor and have your first device automated in minutes. No credit card required to get started.

Get Started Free