> ## Documentation Index > Fetch the complete documentation index at: https://docs.run.cloud/llms.txt > Use this file to discover all available pages before exploring further. # Control a Simulator > Send acknowledged taps, swipes, gestures, text, keys, buttons, and device controls to iOS and Android sessions. The CLI, TypeScript SDK, REST API, and embedded viewer use the same interaction model for iOS Simulator and Android Emulator sessions. Each request targets an active session, carries a correlation id and timeout, and returns a structured acknowledgement or error. ## Authenticate and create a session Install the CLI and sign in: ```bash theme={null} npm install -g runcloud runcloud login ``` For CI, the SDK, or direct REST requests, create an API key in the [run.cloud dashboard](https://run.cloud/dashboard) and put it in the process environment: ```bash theme={null} export RUN_CLOUD_API_KEY="rc_live_..." ``` Create a session with an app installed. The examples below keep the session id in the shell and release it on exit: ```bash iOS theme={null} SESSION_ID=$(runcloud ios create \ --install ./MyApp.tar.gz \ --inactivity-timeout 3m \ --hard-timeout 15m \ --json | jq -r '.id') trap 'runcloud ios delete "$SESSION_ID" >/dev/null 2>&1 || true' EXIT ``` ```bash Android theme={null} SESSION_ID=$(runcloud android create \ --install ./app-debug.apk \ --inactivity-timeout 3m \ --hard-timeout 15m \ --json | jq -r '.id') trap 'runcloud android delete "$SESSION_ID" >/dev/null 2>&1 || true' EXIT ``` Use a simulator-compatible iOS build or an emulator-compatible Android APK. See [Run an iOS Simulator](/ios/run-simulator) and [Run an Android Emulator](/android/run-emulator) for artifact and lifecycle details. ## Coordinate and input conventions Points use normalized display coordinates. The top-left corner is `{ x: 0, y: 0 }`, the center is `{ x: 0.5, y: 0.5 }`, and the bottom-right corner is `{ x: 1, y: 1 }`. Both endpoints are valid. Coordinates follow the current display orientation, so the same point always refers to the same visible part of the screen after rotation. `swipe` and `gesture` durations are in milliseconds. A gesture has 2 to 1,000 ordered steps, starts with `begin`, ends with `end`, and may have `move` steps between them. Every step has either one point or two points, and the point count cannot change during a gesture. `delayMs` is the wait before the next step; each delay can be 0 to 5,000 milliseconds and their sum cannot exceed the request timeout. The final `end` step has no next step, so omit its delay or set it to 0. Swipe duration defaults to 300 milliseconds. Key and button holds default to 50 milliseconds. Scroll deltas are normalized from -1 to 1 and at least one axis must be nonzero. Provide both `x` and `y` when anchoring a scroll. Digital Crown delta must be nonzero and between -10,000 and 10,000. `typeText` accepts 1 to 10,000 US ASCII characters, plus tabs and line feeds. Use `pressKey` when the physical key matters. Modifiers are `shift`, `control`, `alt`, and `meta`; each represents the left modifier key. A key or button hold can last 20 to 30,000 milliseconds. Swipe duration can be 50 to 30,000 milliseconds. ## CLI Replace `ios` with `android` to use the same command on an Android session. These commands return stable JSON with `--json`: ```bash theme={null} runcloud ios tap "$SESSION_ID" 0.50 0.23 --json runcloud ios swipe "$SESSION_ID" 0.50 0.65 0.50 0.54 \ --duration 300 --json runcloud ios type-text "$SESSION_ID" "Run Cloud" --json runcloud ios press-key "$SESSION_ID" enter --json runcloud ios press-key "$SESSION_ID" a --meta --duration 50 --json runcloud ios press-button "$SESSION_ID" home --json runcloud ios rotate "$SESSION_ID" landscape_left --json runcloud ios scroll "$SESSION_ID" 0 0.15 --x 0.50 --y 0.65 --json runcloud ios reload "$SESSION_ID" --json runcloud ios toggle-software-keyboard "$SESSION_ID" --json runcloud ios simulate-memory-warning "$SESSION_ID" --json ``` Pass gesture steps as JSON. Quoting the array prevents the shell from changing it: ```bash theme={null} runcloud ios gesture "$SESSION_ID" --json --steps '[ {"phase":"begin","points":[{"x":0.38,"y":0.43}],"delayMs":100}, {"phase":"move","points":[{"x":0.50,"y":0.45}],"delayMs":100}, {"phase":"end","points":[{"x":0.62,"y":0.46}]} ]' ``` A two-finger gesture uses two points in every step: ```bash theme={null} runcloud ios gesture "$SESSION_ID" --json --steps '[ {"phase":"begin","points":[{"x":0.42,"y":0.44},{"x":0.58,"y":0.44}],"delayMs":100}, {"phase":"move","points":[{"x":0.35,"y":0.44},{"x":0.65,"y":0.44}],"delayMs":100}, {"phase":"end","points":[{"x":0.35,"y":0.44},{"x":0.65,"y":0.44}]} ]' ``` iOS also supports rendering controls: ```bash theme={null} runcloud ios set-render-debug "$SESSION_ID" slowAnimations true --json runcloud ios set-render-debug "$SESSION_ID" slowAnimations false --json ``` Every interaction command accepts `--timeout ` from 100 to 60,000 (default 15,000), `--request-id `, and `--json`. Request ids can contain letters, numbers, `.`, `_`, `:`, and `-`, up to 128 characters. Capture either platform as a PNG: ```bash theme={null} runcloud ios screenshot "$SESSION_ID" --output ./ios.png --request-id ios-shot-1 --json runcloud android screenshot "$SESSION_ID" --output ./android.png --request-id android-shot-1 --json ``` The screenshot result includes the absolute output path, byte size, and SHA-256 digest. The CLI checks the PNG signature before writing the file. ## TypeScript SDK Install the SDK on Node.js 20 or newer: ```bash theme={null} npm install @run-cloud/sdk ``` Platform clients expose a generic `interact` method and named methods for every action: ```ts theme={null} import { writeFile } from "node:fs/promises"; import { Client, isSimulatorCapacityError, RunCloudError, } from "@run-cloud/sdk"; const cloud = new Client(); const assetId = process.env.RUN_CLOUD_ANDROID_ASSET_ID; if (!assetId) throw new Error("RUN_CLOUD_ANDROID_ASSET_ID is required"); const session = await cloud.android.create({ installAssets: [assetId], inactivityTimeout: "3m", hardTimeout: "15m", }).catch((error: unknown) => { if (isSimulatorCapacityError(error)) { console.error("Simulator capacity is temporarily unavailable; retry later."); } throw error; }); try { const tap = await cloud.android.tap( session.id, { x: 0.5, y: 0.23 }, { requestId: "android.tap.1", timeoutMs: 15_000 }, ); console.log(tap.requestId, tap.status, tap.durationMs); await cloud.android.swipe( session.id, { x: 0.5, y: 0.65 }, { x: 0.5, y: 0.54 }, { durationMs: 300 }, ); await cloud.android.typeText(session.id, "Run Cloud"); await cloud.android.pressKey(session.id, "enter"); const png = await cloud.android.screenshot(session.id, { requestId: "android-shot-1" }); await writeFile("android.png", png); } catch (error) { if (error instanceof RunCloudError) { console.error({ status: error.status, code: error.code, message: error.detail, retryable: error.retryable, requestId: error.requestId, action: error.action, }); } throw error; } finally { await cloud.android.delete(session.id); } ``` `cloud.ios` and `cloud.android` provide: | Method | Action | | ----------------------------------------------- | ---------------------------------------------------------------------------------------- | | `interact(id, interaction, options?)` | Send any typed `SimulatorInteraction`. | | `tap(id, point, options?)` | Tap one display point. | | `swipe(id, from, to, options?)` | Swipe between two points. Put `durationMs` in `options`. | | `gesture(id, steps, options?)` | Run a one- or two-finger gesture. | | `typeText(id, text, options?)` | Enter US ASCII text. | | `pressKey(id, key, options?)` | Press a semantic key. Put `modifiers` and `durationMs` in `options`. | | `pressButton(id, button, options?)` | Press a hardware or system button. Put `durationMs` in `options`. | | `rotate(id, orientation, options?)` | Set an absolute display orientation. | | `reload(id, options?)` | Reload the foreground app. | | `scroll(id, input, options?)` | Send normalized `deltaX` and `deltaY`, with optional `x` and `y` anchor. | | `toggleSoftwareKeyboard(id, options?)` | Toggle the on-screen keyboard. | | `simulateMemoryWarning(id, options?)` | Send a memory-warning event. | | `rotateDigitalCrown(id, delta, options?)` | Request Digital Crown input. Current mobile sessions return an unsupported-action error. | | `setRenderDebug(id, option, enabled, options?)` | Change an iOS render diagnostic. | | `screenshot(id, options?)` | Return a PNG as `Uint8Array`. | Interaction and screenshot options accept `requestId`, `timeoutMs`, and `signal`. Aborting a signal stops the caller from waiting; it does not release the session and cannot undo an action that the simulator has already accepted. Use `cloud.simulators` when the platform is selected at runtime. Its methods take the same arguments, plus `{ platform: "ios" | "android" }` in the options object. ## REST API Send an interaction to: ```text theme={null} POST /run-cloud/{platform}/{sessionId}/interactions ``` `platform` is `ios` or `android`. Authenticate with the same bearer API key used by the SDK: ```bash theme={null} curl --fail-with-body -sS \ -H "Authorization: Bearer $RUN_CLOUD_API_KEY" \ -H "Content-Type: application/json" \ "https://api.run.cloud/run-cloud/ios/$SESSION_ID/interactions" \ --data '{ "requestId":"ios.tap.1", "timeoutMs":15000, "action":"tap", "x":0.5, "y":0.23 }' | jq ``` The request is a flat object. Add `requestId` and `timeoutMs` to any action: | `action` | Action fields | | ------------------------ | ------------------------------------------------------- | | `tap` | `x`, `y` | | `swipe` | `from: { x, y }`, `to: { x, y }`, optional `durationMs` | | `gesture` | `steps: [{ phase, points, delayMs? }]` | | `typeText` | `text` | | `pressKey` | `key`, optional `modifiers`, optional `durationMs` | | `pressButton` | `button`, optional `durationMs` | | `rotate` | `orientation` | | `reload` | No action fields. | | `scroll` | `deltaX`, `deltaY`, optional paired `x` and `y` anchor | | `toggleSoftwareKeyboard` | No action fields. | | `simulateMemoryWarning` | No action fields. | | `rotateDigitalCrown` | `delta` | | `setRenderDebug` | `option`, `enabled` | Screenshot is a separate binary endpoint on both platforms: ```text theme={null} GET /run-cloud/{platform}/{sessionId}/screenshot ``` It returns `image/png` for an active session owned by the authenticated user. Send `X-Run-Cloud-Request-ID` with a 1-to-128-character correlation id. If you omit it, the API generates one and returns it in the same response header. Screenshot failures use the same JSON envelope as interactions, with `action: "screenshot"` and a stable `screenshot_*` error code. ## Acknowledgements and errors A successful interaction returns HTTP 200 and a typed result: ```json theme={null} { "ok": true, "requestId": "ios.tap.1", "sessionId": "sim_123", "platform": "ios", "action": "tap", "status": "completed", "acceptedAt": "2026-08-07T10:20:30.000Z", "completedAt": "2026-08-07T10:20:30.042Z", "durationMs": 42, "result": { "pointCount": 1 } } ``` The acknowledgement confirms that the control operation was accepted and dispatched. It cannot prove that the app handled the input. Read app state or take a screenshot when the visible outcome is part of the test. Every action has the documented action-specific `result` object except `reload`, `toggleSoftwareKeyboard`, and `simulateMemoryWarning`, which omit `result`. Errors keep the same envelope in REST, SDK metadata, and CLI JSON: ```json theme={null} { "ok": false, "requestId": "android.crown.1", "sessionId": "sim_...", "platform": "android", "action": "rotateDigitalCrown", "status": "failed", "acceptedAt": "2026-08-07T12:00:00.000Z", "completedAt": "2026-08-07T12:00:00.006Z", "durationMs": 6, "error": { "code": "unsupported_action", "message": "rotateDigitalCrown is not supported on Android", "retryable": false } } ``` Validation errors use `invalid_interaction`. A released, expired, or unknown session uses `active_session_not_found`. An action outside the platform matrix uses `unsupported_action`. Transport failures and timeouts are marked retryable when another attempt may succeed. Check the HTTP status as well as `error.code`; do not retry an invalid request or unsupported action. Interaction codes are `invalid_interaction`, `active_session_not_found`, `simulator_capacity_unavailable`, `unsupported_action`, `duplicate_request`, `interaction_cancelled`, `interaction_timeout`, `interaction_transport_error`, `interaction_invalid_response`, and `interaction_failed`. Screenshot codes are `invalid_screenshot_request`, `active_session_not_found`, `simulator_capacity_unavailable`, `screenshot_cancelled`, `screenshot_timeout`, `screenshot_transport_error`, `screenshot_invalid_response`, and `screenshot_failed`. The CLI reports locally invalid arguments as a non-retryable `invalid_request` JSON error before authenticating. The SDK rejects invalid arguments with `TypeError` or `RangeError` before making a request. The CLI writes the JSON error to stderr and exits nonzero. Pressing Ctrl-C while an interaction is pending returns status `cancelled`, code `interaction_cancelled`, and exit code 130. A CLI timeout returns status `timed_out` and code `interaction_timeout`. Screenshot capture uses `screenshot_cancelled`, `screenshot_timeout`, `screenshot_transport_error`, and `screenshot_invalid_response`. Neither cancellation nor timeout releases the session. SDK API failures throw `RunCloudError`. Interaction failures can populate `code`, `retryable`, `details`, `requestId`, `sessionId`, `platform`, `action`, `interactionStatus`, `acceptedAt`, `completedAt`, and `durationMs`. A caller `AbortSignal` rejects with its abort reason instead. For session creation, `isSimulatorCapacityError(error)` narrows the exact retry-safe capacity 503; other authentication, validation, and service errors do not match it. ## Platform capabilities | Control | iOS | Android | | --------------------------------------------------------------- | :-: | :-----: | | Tap, swipe, one- or two-finger gesture | Yes | Yes | | Text entry and semantic key press | Yes | Yes | | `portrait`, `landscape_left`, `landscape_right`, and app reload | Yes | Yes | | `portrait_upside_down` | No | Yes | | Scroll and software keyboard toggle | Yes | Yes | | Memory warning | Yes | Yes | | PNG screenshot | Yes | Yes | | Digital Crown rotation | No | No | | Render diagnostics | Yes | No | Button support differs by device: | Button | iOS | Android | | --------------------------------- | :-: | :-----: | | `home` | Yes | Yes | | `back` | No | Yes | | `appSwitcher`, `recents` | Yes | Yes | | `power`, `volumeUp`, `volumeDown` | Yes | Yes | | `menu` | No | Yes | | `sideButton`, `actionButton` | Yes | No | | `digitalCrown` | No | No | On iOS, `home`, `appSwitcher`, and `recents` are momentary navigation actions; `durationMs` does not turn them into long presses. Physical buttons honor the requested hold duration on both platforms. Render options are `colorBlendedLayers`, `colorCopiedImages`, `colorMisalignedImages`, `colorOffscreenRendered`, and `slowAnimations`. Orientations are `portrait`, `portrait_upside_down`, `landscape_left`, and `landscape_right`. Current run.cloud iPhone Simulator sessions do not render `portrait_upside_down`; the API, CLI, SDK, and iframe return a non-retryable `unsupported_action` acknowledgement with the supported orientations. Android Emulator sessions support all four values. ## Semantic keys All key names are case-sensitive: * letters: `a`, `b`, `c`, `d`, `e`, `f`, `g`, `h`, `i`, `j`, `k`, `l`, `m`, `n`, `o`, `p`, `q`, `r`, `s`, `t`, `u`, `v`, `w`, `x`, `y`, `z`; * digits: `0`, `1`, `2`, `3`, `4`, `5`, `6`, `7`, `8`, `9`; * editing: `enter`, `escape`, `backspace`, `tab`, `space`, `insert`, `delete`; * punctuation: `minus`, `equal`, `bracketLeft`, `bracketRight`, `backslash`, `semicolon`, `quote`, `backquote`, `comma`, `period`, `slash`; * navigation: `home`, `end`, `pageUp`, `pageDown`, `arrowRight`, `arrowLeft`, `arrowDown`, `arrowUp`; * system and function: `capsLock`, `f1`, `f2`, `f3`, `f4`, `f5`, `f6`, `f7`, `f8`, `f9`, `f10`, `f11`, `f12`, `printScreen`, `scrollLock`, `pause`, `numLock`; * number pad: `numpadDivide`, `numpadMultiply`, `numpadSubtract`, `numpadAdd`, `numpadEnter`, `numpad0`, `numpad1`, `numpad2`, `numpad3`, `numpad4`, `numpad5`, `numpad6`, `numpad7`, `numpad8`, `numpad9`, `numpadDecimal`. Android Emulator sessions return `unsupported_action` for the lock-state keys `capsLock`, `numLock`, and `scrollLock`. Other listed semantic keys are available on both platforms. ## Embedded viewer The React `RemoteControl` handle exposes corresponding typed methods. The acknowledged orientation and reload helpers are named `setOrientation` and `reloadApp` to distinguish them from legacy fire-and-forget controls. It sends each action through the signed iframe and resolves only when the matching `requestId` result arrives: ```tsx theme={null} const control = useRef(null); async function tapCenter() { const result = await control.current?.tap( { x: 0.5, y: 0.5 }, { timeoutMs: 15_000 }, ); console.log(result?.status); } console.log(result)} /> ``` For a raw iframe, post the flat interaction with `type`, `requestId`, and `timeoutMs`: ```ts theme={null} iframe.contentWindow?.postMessage( { type: "run-cloud:interaction", requestId: "viewer.tap.1", timeoutMs: 15_000, action: "tap", x: 0.5, y: 0.5, }, new URL(session.url).origin, ); ``` The iframe replies with `type: "run-cloud:interaction-result"`, the matching request id, platform, action, status, timing, and typed result or error fields. Acknowledged results and failures always include valid `acceptedAt`, `completedAt`, and `durationMs` timing. A browser-side timeout raised before any acknowledgement cannot identify a platform. Verify both `event.source` and `event.origin` before trusting a result. See [Embed a Simulator](/platform/embed-simulator) for a complete secure listener and session lifecycle example. To stop pending work, post `{ type: "run-cloud:interaction-cancel", requestId, action }` to the same exact origin. Cancellation is best effort and cannot undo an action that completed. ## Screenshot examples The maintained examples build a small native app, create a session, install the app, capture a PNG, validate its dimensions, and release the session. ```bash iOS theme={null} git clone https://github.com/newly-app/run-cloud-examples.git cd run-cloud-examples/ios-app-screenshot npm install export RUN_CLOUD_API_KEY="rc_live_..." npm run demo -- --json ``` ```bash Android theme={null} git clone https://github.com/newly-app/run-cloud-examples.git cd run-cloud-examples/android-app-screenshot npm install export RUN_CLOUD_API_KEY="rc_live_..." npm run demo -- --json ``` Both write `screenshots/run-cloud-proof.png`. Pass `--open` to open the signed viewer while the example runs, or `--app ` to use a prebuilt artifact. Building the iOS app requires macOS with Xcode. Building the Android app requires JDK 17 or newer and Android SDK 35 or newer. Source: [iOS screenshot example](https://github.com/newly-app/run-cloud-examples/tree/main/ios-app-screenshot) and [Android screenshot example](https://github.com/newly-app/run-cloud-examples/tree/main/android-app-screenshot).