Flash: driving an iOS app in milliseconds, from inside it

October 13, 2026 · 16 min read

We ship eight iOS apps from one shared codebase: news, sports, streaming, TV. Our agents read their code, write their fixes, and then have to check the result on a running app: open this screen, switch to that tab, tap the third card, confirm the header did not move. A human does that a few times an hour. An agent crawling every tab of every brand does it thousands of times a day.

The obvious tools are XCUITest and Maestro. Both work, both are mature, and both were built for a human who runs a suite before a merge and goes to get a coffee. Put them in an agent's inner loop and every tap costs most of a second, every check needs a screenshot or a tree dump in the model's context, and a ten-step path takes longer than the model took to decide it.

The answer is not a faster runner. It is no runner at all: move the driver into the app. That tool is called Flash, and on the same ten steps it is five to twenty times faster than XCUITest, depending on how you call it, while reading less and reporting more.

We built it for agents, and they use it for more than one job: QA testing, reproducing a bug, debugging with the app's own logs, verifying a code change they just made, and profiling performance while Flash plays a path.

Why driving from outside does not scale for agents

UI automation from outside the app pays the same four costs on every action:

• A second process. XCUITest drives the app from a test runner; Maestro adds a JVM and a driver on top. Every action crosses a process boundary, often twice.
• Coordinates. The runner resolves an element, turns it into a point, and synthesizes a touch at that point. A card that moved four pixels, a sheet half-covering the target, a tab bar in transit: the touch lands somewhere else.
• Waiting. A runner cannot see the app's animations from outside, so it waits for the screen to look still. That wait is paid on every step, and it is the largest cost.
• Reading. To know what is on screen, the runner snapshots the whole tree or takes a screenshot. For a model, a screenshot is the most expensive context there is, and a full tree is hundreds of kilobytes it pays for again on every later turn.

None of these are bugs. They are the price of being outside. Flash removes the outside.

The architecture

Flash is three pieces:

1. An HTTP server inside the app. A Debug-only module starts a small listener on a local port. It never ships to the App Store; it is not even compiled into Release.
2. Routes that act and read. Tap, type, swipe, switch tab, page, scroll to a row, open a screen, open a deep link or a push notification, read the actionable elements, take a screenshot, report whether the screen is idle.
3. A command line on the Mac. flash browses (flash tab Profile, flash open search, flash watch), bench flow replays a path from a text file, and bench maestro replays an existing Maestro YAML through Flash instead of Maestro's driver.

Everything that consumes Flash speaks the same HTTP: the agents, the CI job that produces JUnit reports, the flow canvas in our internal QA studio, where a tester drags "Profile tab" onto a canvas and watches it run, and an acceptance-test package that turns Given/When/Then scenarios written in French into Flash calls and runs them on an iPhone and an iPad at the same time.

Stage 1: a server compiled into the Debug build

The first build did not work. The app built, linked, launched, and nothing listened on the port. No error, no warning, no crash.

The server lives in a Swift package, gated by a compile-time flag so it cannot reach production. The app defined the flag in its own build settings. A -D flag does not cross a Swift package boundary. Inside the package the flag was undefined, the whole server compiled to an empty object file, and the app happily called an empty start().

The tell is the object file itself. Built for an iOS destination:

When the no-op object is larger than the server, the gate is broken. The flag now lives in the package manifest, on the target, and a second gate in the app decides whether start() is called at all. Both have to be on.

The second trap looked like a capture bug. The server resolves the app's window by scene activation state, and it first accepted only foregroundActive. A simulator booted headless, without Simulator.app in front, never reaches that state. The tree came back empty while the app rendered perfectly and its logs kept growing. The fix is to rank scenes by state and accept foregroundInactive when nothing better exists.

The third trap only showed up under load. The server's queue ran at utility priority, which costs nothing on an idle app. With the app busy, a drag that took 250 ms took two seconds. The queue now runs at user-interactive priority.

Stage 2: act by identifier, never by pixel

A tap is resolved in order, and each step is a fallback for the previous one:

1. the accessibility element carrying the identifier, through accessibilityActivate();
2. UIControl.sendActions(.touchUpInside);
3. the nearest accessibility element;
4. the gesture recognizer on the view itself;
5. an explicit failure.

The first path does most of the work, and it is a public API that SwiftUI buttons implement: Flash activates a button the way VoiceOver does, with no point and no synthesized touch. That is why the header can be mid-animation, or half-covered by a toast, and the tap still lands on the right control.

SwiftUI hides its elements from the usual lookups. An identifier such as DSHeader.search.action belongs to an accessibility element nested under a hosting view, invisible to accessibilityElement(at:). The walker descends recursively through accessibilityElements, guards against an element that returns itself, caps the depth, and reads the identifier by key, because these elements do not conform to UIAccessibilityIdentification. There is a second catch: SwiftUI builds that accessibility tree lazily, only when an accessibility client is listening. In a fresh demo app, /actionables first returned six elements, the tab bar and a title, and none of the SwiftUI identifiers. XCUITest and Maestro never see this, because their runner is the client that switches the tree on. An in-app driver has to switch it on itself: the Debug bridge enables accessibility automation for its own process at launch, and every identifier appears. Building that tree is paid once, about half a second on the first call after launch, so a benchmark makes one untimed request before it times anything.

The gesture fallback is the risky one: it reads a recognizer's targets through keys no contract guarantees. Two fences hold it. It is reached only after the three public paths fail, and it refuses any gesture whose view covers more than a quarter of the screen. Without that bound, tapping an icon inside a card fired the whole list's gesture and opened the article. When the fallback cannot fire, the tap returns an error, never a false success. It also bit back once: reading the target's action through key-value coding raised valueForUndefinedKey and took the app down with the tap. The fallback now reads the instance variable directly, and a tap can no longer crash the app it drives.

There used to be a sixth path. For a drag or a long press, /touch built a UITouch with an attached HID event and sent it through UIApplication.sendEvent, the technique KIF and EarlGrey use. It worked, and it was the one piece of Flash that could stop working on any iOS update. It is gone. Flash never synthesizes a touch now: a scroll is /swipe with a distance in points, clamped to the scroll view's insets; a page change sets the content offset and replays the scroll view's own delegate callbacks, from will-begin-dragging to did-end-decelerating, so the pager and its tab strip react as they would to a finger. A long press and a double tap are unsupported. And a tap that leaves the screen unchanged is a finding, not a success: the rule for agents is to report it, retry once, then read the log.

Stage 3: open a screen without touching anything

Tapping through four screens to reach one is the slowest way to get there. Our apps already know how to open any screen from a deep link, through a navigation enum the coordinators understand. Flash reuses it.

The navigation module registers four routes into the server at launch: /step, /back, /screen and /steps. The server never imports the navigation code; the app hands it closures. GET /steps lists the 49 cases one app accepts, with their arguments, so an agent discovers what it can open instead of guessing.

The reply is the part that matters. A runner knows it tapped; it does not know where it landed. Flash waits for the coordinator's did-navigate notification and returns the screen actually reached, with its content type and presentation mode:

{"status": "arrived", "ms": 114, "screen": {"contentType": "LIBRARY", "mode": "push"}}

POST /back returns in 21 ms.

The status is one of three words. arrived means the navigation service reported the screen. presented means a new controller sits on top: a sheet, an alert, a popup, which the navigation service never names. unchanged means nothing moved within 50 ms, and it is the one agents learn from: a Step that needs data the environment lacks, such as a test account or a feature flag, or a back on a stack already at its root, answers unchanged.

Entry points work the same way. POST /notification and POST /deeplink take a URL, and two launch arguments do the same from a closed app, so a test can begin with "the user taps a push notification" and end on the screen it opened.

Stage 4: read the screen for a few hundred bytes

Acting fast is half the job. The other half is knowing what is on screen without paying for it.

/hierarchy returns the full view tree as a flat array. On one home screen it is 404,648 bytes, and almost none of it is actionable. /actionables returns only what can be triggered: identifier, frame, traits and a kind (button, card, element), about 35 entries on the same screen, in 20 ms. A view survives only if it is on screen and a hit test at its centre reaches it, so hidden views, off-screen views and views covered by a sibling drop out. The full tree has the same filter on request: /hierarchy?visible=1 stops at the window's edge and at the first clipping view that falls outside it, and answers in 25 to 40 ms instead of 53.

Lists get their own reader. /rows names every row of the main list, built or not, in about 5 ms. /rows?all=1 labels an endless feed to its end in one in-app sweep, chunk by chunk, and a long-poll variant answers as soon as the list holds more rows than it did, so a crawler waits on the list growing, not on a timer.

Every reply also says where its time went. A Server-Timing header splits each request into queue, the wait for the app's busy main thread, work, Flash's own code, app, the app code Flash called, and wait, the time spent waiting for the app to react. An agent that sees a slow reply reads the header before blaming the tool: a large work is Flash, a large queue or wait is the app, and that is an app finding.

flash watch goes one step further. It reads the tree once, runs a set of small oracles over it in the shell, and prints only findings: a card whose title is blank while its siblings have one, an image missing where its peers have it. The 400 KB never leaves the shell; the model reads a few hundred bytes. One rule is printed in the tool's own help: an empty report means the oracles found nothing, not that the screen is correct.

Screenshots still exist, taken in the app itself: 9 ms for a 1x JPEG of the window or of one element by identifier, against a second or more through simctl. The rule for agents is to reach for them last.

Stage 5: the command line, and the flows

The CLI is deliberately thin. flash prints what it read and decides nothing; bench flow reads a text file, one step per line:

tab Profile
assert profile.login
tab Sections
tab Home
tap home.top.video
tap home.top.sports
tap home.top.tech
assert feed.tech.header
tab Profile
tab Home

A step that fails prints why and the flow moves on, so a run reports every broken step, not only the first. The first version did not do that. It was a shell script, and a failing tap exited the whole script from inside a shell function, with its output sent to /dev/null: the flow stopped at step seven with exit code 1 and not a single line saying why. The studio showed a red run and nothing else. That bug, and the cost of a process per step, are why the shell is gone. flash and bench are now one Python process each, with a raw-socket HTTP client instead of curl, started with python3 -S so it skips the site packages. A tab switch from the command line went from 754 ms to 145 ms; a chain of four steps from 8.1 s to 1.2 s. flash run demo + tab Profile + open settings runs the three in one process and stops at the first failure.

Flows wait on state, never on a timer. /idle walks the window's layers and reports busy while a scroll view is tracking or decelerating, or while any layer runs a finite animation; a spinner or a shimmer, which never ends, is ignored. It answers in 2 ms and the CLI polls it every 20 ms. Every sleep(1) the shell version carried is now a wait on /idle, including the one before each measurement in a flow.

A flow can also leave evidence. With a proof directory set, every step writes a JSON file with what it targeted, the element's frame and text, and a 1x JPEG of the screen at that moment, taken in the app for 9 ms. A measurement step prints what it got, what it wanted, the tolerance and the gap, so a run that fails on "header 2 pt too low" says exactly that, with the screenshot next to it.

Animations are off by default. flash run launches the app without them, and tab, row, page, open and back switch them off before they move, unless --animate asks to keep them for a human watching. A push then costs about 30 ms instead of a 0.5 s transition, and the reply still waits for the screen to appear.

Stage 6: the agent's other eye, the app's log

Flash says where the app is. It does not say why a screen came up empty. That part comes from a logging library compiled into the same Debug builds: it records every screen shown, every tap, every HTTP call with its status and latency, and what the app printed, as a timeline.

Our MCP server ties the two together. One tool builds the app if needed, launches it on a simulator it reserves, and returns the Flash port. The agent drives with flash. When something looks wrong, another tool returns the app's log as a short digest: the screens visited, the endpoints called, the errors, and the rendering anomalies the log already detects, such as a page that loaded with zero sections. A third tool profiles the app while it replays a list of Flash calls, and splits each call's time between the app, Flash and the transport.

So an agent that sees a blank card does not stop at "the card is blank". It can read, for instance, that the image request answered 400.

A real session

Our apps cannot appear here, so I built one that can. Pulse is a small SwiftUI news app with made-up stories: four bottom tabs, six top tabs on a swipeable pager, twelve sections, article pages, search, and a profile area with login, sign-up and settings. About thirty screens, an identifier on every control. It embeds the same Flash package as our apps, plus a forty-line bridge that maps /step to its router.

The recording below is one take, not edited: Claude Code in a terminal on the left, the simulator on the right, both captured at the same time. The only context Claude has is a one-page CLAUDE.md listing the commands, and it may run flash and bench, nothing else. The prompt:

Pulse is open on the simulator. Search for "river", open the Tech result, save it for later, then open Settings > Font size. Check each step landed, then give me the timings in a short table.

Claude listed what it could tap with bench act, then ran the path, checking each step with the reply of the step itself:

Five actions and their checks took 650 ms. The whole session took 36 s, and almost all of it was the model thinking. That is the point: the driver is no longer the slow part of the loop.

Two things in the take are worth seeing. Claude did not type into the search field and tap through results; it found that search is a Step that takes a query and opened the screen with the query already in it. And the first navigation call failed: Claude Code's sandbox blocked the CLI's temporary file. Claude said so, ran the step again outside the sandbox, and named the setting that would allow it. A tool that answers with a reason is one an agent can recover from.

The numbers

Start with the loop as an agent lives it. Same prompt, same model (Claude Code on Opus 5.5), same simulator: open Sections, the Tech section, its first story, then Profile, Settings and Font size. On the left, Claude Code uses its built-in iOS Simulator pane: take a screenshot, read it, tap at the coordinates it found, repeat. On the right, Claude Code uses Flash: one Bash call that chains six flash open commands, and the app's router opens each screen and replies with the screen that arrived and how long it took.

The six Flash steps take 418 ms in total, 34 to 127 ms each. Flash's 4.0 s is 2.7 s of the model reading the prompt, then one batched tool call of 1.3 s. The loop's 21.6 s is 19.1 s of the model reading screenshots between taps and 2.4 s of tools.

The same ten steps (three bottom tabs, three top tabs, two checks, back home) on Pulse, on one iOS 18.6 simulator, against the same app process, run three times per driver, one driver at a time.

Three details matter more than the headline ratio. Maestro varied by a factor of two between identical runs, from 8.6 s to 18.9 s once startup is removed; Flash varied by 160 ms. My first Maestro runs all failed, and not because of the app: a Maestro driver left running on another simulator still held its port, so Maestro read that other app's screen and reported that "Profile" did not exist. And the fastest driver is also the one with the most information, because every Flash reply says what was hit and, for navigation, where the app actually landed.

The same path on one of our production apps, which I cannot show, gave the same order: 0.9 s through Flash, 5.4 s through XCUITest, 7.7 s through Maestro. These numbers come from one Mac and one simulator. They are a measurement, not a benchmark suite. The ratio is what I would expect to travel; the absolute values will not.

What it costs

• It is Debug-only, by construction. Flash tests the Debug build. Release-only bugs, such as an optimizer or a stripped symbol, need the real runners.
• It still reads private state in one place: the gesture fallback reads a recognizer's target through its instance variable. It is fenced, it fails loudly, and it can break on an iOS update. The public path, accessibilityActivate(), carries most taps, and nothing in Flash synthesizes a touch anymore.
• It needs identifiers. A control with no accessibility identifier is reachable only by text or by position, which brings back the fragility Flash was built to avoid.
• It changes the app it tests. Turning animations off makes runs fast and deterministic, and hides animation bugs. Those still need a run with animations on.

Limits, and what comes next

Typing by identifier fails on SwiftUI Form fields: the identifier sits on an accessibility element, not on the text field behind it, so Pulse's login fields answer 404 and typing falls back to whichever field has focus. A search box with one field works; a login form with two does not, yet. That is the next change.

A long press and a double tap went away with the synthetic touches, and have no replacement yet.

The oracles find what they were written to find. An empty flash watch on a screen that is wrong in a way no oracle checks reads exactly like a clean one. The fix is not more oracles; it is making every agent say "Flash found nothing", never "the screen is correct".

And the whole thing is simulator-only. The server listens on the device's localhost, which is exactly why it is safe, and exactly why it cannot drive a phone on a desk. A system sheet is out of reach for the same reason: a web login or a permission alert runs in another process, and that part of a test still goes through a runner.

The video and the numbers come from Pulse, a demo app with made-up stories built for this post; the Step example in Stage 3 uses generic stand-ins for internal names. The idea is not internal. If your agents drive a UI through a runner built for humans, most of their time goes to the runner. Put the driver where the app's own state is, ask it by name, and make it say what it did.