English · 日本語
iOS on a real device and in a device cloud¶
A device cloud runs iOS on real hardware — there is no Simulator on the far side. Bajutsu's iOS backend targets the Simulator, so it cannot reach that hardware, and reaching a device cloud takes real new code rather than a configuration switch. This page explains why the Simulator-bound backend falls short, the one change that fixes it — making the XCUITest backend drive a real device — and the two routes that change opens: a batch route through AWS Device Farm and a live route through a reserved device behind an Appium endpoint. The same real-device work also lets Bajutsu drive a locally attached iPhone or iPad, which was effectively Simulator-only before.
Why the Simulator backend does not reach a real device¶
The gap is structural, not a missing option. The iOS backend, driving the Simulator, depends on something a real-device cloud does not provide.
simctldrives the Simulator only. It is the command-line control surface for Apple's iOS Simulator, so it has nothing to target on a cloud that runs physical devices — there is no simulator there to command. The XCUITest backend leans onsimctlfor the Simulator bring-up (erase, install, permission grants) around each run, none of which a physical device offers.
What the clouds do speak on iOS is Apple's own XCTest, and on AWS Device Farm also Appium's
XCUITest driver. Both build on the same XCUITest machinery that Bajutsu's
XCUITest backend (BE-0019)
already drives through xcodebuild. The path forward is therefore to generalize that backend to a
real device, not to write a new iOS backend from scratch.
The reusable core: XCUITest real-device targeting¶
The XCUITest backend targets the Simulator by default. A single config key on the target selects a real device instead:
targets:
my-app:
platform: ios
subject: { id: com.example.myapp }
runsOn:
ios:
kind: device # "simulator" (the default) or "device"
With runsOn.ios.kind: device, the backend generalizes its xcodebuild -destination from a named
Simulator to platform=iOS, so the same xcodebuild test-without-building driving layer runs
against a real device. The device the destination resolves to comes from the run's udid — a locally
attached device's udid, or the udid a device cloud hands the run.
A real device also skips the Simulator bring-up that simctl performs, which drops three
preconditions the Simulator path takes for granted: erasing the device, installing the app from a
local subject.path, and granting permissions up front. A scenario that needs one of those on a real
device fails loudly rather than silently doing nothing, and the Device Farm route below installs the
app through the cloud instead. This same key drives a locally attached iPhone or iPad, so the
real-device work is useful on its own, before any cloud is involved.
Reaching a real device's channels¶
On the Simulator, the host and the app share one loopback address, 127.0.0.1. Every channel
between Bajutsu and an iOS app relies on that. A real device has a loopback of its own. With
runsOn.ios.kind: device, each channel crosses the host–device boundary another way. The table lists
how.
| Channel | Who listens | Direction | On a real device |
|---|---|---|---|
| XCUITest runner | the runner, on the device | host → device | usbmuxd bridge |
nativeZ responder |
the app, on the device | host → device | usbmuxd bridge |
| WebView bridge | the app, on the device | host → device | usbmuxd bridge |
| Network collector | Bajutsu, on the host | device → host | an exchanged host address |
Host to device: the usbmuxd bridge¶
usbmuxd is the macOS service that Xcode and iproxy use to reach a device. Its Connect request
joins a socket to a port on the device's loopback. Bajutsu speaks the usbmuxd protocol itself, over
/var/run/usbmuxd. The host needs no extra tool. Bajutsu listens on the host's 127.0.0.1 for each
channel. It tunnels every connection to the device port. The Android resident channel does the same
with adb forward.
The bridge prefers a device attached over Universal Serial Bus (USB). It falls back to a network
attachment when usbmuxd lists nothing else. Suppose usbmuxd does not list the device at all. An
unplugged or untrusted device is the usual cause. The runner's spawn then fails at once and names
that reason, instead of timing out. The nativeZ field is diagnostic. A nativeZ bridge that cannot
open leaves the field absent and lets the run continue.
The listeners on the device stay on its loopback. The runner has no authentication, so it must stay out of reach of the network.
Device to host: an exchanged host address¶
The app reports network exchanges to a collector on the host. usbmuxd carries no connection that the
device opens. This channel needs a host address the device can route to. On a real device, the
collector binds every interface of the host. The app receives one collector URL per candidate host
address. The app sends each candidate a signed GET /ping. It keeps the first one, in the host's
order, that answers with the collector's signature. The search runs in the background, so the launch never waits for it. It
retries for up to two minutes. Up to 1,000 reports the app makes meanwhile wait. They go out in
order once a collector answers. The app writes any report it drops to the device log. Every
request must prove the per-run token, which this run's app alone holds. The token itself never
crosses the network, as
how the device-to-host route is authenticated
describes.
Bajutsu resolves the candidates at each run. A host whose address changes keeps working that way. The candidates come from the first of these sources that has a value:
- The
BAJUTSU_HOST_ADDRESSenvironment variable. - The target's
driver.runner.hostAddress(configuration). - Every routable IPv4 and IPv6 address of the host's active interfaces. The list includes the IPv6 address of the CoreDevice tunnel. Xcode itself uses that tunnel for a real-device test.
Suppose a run records network exchanges and no candidate exists. The run then fails before any device work, and the error names both settings. A device that reaches none of the candidates records no exchanges. A network assertion then fails on the empty record.
Two properties of this route need care on the device side:
- From iOS 14, an app asks for Local Network permission before its first local-network
connection. The search makes such a connection, so a fresh install raises the prompt during
launch. Every Device Farm job is a fresh install. Declare a
systemAlert: localNetworkentry withchoice: grantin the target'shooks.interruptsor the scenario'sinterrupts(scenarios). The guard then answers the prompt, including one already up when the scenario starts. Until the grant, the search gets no answer and the reports wait. We measured this on an iPhone with iOS 27.0.1. iOS showed the prompt even with noNSLocalNetworkUsageDescriptionentry. - The probe and every report travel as plain HTTP over the chosen route. The token stays off the
wire. Each report's captured bodies still travel in the clear. On a shared network, pin
BAJUTSU_HOST_ADDRESSto the CoreDevice tunnel's address.
How the device-to-host route is authenticated¶
The device reaches the collector over a network that Bajutsu does not control. On Amazon Web Services (AWS) Device Farm, the network belongs to the farm. A reader of a request carrying the per-run token could copy the token. With the token, that reader could write fabricated exchanges into the run's evidence. The app and the collector prove the token to each other instead. Each side computes a Keyed-Hash Message Authentication Code (HMAC) with the token as its key:
- Requests are signed. Each request carries
Authorization: Bajutsu-HMAC-SHA256 nonce=<nonce>, signature=<signature>. The signature covers the method, the path, a fresh random nonce, and a digest of the body. The digest uses the Secure Hash Algorithm (SHA) with 256 bits. - A replay is refused. The collector keeps every nonce it has accepted for the whole run. A
copied request gets a 409 and adds nothing to
network.json. - Answers are signed. The collector signs each answer against the request's nonce, in the
X-Bajutsu-Signatureheader. The app applies a/commandsanswer once its signature verifies. A probe answer without a valid signature never wins the search. - The bearer header is refused. Beyond the loopback, the collector answers 401 to a bearer header. The run log warns once when such a header carried this run's token. That warning means the app's BajutsuKit predates signed requests and needs a rebuild.
Bajutsu announces the signed method in the launch environment. It sets BAJUTSU_COLLECTOR_AUTH=hmac
beside BAJUTSU_COLLECTOR_TOKEN. An app built against an older BajutsuKit ignores the announcement.
A newer BajutsuKit driven by an older Bajutsu sees no announcement. That BajutsuKit then sends the
bearer header the older collector expects.
The signed method keeps the token secret. It also proves that each request and answer is genuine and fresh. Two properties stay out of its reach:
- Readable bodies. A party on the route can read the captured bodies in each report.
- No availability guarantee. A party on the route can drop or delay a request. The step that waits on that request then fails on its existing timeout. No forged exchange or command reaches the verdict.
Checking the channels before a run¶
Run bajutsu doctor --udid <udid> --environment-only for a runsOn.ios.kind: device target. It reports
both routes. A real device needs no booted Simulator. In that check's place,
doctor checks whether usbmuxd lists the device, and over which attachment. A device usbmuxd does not list
fails that check, and doctor exits non-zero. Doctor also lists the host addresses that the app would
receive. That list informs alone, since a scenario recording network exchanges is the one that needs it. A
device-cloud job can run doctor before bajutsu run to see which route its host lacks. The Device
Farm route depends on the same two routes. Nobody has verified them on Device Farm yet.
The signed device runner¶
A real device installs an XCUITest runner signed by a team the device trusts. Bajutsu cannot ship
such a runner prebuilt, unlike the Simulator runner. Each user builds the device runner with
bajutsu runner build --device. One build serves until an input listed under Building changes. The build signs with that user's own Apple Developer account. A target with
runsOn.ios.kind: device and no driver.runner.testRunner then resolves to that build. The target config
stays free of anything specific to one user.
The signing file¶
Signing belongs to the user, not to a target. The runner stays generic, and the target config travels through the repository. A team ID written in the config would reach every other user. The settings live in a per-user file outside the repository instead. Bajutsu looks for the file in this order, and the first hit wins:
--signing <path>onbajutsu runner build(the build command alone has this flag).- The
BAJUTSU_SIGNING_FILEenvironment variable. $XDG_CONFIG_HOME/bajutsu/signing.yaml, falling back to~/.config/bajutsu/signing.yaml.
A run consults entries 2 and 3 alone. A user who builds with --signing must point
BAJUTSU_SIGNING_FILE at the same file when running. The alternative is to copy the products out
with --out and name them in driver.runner.testRunner.
bundleIdPrefix: com.acme # host app = com.acme.bajutsu.runner-host,
# UI-test bundle = com.acme.bajutsu.runner-uitests
# bundleIds: # instead of bundleIdPrefix: name both identifiers yourself
# host: com.acme.e2e.runner-host
# uitests: com.acme.e2e.runner-tests
teamId: ABCDE12345 # required
signing: automatic # automatic (the default) or manual
# manual: # required when signing: manual (ignored under automatic)
# identity: "Apple Development: Jane Doe (ABCDE12345)"
# profile: "Acme Bajutsu Wildcard" # one profile covering every identifier, or
# profiles: # one profile per signed product
# host: "Acme Bajutsu Host"
# runner: "Acme Bajutsu Runner" # the UI-test bundle's .xctrunner app
The file holds no secret. A team ID, a certificate name, and a profile name merely point at keys
and profiles. Those stay in the Keychain and in the profile directories: Xcode 16 and later keep
profiles in ~/Library/Developer/Xcode/UserData/Provisioning Profiles/, and earlier releases use
~/Library/MobileDevice/Provisioning Profiles/. Bajutsu searches both.
| Key | Meaning |
|---|---|
bundleIdPrefix |
Derives both runner identifiers. Give this key or bundleIds, never both. |
bundleIds.host, bundleIds.uitests |
Name the two identifiers outright, for an account that already holds App IDs and profiles under other names. The two values must differ. |
teamId |
The Apple Developer team that signs both products. |
signing |
automatic lets Xcode mint the profiles. manual pins the certificate and profiles below. |
manual.identity |
The codesigning identity, by name or by its certificate hash. |
manual.profile / manual.profiles |
One profile for every product, or one each for the runner's main app (profiles.host) and the .xctrunner app (profiles.runner). Give exactly one form. When two unexpired profiles share a name, give the profile by the identifier in its UUID field instead. A profile Xcode manages, such as iOS Team Provisioning Profile: *, does not qualify: Xcode refuses it under manual signing, so the check before the build rejects it. Create a development profile in the Apple Developer portal, or use signing: automatic. |
The UI-test bundle's .xctrunner app takes the identifier <uitests identifier>.xctrunner. A
manual build with one profile per product needs a profile for that identifier too.
Keep the runner identifiers distinct from the app under test. A runner that shares the app's identifier overwrites the app when it installs. Bajutsu cannot check for that clash, because the build knows no target.
Building¶
bajutsu runner build --device [--signing PATH] [--out DIR] [--force]
The command works in three steps:
- Check the prerequisites: macOS,
xcodebuild, andxcodegen. Manual signing adds the identity and every named profile. Each gap appears with its fix. - Copy the runner sources into a scratch directory, and rewrite that copy for the signing file. The checkout stays untouched. In a wheel install, the sources come from a copy the wheel ships.
- Build the scratch copy with
xcodebuild build-for-testing, and print the path of the builtBajutsuRunner.xctestrun.
The products land in Bajutsu's cache under xcuitest-runner-device/<key>/Products/. The key covers
every input that changes the signed products:
- the runner sources;
- the signing fields, with both resolved identifiers;
- the Xcode build version;
- for manual signing, the certificate hash and a digest of each profile file.
A certificate or profile renewed under the same name yields a new key. A repeat build
with unchanged inputs reuses the cached products, and --force rebuilds anyway. --out DIR
copies the Products directory to DIR as well. Device Farm packaging needs that copy: the
.xctestrun expects its test bundles beside itself.
What a run does¶
A runsOn.ios.kind: device run with no driver.runner.testRunner reads the signing file. It computes
the same key. The run then uses the cached .xctestrun. An explicit testRunner still wins over the cache.
A run never builds the runner itself. A signing build is slow, can raise a Keychain prompt, and can register identifiers with Apple. Each miss fails at once and names the fix:
- No signing file: the error names both lookup locations.
- No build for the current inputs: the error names
bajutsu runner build --device. - An expired profile: the error names
bajutsu runner build --device --force.
bajutsu doctor reports which signing file a device target would use. The report builds nothing and
computes no key.
Two routes to a device cloud¶
Both routes share the real-device XCUITest core above; they differ in how a device is reserved and how the run reaches it.
Batch — AWS Device Farm¶
AWS Device Farm is a batch service: it runs your commands on its host, which already has the
reserved device attached, rather than lending you a device to drive over the network. Bajutsu reaches
it through a CI-side submitter that packages the app plus your scenarios, uploads them with a
test-spec that runs bajutsu run --backend xcuitest against the reserved device's udid, and collects
the artifacts. The verdict still comes from Bajutsu's own machine-checkable assertions, never from
Device Farm's own pass/fail classification.
The submitter, the test-spec it renders, the re-signing caveats, and the manual proof-of-concept
procedure are all documented on the AWS Device Farm page — the iOS run reuses that
same batch machinery, selecting the iOS app upload and the xcuitest backend by platform.
Live — an Appium endpoint provider¶
Where a cloud reserves a single iOS device and exposes it behind an Appium / WebDriver endpoint —
a self-hosted grid, for example — Bajutsu models that reservation as a device provider on the live
seam (BE-0236).
Which grid serves a run is infrastructure, not a property of the target, so the endpoint lives in an
appium environment of a worker capability file (worker.yaml) rather than in the target config
(BE-0452):
version: 1
environment: appium
appium:
endpoint: https://grid.example.com/wd/hub # the reserved device's Appium / WebDriver address
platform: ios # the one value today
drivers: [xcuitest, adb, playwright, fake]
Pass the file with --worker-config to run / record / crawl / repl / audit / doctor /
bajutsu mcp. Each target of the grid's platform then takes the endpoint as its device, in place of
a local udid. The scenario's other targets stay on this host. Bajutsu refuses a scenario with two
grid targets. --udid no longer accepts a URL; name the grid in this file instead.
Bajutsu never boots the reserved device or installs onto it through simctl. The grid owns the
reservation. The device arrives with its build in place, and nothing needs releasing.
Bajutsu drives that endpoint over a live W3C WebDriver transport. The run routes it
around the simctl / xcodebuild udid machinery entirely — that machinery structurally cannot
carry a URL. A tap-and-assert flow works end to end. The semantic actions the local runner drives
natively map onto Appium's XCUITest mobile: commands over the endpoint: tap, query, screenshots,
condition waits, the input steps (type / delete, swipe / scroll), and the two-finger
pinch / rotate gestures. As on the local backends, an ambiguous selector still fails before any
actuation — the selector resolves in Bajutsu, not on the grid.
What the WebDriver transport cannot drive is narrowed away up front, the same way the real-device
path narrows the simctl-backed families:
- Native text selection (
select/copy) has no first-class Appium XCUITest command — the local runner does select-all and clipboard copy natively, but the endpoint exposes no faithful counterpart.pasteis refused too: the local driver pastes through the edit menu, and that composition is not wired onto the live route. simctldevice control and permission grants do not apply on the live route either, exactly as on any other real device (see the caveats below).
So on the live route the preflight (BE-0082)
advertises only what the transport actually drives and skips a scenario that needs one of the
narrowed capabilities — with a clear reason, before any device work — rather than failing late with
UnsupportedAction mid-run.
A worked example config lives at
demos/showcase/live/showcase.live.config.yaml:
it mirrors the local showcase-swiftui target but carries no local subject.path or
driver.runner.testRunner, since the reserved device already holds the build and the live route
speaks WebDriver rather than the runner channel. The
demos/showcase/live/worker.yaml file beside it names the grid. Point that file's
endpoint at your own grid, then run the shared showcase suite against it:
bajutsu run --target showcase-swiftui-live --config demos/showcase/live/showcase.live.config.yaml \
--worker-config demos/showcase/live/worker.yaml
Real-device caveats: re-signing and capability degradation¶
Running on a real device rather than the Simulator changes two things Bajutsu accounts for up front.
Both are properties of physical hardware, not of any one cloud, so they hold for every real device
the XCUITest backend drives — whether reached by runsOn.ios.kind: device (Device Farm or a
locally attached device) or over the live WebDriver route above.
- Re-signing strips entitlements. A device cloud re-signs the uploaded app with its own provisioning profile so it installs on the reserved device, and the re-sign drops the entitlements the new profile does not carry — commonly Push and App Groups. An app feature that depends on a dropped entitlement behaves as the re-signed build does, so a scenario asserting on such a feature should expect the re-signed behavior rather than the App Store one.
simctldevice control and permission grants do not apply. Bajutsu's iOS device control (setLocation, the clipboard steps,push,clearKeychain,background/foreground, and the status-bar overrides) and its permission grants are all backed bysimctl, which reaches only the Simulator. On a real device the XCUITest backend advertises neither, so a scenario that uses one is skipped by the preflight (BE-0082) with a clear reason, before any device work, rather than failing late with asimctlerror mid-run. The on-device capabilities the XCTest runner drives itself — query, elements, screenshots, taps, and two-finger gestures — are unaffected.
The AWS Device Farm page covers the same two caveats in the batch context, including the specific entitlement keys Device Farm's re-sign drops.
References¶
- AWS Device Farm — the batch route: the submitter, the test-spec, and the manual proof of concept.
- Drivers — the
Driverinterface and the backends behind it, including XCUITest. - Configuration — the
runsOn.ios.kindtarget key. - Self-hosting — the
appiumenvironment of a worker capability file. - Command reference —
bajutsu runner build --device. - BE-0019 — XCUITest backend
- BE-0236 — device-cloud provider abstraction
- BE-0238 — iOS device-cloud execution
- BE-0456 — per-user signed device build of the XCUITest runner