English · 日本語
Scenario specification (authoring reference)¶
A scenario is Bajutsu's only persisted artifact: plain YAML, version-controlled in git and reviewable in a PR. record (AI) writes it the first time; humans own and edit it afterward. run executes this structure without AI.
Implementation: bajutsu/scenario/ (pydantic models under models/, extra="forbid" rejects unknown keys).
The normative grammar — every production, type, default, and validation rule — is in dsl-grammar. This page is the authoring guide: how to write a scenario, by example.
Related: cookbook (worked examples) · dsl-grammar (formal grammar) · selectors (how selectors and assertions evaluate) · evidence · run-loop (execution)
File shape¶
One file = a list of scenarios, or a { description, scenarios } mapping when you want a
file-level description. load_scenarios() accepts either form; a top level that is neither is
rejected.
- name: ... # scenario 1
steps: [...]
- name: ... # scenario 2
steps: [...]
With a file-level description (and an optional per-scenario description):
description: What this file covers.
scenarios:
- name: ...
description: What this scenario checks.
steps: [...]
Both the file description and each scenario's description appear in report.html (the
summary header and each scenario card) and in the bajutsu serve UI.
Schema version¶
The mapping form may carry a top-level schema integer marking the scenario schema version. A file
that omits it is treated as version 1, so every existing scenario is valid unchanged:
schema: 1
scenarios:
- name: ...
steps: [...]
When a scenario declares a schema newer than the running bajutsu understands, the load fails
with a clear upgrade-path message instead of an opaque "unknown field" error — the case that arises
once a scenario tree is read across versions (for example, a config sourced from a pinned Git ref).
The current version is SCHEMA_VERSION in bajutsu/scenario/models/scenario.py. Bump it only for a
load-breaking change — removing a required field's meaning, or a change an older bajutsu would
misinterpret rather than merely reject; a purely additive optional field needs no bump.
Top-level structure (Scenario)¶
| Key | Type | Default | Description |
|---|---|---|---|
name |
str | required | Scenario name (used for the report / JUnit testcase / codegen method name) |
description |
str | none | Optional human description; shown on the scenario's report card and in the serve UI |
from |
str | none | Provenance — the natural-language goal record authored this scenario from (provenance). Authoring metadata only; run ignores it |
tags |
list[str] | [] |
Selection labels; the CLI --tag / --exclude flags pick which scenarios run (reuse, data, and tags) |
targets |
list[str | list[str]] | [] |
Every target this scenario drives (below). Each entry names a targets.<name> config unit, resolved from the config the run loads, or is an array of two or more names that share one device (device groups) |
primaryTarget |
str | none | The target a step or top-level expect entry runs against when it omits target; must be the first name in targets, the first member of the first group (below) |
installs |
list[str] | [] |
The device-group members that install and launch at the start beside the primary; every other member of a group of two or more installs later through an installApp step (device groups) |
data / dataFile |
list / str | none | Data-driven rows — inline data, or dataFile (a CSV path). Expands into one run per row, substituting ${row.col}. Mutually exclusive (reuse, data, and tags) |
preconditions |
object | {} |
Per-test environment setup: the target's subject group merged over the target's, the device state under device, and inheritSetup (below) |
setup |
list | [] |
Setup steps run as their own phase ahead of steps; a failure there aborts the scenario (below) |
steps |
list | required | The ordered actions (below) |
expect |
list | [] |
Final assertions after all steps pass (selectors) |
cleanup |
list | [] |
Teardown rules — each { on: always \| success \| failure, steps }, run once the verdict exists, on every path out of steps (below) |
capturePolicy |
list | [] |
Repeatedly-firing evidence rules (evidence) |
network |
object | none | { filter: { domains: [...] } } — filter.domains scopes which observed requests are interleaved into the report's Steps timeline (by URL host; a parent domain matches subdomains). Unset shows all; the Network tab always lists them all (reporting) |
mocks |
list | [] |
Deterministic network stubs — a matching outgoing request gets a canned response instead of hitting the network (network mocks) |
redact |
object | none | Masking applied before evidence is written (evidence) |
permissions |
dict | {} |
Declarative OS permission state — { <service>: grant \| revoke } — applied before the app launches (below) |
interrupts |
list | [] |
What the run clears on its own when it surfaces at an unpredictable point, layered over the target's hooks.interrupts. Each entry takes one of three forms, decided by the key it writes: a screen ({ condition, steps }, below), an operating-system prompt ({ systemAlert, choice }, below), or Apple's TipKit popover ({ tipKit }, below) |
- name: filter narrows the catalog
preconditions:
subject:
launch:
env: { SHOWCASE_UITEST: "1" }
steps:
- tap: { label: "Search", traits: [button] }
- wait: { for: { id: search.field }, timeout: 10 }
- type: { text: "Horse 3", into: { id: search.field } }
- wait: { for: { id: search.row.3 }, timeout: 5 }
expect:
- count: { sel: { idMatches: "search.row.*" }, equals: 1 }
- value: { sel: { id: search.count }, equals: "1" }
(real file: demos/showcase/scenarios/search.yaml)
preconditions (environment setup)¶
Implementation: Preconditions in scenario/models/scenario/preconditions.py. The module
bajutsu/common/config/preconditions.py merges it over each target. The runner's launch_driver
reads the merged result to build the launch sequence (run-loop).
A scenario's preconditions holds two groups and one flag. subject uses the target's own group
name. A field thus has one spelling in the config and in the scenario
(BE-0452).
device holds the device state the scenario applies, keyed by destination. Today device.ios is
the one block
(BE-0469).
inheritSetup decides whether the targets' hooks.setup runs before this scenario.
A scenario field overrides the same field of the target (configuration). Two
fields combine with the target's instead:
subject.launch.envmerges key by key.subject.launch.argsappends after the target's arguments.
A system alert or a TipKit tip, which the run answers on its own, is not a precondition. A
scenario declares one as an entry of its own top-level interrupts. Those entries layer over the
target's hooks.interrupts (below).
preconditions:
subject:
reinstall: erase
launch: { env: { FEATURE_X: "1" }, args: ["-debug"], deeplink: "showcase://cart" }
device:
ios: { locale: ja_JP, seedPhotos: [photos/cat.jpg] }
inheritSetup: false
| Key | Type | Default | Description |
|---|---|---|---|
subject.reinstall |
overwrite | clean | erase |
unset (the target's subject.reinstall, then clean) |
How much state is reset before each run, weakest first: overwrite installs over the existing app (keeps its data); clean uninstalls and then installs (fresh app and data); erase wipes the whole device (simctl erase: apps, data, settings), then installs. The install needs the target config's subject.path. Android has no device-wide wipe, so erase there acts as clean. iOS and Android; a web target accepts erase alone, which its fresh browser context already satisfies (BE-0473) |
subject.launch.env |
dict | {} |
Launch environment, merged onto the target's subject.launch.env key by key. iOS injects it through SIMCTL_CHILD_*, Android as intent extras; on the web the Playwright code generator reads it |
subject.launch.args |
list[str] | [] |
Launch arguments, appended after the target's subject.launch.args. iOS only |
subject.launch.deeplink |
str | unset (the target's subject.launch.deeplink) |
A URL the app opens after launch (simctl openurl on iOS, an intent on Android). On Android, a deep link that no activity of the app resolves fails the launch with am start's own reason (BE-0487) |
device.ios.locale |
str | unset (the target's runsOn.ios.locale) |
Force the locale and language at launch (-AppleLocale / -AppleLanguages), and pin the Simulator's system language. A value that differs from the target's forces a cold respawn |
device.ios.seedPhotos |
list[str] | unset (the target's runsOn.ios.seedPhotos) |
Images seeded into the Simulator's photo library on the erase path, in order; paths resolve against the scenario file, and a scenario's list replaces the target's |
device.android.seedPhotos |
list[str] | unset (the target's runsOn.android.seedPhotos) |
Images seeded into a directory Bajutsu owns in the device's shared storage, in order, on every reinstall level; paths resolve against the scenario file, and a scenario's list replaces the target's |
inheritSetup |
bool | true |
false skips the hooks.setup of every target this scenario declares; their hooks.cleanup still runs (below) |
A scenario cannot set id, path, or build. Nor can it set startWhen or idNamespaces. The
target owns these fields, which identify or build the app. evidence.secrets stays a target-level union,
so a scenario cannot set it either.
Each field is checked against the targets the scenario drives, before any device work. A
subjectfield that no driven target's platform has fails the run and names the field, rather than being ignored:subject.launch.argsin a web-only scenario, for one. AtipKitentry in the scenario'sinterruptsfails the same way when no driven target is iOS. Each target applies thesubjectfields its own platform has, so an iOS-and-web scenario can still setsubject.launch.argsfor its iOS side. A scenario file that several single-platform targets pick up throughpaths.scenariostherefore names itstargets, or sits in a directory only the matching targets read. Thedeviceblock is keyed by destination. Adevice.iosblock in a scenario that drives no iOS target has no effect and is not an error.
A scenario declares no device condition. Where a run happens belongs to the target. A condition
such as a model or an OS range thus lives in the target's
runsOn. A scenario for a screen that iPad alone has
names a target that runs on an iPad:
# bajutsu.config.yaml
targets:
app-ipad:
platform: ios
subject: { id: com.example.app }
runsOn:
ios: { model: "iPad Pro 13-inch (M4)", os: ">=18" }
# the scenario
targets: [app-ipad]
A list or a range on that target multiplies each of its scenarios into runs, as configuration describes.
Earlier releases accepted
preconditions.runsOn. The loader now rejects it. Apreconditions.runsOn.ios.localeorseedPhotosfails and namespreconditions.device.ios.<field>. A condition such aspreconditions.runsOn.ios.modelfails and names the target path that declares it instead,runsOn.ios.model.
subject.reinstallresolution order is this scenario's ownsubject.reinstall> the target'ssubject.reinstall> built-inclean, and the CLI flag then moves the result:--erasemakes iterase, and--no-eraselowers anerasetocleanwhile leavingoverwriteandcleanalone. An unset scenario value (the common case) inherits whatever the target config sets._filter_scenarios(run/cli.py) applies the flag before the run starts. A target that stateserasetherefore uninstalls under--no-erase; a scenario that must keep the app's data writesoverwrite. The removedpreconditions.runfails at load, and the error names where each of its fields went (below). See BE-0473 and configuration.Seeding iOS photos needs a wiped device. After merging, a non-empty iOS
seedPhotosrequires the effectivesubject.reinstall(the scenario's, else the target's, then moved by the flag) to beerase, because the photo library is seeded only on the erase path. A target-levelsubject.reinstall: erasesatisfies the rule. In a scenario that drives several targets, the primary alone seeds photos. A non-primary target with a mergedseedPhotosfails the run.record,crawl, andreplrefuse--no-eraseon a target that seeds photos. Both refusals exit 2 before any device work. Android needs noerase: each scenario empties the directory it seeds, so the device shows exactly the listed images there, whatever the reinstall level. A scenario with an empty list still empties a directory an earlier seed created. The primary-only rule holds on Android too.
reinstall: cleanalso resets permission state (permissions, below): iOS viasimctl privacy reset all, Android viapm clear.overwriteresets neither.overwriteexists to keep a scenario's data across runs, and permission grants are part of that same preserved state, so a scenario relying on an earlier one's grant underoverwritekeeps seeing it.A run stored before BE-0452, BE-0469, or BE-0473 still loads. The scenario loader rejects the old flat keys:
erase,reinstall,launchEnv,launchArgs,deeplink,locale, andseedPhotos. It rejectspreconditions.runsOn,preconditions.run, and the top-levelsystemAlertHandlingandiosTipKitHandling. Each message names the key's new home. The offline readers of a run directory translate the old keys in its storedscenario.yamlinstead. They are the report re-render,trace, andtriage. A storedpreconditions.runsOnkeeps its iOSlocaleandseedPhotosunderdevice.iosand drops its conditions, which already chose the devices that run used. A storedpreconditions.eraseorpreconditions.run.erase: truebecomessubject.reinstall: erase, and a storedfalseis dropped. A storedpreconditions.run.systemAlertHandlingbecomes onesystemAlertentry per rule in the scenario'sinterrupts, a storedtipKitHandlingbecomes atipKitentry, and a storedinheritSetupmoves topreconditions.inheritSetup.
Not every environment can apply every precondition. A real iPhone cannot run simctl erase, and a
browser has no app launch to pass subject.launch.env to. When the target's environment cannot apply a
precondition, the scenario fails before any device work. The target can declare the precondition
ignored instead (configuration).
bajutsu lint --config <file> --target <name> reports the same finding with no device
(cli).
tipKit (a blocking TipKit tip)¶
Write - tipKit: dismiss in the scenario's interrupts to turn the TipKit guard on. The guard dismisses a
blocking Apple TipKit tip, so no scenario has to hand-author that recovery. A tip is the
framework-owned popover. The guard requires a tip's dismiss scrim (PopoverDismissRegion) and
its container together. A plain confirmationDialog installs the same scrim, so the scrim alone
cannot identify a tip. The container, TipView, belongs to TipKit alone. We measured it on both the
SwiftUI and UIKit presentations. An interrupts entry for a tip keys on TipView for the same
reason. The guard applies to iOS alone and stays off by default. A scenario sometimes asserts on
the tip itself, and the guard would dismiss it.
- name: open the catalog past a blocking tip
interrupts:
- tipKit: dismiss
steps:
- tap: { id: catalog.open }
A target turns the guard on for each of its scenarios with the same entry in hooks.interrupts.
The scenario's entry, when it writes one, wins over the target's. A scenario that asserts on a tip
writes - tipKit: leave to turn off its target's dismiss. Absent from both, the guard stays off.
The command-line flag --ios-tipkit-handling / --no-ios-tipkit-handling sets every scenario's
entry to dismiss or leave.
systemAlert (the system-alert guard)¶
A systemAlert entry in an interrupts list names an operating-system prompt and the run's answer.
The scenario's own top-level interrupts takes the entry. The target's hooks.interrupts takes it
too (configuration).
The iOS backend cannot see or tap SpringBoard-level prompts (a notification or App Tracking Transparency request, "Allow Paste"). These prompts cover the app and collapse its element tree, silently blocking a step. The alert guard clears them reactively.
On the iOS XCUITest backend, the guard takes a deterministic native path (BE-0315). It reuses BE-0316's SpringBoard query to read which buttons the alert offers, then taps the one a systemAlert entry names. The path takes no screenshot and no model round trip, so the guard clears the common prompts in well under a tenth of a second and runs without ANTHROPIC_API_KEY.
Where nothing deterministic can act, the guard does nothing (BE-0402). Three cases fall there: a backend without the capability, an alert that no entry identifies, and a non-SpringBoard surface that the query cannot enumerate and no entry identifies either. (The in-tree path below clears the surfaces an entry does identify.)
The blocked step or wait runs on to its own timeout, exactly as it would with no guard configured, and that timeout names what the guard saw: wait timeout: for {'id': 'submit'} (10.0s) — an unhandled system alert is blocking the screen (buttons: Allow, Don't Allow). Where nothing enumerated the screen, the message is the hedged … — the screen appears blocked, possibly by a system alert or another overlay outside the app's view.
Before BE-0402, all three cases fell back to an AI-vision guard reading a screenshot. run no longer does, so no flag of run's reaches a model at all. (record and crawl keep that guard for authoring — details.)
For a wait step (for/gone/settled/screenChanged), the guard acts mid-wait. On XCUITest, each of the wait's own tree reads answers a prompt an entry names before it returns the tree, checking SpringBoard at most once per pollInterval (default one second) (BE-0461). The wait recovers before its own timeout elapses rather than waiting for the step to fail first (BE-0269).
On the XCUITest backend, a frozen screen stops a for or screenChanged wait before its deadline. A screen counts as frozen when both conditions below hold. A fresh SpringBoard check finds no alert. Yet for 10 seconds, no element below the application has carried a label or an identifier. We measured such a screen on an iOS 26.5 Simulator. A notification request can land while iOS presents its Save Password alert. That alert then halts mid-presentation. It never draws and is in no process's accessibility tree, yet it stays modal over the app. Nothing the guard can reach clears it, so the wait fails at that point with a reason like this:
wait stopped: for {'id': 'Close'} after 10.4s — no element below the application has carried a label or identifier for 10s while no system alert is up — …
A scenario opts into this stop by declaring a systemAlert entry for an in-app prompt such as savePassword. Without one, a long empty tree may be the app's own unlabelled loading screen. The wait then keeps its own timeout. An app with a slower unlabelled loading screen raises the bound with frozenScreenTimeout on that entry. A value of 0 turns the stop off. A gone or settled wait never stops this way, because a frozen tree already satisfies it. A handleSystemAlert step does not either: it waits on SpringBoard, which can still raise its prompt.
The guard is on by default. On XCUITest it rides the reads a step already makes, and a step that passes takes no retry, so a passing scenario pays one SpringBoard check per pollInterval at most. The guard needs no ANTHROPIC_API_KEY and consults none. No scenario or target key turns the guard off; bajutsu run --no-system-alert-handling turns it off for the whole run. A scenario declares what the guard answers with systemAlert entries in its own top-level interrupts, which layer over the target's hooks.interrupts (below).
The systemAlert entries are the guard's whole declaration
(BE-0406,
BE-0473):
a scenario names the prompts it expects and the choice to make on each, and every answer path
matches against those alone. BE-0406 removed the ordered labels list BE-0401 had introduced. A
button label names a button and never the alert it sits on, and "Cancel" and "Close" are ordinary
application vocabulary, so a label licensed a tap on a screen no scenario had described.
record and crawl still keep a free-text vision instruction for authoring, but they read it only
from their own --alert-vision-instruction flag, never from a scenario or a config.
An entry takes systemAlert, the prompt, and choice, the answer. Two optional keys tune the
guard. Each row below is one entry of the scenario's top-level interrupts:
| Entry | Meaning |
|---|---|
(no systemAlert entry) |
the scenario answers no prompt of its own; the target's entries still apply, and an alert no entry names is reported on the blocked step's failure |
{ systemAlert: notifications, choice: grant } |
answer a named, covered prompt by its own choice, regardless of which label it shares with another prompt |
{ systemAlert: notifications, choice: leave } |
answer nothing for that prompt, even when the target's entry answers it |
{ systemAlert: notifications, choice: grant, pollInterval: 2 } |
also check SpringBoard at most every 2 s instead of the one-second default |
{ systemAlert: savePassword, choice: deny, frozenScreenTimeout: 30 } |
also tolerate a frozen screen for 30 s instead of the 10-second default before a guarded wait stops (0 turns the stop off); accepted only on a prompt raised into the app's own tree, such as savePassword |
- name: grant notification permission
interrupts:
- systemAlert: notifications
choice: grant # accept the prompt instead of leaving it alone
steps:
- tap: { id: sys.requestNotif }
- wait: { for: { id: sys.notif.authorized }, timeout: 4 } # the guard taps Allow, then this passes
choice: leave is how a scenario proves that its own handleSystemAlert step does the work: the
entry turns off the target's answer for that one prompt, while every other prompt the target answers
still applies to the scenario.
An entry for a prompt the tree can reach also arms a second, in-tree path on iOS, for a prompt that is
not a SpringBoard alert at all. The one such prompt covered today is savePassword, which iOS
raises in the app's own process: its buttons reach the element tree with a label and no identifier,
and the SpringBoard query never sees it, so only a tap in the tree can clear it. The runner makes that
tap inside a read, and only after a fresh SpringBoard check in the same read comes back empty — because XCUITest resolves
whatever out-of-process alert is interrupting before it synthesizes an element interaction, and the
app's tree cannot see that alert. So when both prompts are up, the SpringBoard alert is answered
first, and the app-attached alert is cleared from the tree afterwards.
Which button an interrupting alert receives is your policy's decision too. XCUITest resolves such an
alert before the interaction it interrupts, and left alone answers it with the alert's own default
button — granting a permission your entries may have refused, with nothing in the run's report. The
runner therefore installs an interruption monitor that presses the button your entries name, by the
same discipline the native path applies, and the dismissal is reported as an ordinary alert event. An
entry this monitor can never meet is dropped rather than pushed: savePassword never interrupts an
interaction from another process. An alert none of your entries identifies is still left to XCUITest's
own default handler — nothing here can change that tap — but the step or expect it interrupted
fails, naming the buttons it offered, rather than the run continuing as if nothing had answered on
your behalf.
(real file: demos/showcase/scenarios/save_password_browser.yaml)
An entry names a prompt, and the guard taps only once it has identified that prompt on screen: every one of the prompt's identifying labels must be present, each exactly once. An alert no entry identifies is left alone and reported on the blocked step's own failure, rather than answered by a guessed button.
The CLI flag --no-system-alert-handling turns the guard off for every scenario, and
--system-alert-handling keeps it on. --alert-poll-interval supplies pollInterval for a single
run, between the scenario and the target config (see layering below).
(real file: demos/showcase/scenarios/permission.yaml)
Preventing a prompt instead of answering it¶
A prompt the app never raises needs no answer. The supported way for an app to skip its own prompts
under test is a launch argument or environment variable in the target's config, targets.<name>
(subject.launch.args and subject.launch.env), which the app reads at start-up:
targets:
my-app:
platform: ios
subject:
id: com.example.myapp
launch: { env: { UITEST_SKIP_ONBOARDING_PROMPTS: "1" } } # the app checks this and asks nothing
Prevention makes the guard's work rarer, but it cannot replace the guard. Three prompts stay reactive:
- Notifications are not a privacy service
xcrun simctl privacycan grant (BE-0276). - App Tracking Transparency (ATT) has no
simctl privacyservice either: the tool's own service list names none. - Save Password depends on the Simulator's AutoFill Passwords setting. We found no supported way
to turn that setting off before a run, so the scenario answers the prompt with a
savePasswordentry (BE-0461).
Notification banners are swiped away, never tapped¶
A foreground notification banner interrupts a step the way a system alert does. The same
interruption monitor answers a banner. Your systemAlert entries never do
(BE-0416).
A banner is not an alert. It offers no button an entry could name. Its one tappable region carries the notification's own text. Pressing that region opens the notification's app. A run that pressed it would leave the scenario under test.
The runner swipes the banner upward instead, anchored to the banner's own measured frame. The runner reads that frame after the banner has come to rest. A banner still sliding in sits above the screen, so a swipe from its frame would travel downward. The runner then confirms the banner has gone before claiming the interruption. Once the banner leaves, the step's interaction lands on the element the scenario named.
A swipe can miss the banner. The runner then waits up to 10 seconds for the banner's own auto-dismissal. Handing the banner back to XCUITest is the last resort. On iOS 26.5, XCUITest's own handler ignores a banner. Each XCUITest lookup of the banner records an error, and those errors stop the runner.
A confirmed dismissal reaches the run's report on the step it interrupted, under its own kind. The text recorded beside that dismissal belongs to the notification, not a button. That kind keeps a reader from misreading the text as a button the guard chose.
A banner still on screen after that 10-second wait carries no such record. The runner logs the attempt to its own output and hands the banner back to XCUITest. A temporary banner leaves within the wait in the common case. A persistent-style banner never dismisses itself, so it can reach the hand-off. On iOS 26.5, that hand-off can still stop the runner.
Nothing about a banner is yours to declare. The banner path runs regardless of whether the system-alert guard is on. Before BE-0416 a governed run met the banner on the alert path instead. No entry matched, so the step failed. Its reason named the notification's body text among the buttons the run had expected.
That monitor answers merely a banner that is interrupting something. A plain query never invokes
it. A banner sitting on screen with nothing tapping through it therefore stays up. Left alone, it
corrupts after.png and every visual-regression comparison built from it. A second, proactive
sweep closes that gap: the runner looks for a banner and swipes it away before the shot starts. It
does this at three sites — once per step, right before that step's own screenshot; once more before
the expect-phase capture a visual assertion there reads; and once more before a step's own
assert captures its own visual entry (below). The per-step sweep rate-limits its query to
the system-alert guard's own poll interval, so a passing scenario pays it once per interval, not once
per step. The expect-phase and step-level visual captures each pay their own query
unconditionally instead, since either runs far less often than every step. Like the interruption
path above, all three sites run with no scenario or CLI toggle of their own, on the iOS XCUITest
backend alone.
Answering more than one prompt differently¶
One onboarding flow often meets several prompts and means a different answer on each. A
systemAlert entry pairs one covered prompt with one choice, reusing handleSystemAlert's own
prompt / choice vocabulary, so the file says which prompt it expects rather than which words it would accept seeing
tapped:
- name: onboarding — accept notifications, refuse tracking
interrupts:
- systemAlert: notifications
choice: grant
- systemAlert: tracking
choice: deny
steps:
- tap: { id: onboarding.start }
- wait: { for: { id: home.title }, timeout: 10 }
The guard identifies which alert is on screen from an entry's prompt — every label that identifies that prompt, resolved for the run's locale, must be present on the alert — not from the order entries appear in; two entries naming the same prompt in one list fail at load. A prompt that renders differently by context or by iOS version contributes one matchable shape per rendering, so a single entry still answers every shape its prompt has.
An alert no entry identifies — one outside the label table, or any alert at all on a backend with neither answer path — is left alone and named in the blocked step's own failure. That silence is the point: the guard answers the screens a scenario described, and nothing else.
Not every prompt reaches every path. The label table records, per prompt, which of the three surfaces can declare and answer it:
prompt |
handleSystemAlert step |
guard, native SpringBoard probe | guard, in-tree dismissal |
|---|---|---|---|
notifications |
✅ | ✅ | — |
tracking |
✅ | ✅ | — |
paste |
✅ | ✅ | — |
savePassword |
— | — | ✅ |
localNetwork |
✅ | ✅ | — |
SpringBoard owns every prompt but savePassword, so the step and the native probe both reach them and the tree never
sees them. savePassword is the mirror image: iOS raises it inside the application's own process, so
springboard.alerts never enumerates it and the in-tree dismissal alone can clear it. A
handleSystemAlert step naming savePassword is therefore rejected at parse time — that step reads
the SpringBoard query alone, so it could only poll an empty button list until its deadline
(BE-0406).
localNetwork is the iOS Local Network permission prompt. On a real device, Bajutsu's own network
collector raises it the first time the app reports to the host
(iOS on a real device). Its buttons
are the same as the notification prompt's, so the buttons alone cannot tell the two apart. The guard
reads the alert's title too. It reduces the title to Apple's template, with the app's name replaced
by %@. The localNetwork entry names that template, and the notifications entry excludes it. A
notifications entry therefore never answers the Local Network prompt, and the reverse holds too. A
real-device target that records network exchanges declares it like any other prompt:
interrupts:
- { systemAlert: localNetwork, choice: grant }
The app raises this prompt during launch, before the scenario's first step. The guard otherwise first looks inside a pending wait, so a first tap could land on the
prompt. To prevent that, the guard reads the SpringBoard query once before the first step and answers a declared
prompt already on screen.
That read is native alone. It never taps the app's own buttons. It leaves an undeclared alert
alone. A first handleSystemAlert step keeps its own prompt. The step may choose the opposite of an
entry's choice. A step names a prompt by prompt, so the title still tells a look-alike apart. A
Local Network prompt in front of a notifications step still goes to its entry. A step written with
sel names no prompt, so the guard skips the read for it.
A handleSystemAlert step checks the title the same way. A step naming notifications never taps
the Local Network prompt, even though both offer its button.
savePassword also renders three ways, where the other four prompts each render one:
| Rendering | English | Japanese |
|---|---|---|
| a web form | Save Password / Never for This Website / Not Now | パスワードを保存 / このWebサイトでは保存しない / 今はしない |
| the app's own fields, iOS 18.6 | Save Password / Not Now | パスワードを保存 / 今はしない |
| the app's own fields, iOS 26.5 | Save / Not Now | 保存 / 今はしない |
One entry still covers all three. choice: grant follows the accepting button as it moves, and
choice: deny taps the refusing one, which reads "Not Now" throughout. The covered languages are
English and Japanese, as for the other prompts.
This reactive guard and the proactive handleSystemAlert step below share the same native
SpringBoard mechanism for the prompts SpringBoard owns (BE-0316's query + tap); they differ in when
they fire — the guard automatically wherever a prompt surfaces, the step at the one point an author
places it.
Layering: a key reaches a run from three places¶
A declaration reaches a run from the scenario, the command line, and the target config
(BE-0177,
BE-0473).
The scenario's own interrupts and the target's hooks.interrupts take the same three forms, and
each form combines by its own rule:
| Entry form | How the target's and the scenario's entries combine |
|---|---|
| Screen | Concatenated, the target's entries first |
| System alert | Per prompt: the scenario's entry replaces the target's choice for that prompt and inherits an omitted frozenScreenTimeout from the target's entry; the other prompts from both apply |
| TipKit | The scenario's entry (dismiss or leave), else the target's; absent from both means off |
The two tuning values resolve differently, because the guard holds one of each per scenario:
| Value | How the layers combine |
|---|---|
pollInterval |
The first layer that sets one wins: the scenario, then --alert-poll-interval, then the target. Within one layer, the smallest value wins, since a shorter interval satisfies every entry that asked for a longer one. The built-in value is one second |
frozenScreenTimeout |
The most tolerant value among the entries that answer a prompt: the largest, with 0 (the stop off) over any value. A stricter bound would stop a wait on a screen another entry's app was declared to hold that long. The built-in value is 10 seconds |
| on / off | --no-system-alert-handling turns the guard off for every scenario; no scenario or target key does. --ios-tipkit-handling / --no-ios-tipkit-handling set every scenario's tipKit entry to dismiss or leave |
Specificity settles a conflict, not the layer a declaration came from. A target entry for the
tracking prompt answers tracking even inside a scenario that declares notifications alone. Each
entry reaches the one prompt it names, so both stay in effect. To override the target's entry,
write the scenario's own entry for the same prompt. That override reaches the one prompt you named.
choice: leave is the override for a scenario that means to answer nothing. The removed
systemAlertHandling: false named no prompt, whereas a leave entry names the prompt it leaves. A
prompt the target adds later thus still applies to the scenario. Sometimes a target entry answers
inside a scenario that declares systemAlert entries of its own. run then prints a notice naming
the prompt, once per affected scenario and prompt.
Migrating from the old keys¶
BE-0401, BE-0406, and BE-0473 removed the keys below with no aliases. Each fails to load with an
error naming its replacement, rather than parsing into something that behaves differently from what
it says. A run directory stored before the change still loads in the offline readers (the report
re-render, trace, and triage), which translate its scenario.yaml to the new keys.
| Removed | Write instead |
|---|---|
preconditions.run.systemAlertHandling: { rules: [{ prompt: notifications, choice: grant }] } |
- { systemAlert: notifications, choice: grant } in the scenario's interrupts (BE-0473) |
preconditions.run.systemAlertHandling: false |
choice: leave on each prompt the scenario answers itself, or --no-system-alert-handling for the whole run (BE-0473) |
systemAlertHandling.pollInterval / .frozenScreenTimeout |
the same keys on the systemAlert entries they serve (BE-0473) |
preconditions.run.tipKitHandling: true / false |
- tipKit: dismiss / - tipKit: leave in the scenario's interrupts (BE-0473) |
preconditions.run.inheritSetup |
preconditions.inheritSetup (BE-0473) |
visionInstruction |
nothing; run reaches no model (BE-0402), and record / crawl take --alert-vision-instruction |
labels: ["Allow"] |
- { systemAlert: notifications, choice: grant } — name the prompt you expect, since the label named no prompt at all (BE-0406) |
instruction: ["Allow"] |
the same entry form — BE-0401 renamed this key to labels, which BE-0406 then removed |
instruction: "tap Allow" |
the same entry form — BE-0401 renamed it to visionInstruction, which BE-0402 left reaching no command |
a scenario's top-level systemAlertHandling: / alertHandling: / dismissAlerts: |
systemAlert entries in the scenario's interrupts |
a scenario's top-level iosTipKitHandling: |
- tipKit: dismiss in the scenario's interrupts |
--alert-labels "Allow,OK" / --alert-instruction "Allow,OK" |
a systemAlert entry in the scenario or the target config — no flag carries a prompt paired with a choice legibly. A run request to the serve HTTP API carrying alertLabels is refused with 400 |
--alert-instruction "tap Allow" |
--alert-vision-instruction "tap Allow" (record / crawl only — run retired it, BE-0402) |
--alert-handling / --dismiss-alerts |
--system-alert-handling |
handleSystemAlert (the deterministic system-alert step)¶
The systemAlert guard above is reactive: it fires automatically wherever a prompt surfaces.
handleSystemAlert is its proactive counterpart: an explicit,
deterministic step the author places at the exact point a prompt is expected, which taps the
prompt's button by a native accessibility query, with no screenshot and no model
(BE-0316). Reach
for it to test a request-and-grant flow itself: fire the OS permission request, then grant or deny the
prompt that follows, deterministically.
- name: grant the notification prompt mid-flow
steps:
- tap: { id: perm.requestNotif } # fires the OS permission request
- handleSystemAlert: { sel: { label: "Allow" }, timeout: 5 } # tap the prompt's button by label
- wait: { for: { id: perm.notif.authorized }, timeout: 5 } # request granted, app state updates
To dismiss the prompt rather than accept it, target the dismissive button
(handleSystemAlert: { sel: { label: "Don't Allow" }, timeout: 5 }).
selis label-based only. A SpringBoard alert button carries no app-assigned identifier, trait, or value — only its visible text — soselacceptslabel/labelMatches/indexand rejectsid/idMatches/traits/value/withinat parse time.- The label a run must match is the one the target's
runsOn.ios.localerenders. SpringBoard owns the prompt, so it used to render in whatever system language the Simulator happened to carry. Alabel: "Allow"therefore worked by accident on an English machine and failed on a Japanese one. A run now pins the Simulator's own system language to thatlocalebefore the app launches, solabel/labelMatchesresolve identically on CI, on a teammate's Mac, and on a contributor's Simulator alike (BE-0320). timeoutis required, exactly as forwait: a condition wait for the prompt needs an explicit bound. The step waits the prompt in, then taps — no fixed sleep.- The reactive guard runs while the step waits
(BE-0406).
That wait is the runner's, not the backend's, so the system-alert guard clears an interruption
during the step instead of only after it has already spent its timeout and failed. This is what
the step needs when another prompt lands first — iOS's save-password alert, which the app's own
process raises and the SpringBoard query cannot see, covers the screen and stops the prompt this
step waits for from ever being raised. Without the guard running here, the step spent its whole
timeout and failed naming the permission prompt it never saw. The guard leaves this step's own
prompt alone, so a scenario may hold a
systemAlertentry for it with the opposite choice and the step still decides it. - Zero or many both fail. No prompt within
timeoutfails the step; more than one button matching the label is ambiguous and is never tapped unlessindexselects the nth — the same rule every selector follows, applied to the alert's buttons. An ambiguity is polled through rather than failed on at once, because a redraw can resolve one; a duplicate still there at the deadline fails the step and says so. The timeout names what the step saw — nothing on screen, an alert whose buttons the selector did not name, or one naming its label twice:
no system alert appeared within 5.0s: {'label': 'Allow'}
no system alert button matching {'label': 'Allow'} appeared within 5.0s (the alert on screen offered: Save Password, Not Now)
system alert button {'label': 'Allow'} is ambiguous and stayed ambiguous for 5.0s (the alert on screen offered: Allow, Allow) — add index to pick one
Where the guard met a prompt nothing could clear, its own note rides along as it does on a
blocked wait.
- iOS (XCUITest) only. Only that backend declares the capability, so a scenario naming
handleSystemAlert against the Android or web backend fails preflight, before any device work.
Android surfaces a system dialog in its ordinary element tree, so a plain tap reaches it there;
the web backend has no OS-level prompt at all.
When to reach for handleSystemAlert versus the two alert fields it stands beside:
| Field | For | Timing | Mechanism |
|---|---|---|---|
permissions |
an OS permission prompt you can avoid outright | pre-launch, before the app starts | deterministic device mutation |
handleSystemAlert |
a known mid-flow prompt you mean to tap | an explicit step where you place it | deterministic (native accessibility tap) |
systemAlert entries in interrupts |
an unexpected prompt: out of process, or the in-process save-password sheet | reactive, when a step or wait is blocked | native SpringBoard query on XCUITest (no model, reusing BE-0316), plus an in-tree tap for the sheet that query cannot see; a prompt no entry identifies is reported on the failure |
Naming the intent instead of the text¶
For the prompts permissions cannot pre-answer — notification authorization, which is not a
TCC (Transparency, Consent, and Control) service; App Tracking Transparency (ATT), which has no
simctl toggle at all; and the cross-process paste consent, which iOS records as
kTCCServicePasteboard yet exposes through no simctl toggle either
(BE-0369)
— the step takes a prompt and a choice in place of sel, and the run
resolves the label the pinned locale renders
(BE-0320):
- handleSystemAlert: { prompt: notifications, choice: grant, timeout: 5 }
prompt is notifications, tracking, paste, or localNetwork; choice is grant or deny.
The step rejects choice: leave at load, because a step that taps no button means nothing.
leave belongs to a systemAlert entry alone. The guard's own systemAlert entries take one more prompt
this step cannot, savePassword, which the step rejects at parse time —
see the surfaces table above. One step names the
button by its meaning, so the same file grants the prompt under en_US and under ja_JP without an
author transcribing either language's text. That pays off even for English alone, because English deny buttons
spell their apostrophe typographically (Don’t Allow, Don’t Allow Paste), not as the ASCII
character a hand-typed label carries.
The label table covers English and Japanese. Under any other language the step answers by the button's position instead (BE-0445): on all three prompts SpringBoard lists the deny button first and the grant button second, which was measured under English and Japanese on iOS 18.6 and 26.5, and under Arabic on 26.5, by tapping each button and reading the authorization status the app was left with.
The order is the order
SpringBoard reports, not the order on screen: Arabic draws the notification prompt's deny button on
the right, and it is still the first button. choice: grant taps button 2 of 2 and choice: deny taps button 1 of 2.
The rule names nothing on an alert with any other number of buttons, and the step then waits out its timeout and names the buttons it saw, rather than tapping by position on an alert the rule was never measured against.
The rule names a button, not a prompt: SpringBoard's own title and message are localized too, so nothing language-independent tells one two-button prompt from another, and any two-button SpringBoard alert up while the step waits is answered by position. Place the step right after the action that raises its prompt.
A run with no locale at all (record's replay) answers the same way. The step's
report row shows the button it tapped and the rule that chose it — sel, label table: <locale>,
or position: button 2 of 2. Every other alert keeps naming its button through sel, unchanged.
The position rule belongs to this step alone. A systemAlert entry must first tell
which declared prompt is on screen, which a position cannot do, so under a language the label table
does not cover it still fails the scenario loudly before any device work, naming what is covered.
Two limits are worth knowing before reaching for it:
- The Simulator only. The pin is a
simctloperation, so a target onrunsOn.ios.kind: deviceruns against whatever system language the physical device carries — the intent form would resolve a label nothing guarantees is on screen. Name the button withsel.labelthere. - The interruption monitor carries no English assumption either. A
systemAlertentry resolves its labels for the pinned language before it reaches the monitor, and there is no built-in fallback list behind it any more — an alert none of your entries identifies is always declined, in everylocale, and the step orexpectit interrupted fails naming its buttons rather than guessing at one. Write an entry for any prompt a run must decide itself.
(real files:
demos/showcase/scenarios/permission_system_alert.yaml,
demos/showcase/scenarios/paste_system_alert.yaml)
permissions (pre-launch permission state)¶
The system-alert guard reacts to a permission prompt only after it appears, and only by tapping it —
useful when the prompt is unexpected, but it cannot revoke a permission or guarantee the app
starts from a known state. When the permission is known ahead of time, permissions sets it
before the app process starts, so the prompt never appears at all: a deterministic,
machine-checkable device mutation with no model call
(BE-0276).
- name: profile — camera already granted
permissions:
camera: grant
location: grant
contacts: revoke
steps:
- tap: { id: profile.avatar.upload } # no camera-permission prompt — already granted
Each entry is <service>: grant | revoke, where <service> is one of a small backend-agnostic
vocabulary: location, camera, microphone, contacts, photos, calendar, notifications.
Each backend maps a service to its own native mechanism:
- iOS drives
simctl privacy <udid> <grant|revoke> <tcc-service> <bundle>— the same TCC (Transparency, Consent, and Control) database SpringBoard's permission prompts read. - Android drives
pm grant/pm revoke, reusing the plumbing behind the config-levelsubject.grantPermissionslist (drivers); a scenario'spermissionslayers on top of that config-level default and can revoke what it grants.
iOS has no TCC service for notifications (iOS notification authorization is not part of
TCC), so a scenario naming notifications on an iOS target fails preflight — before any device
work, naming the unsupported capability; a systemAlert entry remains the reactive path for that one
prompt. Android's POST_NOTIFICATIONS is a runtime permission (API 33+), so Android supports the
whole vocabulary. Every other unsupported combination (a service unsupported on the chosen backend)
fails preflight the same way, named individually.
permissions has no app-level XCUITest / Espresso equivalent, so codegen emits a labeled
// TODO per service rather than generating code for it — bajutsu applies the field itself, before
the generated test's own launch step.
interrupts (handling unpredictable interstitial screens)¶
An if step (below) checks its condition at one point in the step
sequence — the right tool when you know exactly which step precedes the screen you are branching on.
It is the wrong tool when a screen's appearance is not tied to any one step: an onboarding overlay, a
tutorial, or an in-tree permission prompt can each surface a few steps earlier or later than
expected, or not at all, depending on account state, network timing, or an A/B cohort. A single if
only catches the screen when it appears exactly where the if sits; every other timing slips through
and fails the rest of the scenario against a screen it was not written to expect.
interrupts handles that case. The list holds three forms of entry, and the key an entry writes
decides its form. This section covers the screen form. The other two forms are
systemAlert, for an operating-system prompt, and
tipKit, for Apple's TipKit popover. An entry that mixes keys of
two forms fails at load and names both forms.
A screen entry names a condition — the same assertion DSL if uses —
and the steps that clear the screen. The runner checks each entry opportunistically, wherever in
the sequence the screen happens to appear, and runs the entry's steps when the condition matches.
That check is free where it rides a tree already read for this step — a wait's poll tick, or
the fresh before a screenChanged-policy step reads when it has no carried-over tree to reuse.
Every other non-wait step pays one extra driver.query(), including a screenChanged-policy step
whose before is the previous step's carried-over tree (BE-0234), which the guard re-reads rather
than trust as current.
After the handler runs, the interrupted step resumes where it
left off: a wait keeps polling toward its original timeout, and an act step takes its action. An
author therefore no longer has to predict the one spot to place an if.
# config.yaml — an app-wide default: this app's onboarding screen, on every scenario
targets:
myapp:
platform: ios
subject: { id: com.example.myapp }
hooks:
interrupts:
- condition: { exists: { id: onboarding.skip } }
steps:
- tap: { id: onboarding.skip }
# scenario.yaml — this scenario's own addition, appended to the config-level list
- name: log in
interrupts:
- condition: { exists: { id: att.dialog } } # App Tracking Transparency prompt
steps:
- tap: { id: att.allow }
steps:
- tap: { id: login.button }
- wait: { for: { id: home.title }, timeout: 10 } # an interstitial mid-flow is cleared, then this passes
An interrupts list set at the config level (targets.<name>.hooks.interrupts) is an app-wide
default; a scenario's own interrupts is appended to it, config entries checked first — the
same config-then-scenario layering every form of entry follows (layering). An entry's steps share the
enclosing scenario's vars.* bindings, exactly as if's branches do.
A handler may itself use a
component, which expands before the run. A config-level entry may too: a
use under targets.<name>.hooks.interrupts resolves when the config loads, against the config
file's directory (reuse).
If a handler's
own steps never clear its condition (a broken selector, a screen that re-renders identically),
the entry fires only a small bounded number of times per step, and then the step falls back to its
ordinary outcome (pass, fail, or timeout). A mis-set entry thus fails the step cleanly rather than
hanging the run.
A scenario may declare two or more targets.
Each interrupts entry then watches one of them, named by the entry's own target field. An entry
without target watches the primary target, the first name in targets. A step's target becomes
required at two declared targets. An entry's target never does. Most entries concern the primary
target. Requiring the field would repeat one name on each of them. A recovery step in the entry's
steps may omit target too. That step then runs on the target the entry watches, not on the
primary.
- name: a web consent banner during a cross-platform flow
targets: [showcase-app, showcase-web]
interrupts:
- condition: { exists: { id: onboarding.skip } } # no target: watches showcase-app
steps:
- tap: { id: onboarding.skip }
- target: showcase-web
condition: { exists: { id: cookie.accept } }
steps:
- tap: { id: cookie.accept } # runs on showcase-web
steps:
- target: showcase-app
tap: { id: post.like }
- target: showcase-web
wait: { for: { id: post.likeCount }, timeout: 10 }
A config-level entry never names a target. The targets.<name> block that declares the entry
already fixes its target. A target there would repeat that name or contradict it, and neither
adds anything. The config load refuses target on such an entry and on any step in its steps.
The check is the deterministic assertion DSL, never a model call, so interrupts adds no AI to the
run verdict — and neither does a systemAlert entry since BE-0402. The difference between them is
what they reach: the alert guard answers out-of-process system prompts the accessibility tree
cannot see, through the SpringBoard query, while interrupts handles a screen the tree can
see with a machine-checkable condition. When to reach for which:
| Field | For | Timing | Mechanism |
|---|---|---|---|
if |
a screen at a known point in the sequence | one scripted check | deterministic (assertion DSL) |
interrupts |
a screen at an unpredictable point, visible in the tree | checked opportunistically throughout | deterministic (assertion DSL) |
handleSystemAlert |
a known out-of-process prompt you mean to tap mid-flow | an explicit step where you place it | deterministic (native accessibility tap) |
systemAlert entries in interrupts |
an unexpected prompt: out of process, or the in-process save-password sheet | reactive, when a step or wait is blocked | native SpringBoard query on XCUITest (no model, reusing BE-0316), plus an in-tree tap for the sheet that query cannot see; a prompt no entry identifies is reported on the failure |
permissions |
an OS permission prompt you can avoid outright | pre-launch, before the app starts | deterministic device mutation |
No native XCUITest / Espresso / Playwright construct maps onto "check this condition opportunistically
throughout the whole test," so codegen emits a labeled // TODO naming the field and each
configured condition rather than generating code for it — bajutsu run is the faithful path.
setup / cleanup (setup and teardown phases)¶
Setup written at the front of steps runs indistinguishably from the scenario's own steps: the
report lists them in one numbered sequence, and a setup failure surfaces as an ordinary step failure
with no marker showing it came from setup. Teardown has no mechanism at all. The only place to put cleanup is the tail of
steps, and the step loop breaks on the first failure (the run loop), so a trailing
cleanup step runs only when every preceding step already passed — exactly the run that needed
cleanup least. A scenario that signs up a test user, then hits a broken button three steps later,
leaves that user behind.
setup and cleanup close both gaps. setup is an ordered list of steps that runs first, reported
as its own section, and a failure there aborts the scenario before steps and expect run at all.
cleanup is a list of rules, each pairing an outcome — always, success, or failure — with the
steps to run for that outcome. The runner evaluates the rules once the scenario's verdict exists,
and reaches the phase on every path out of steps, the failing path included. Both fields reuse the
ordinary step grammar and the ordinary assertion DSL, so a hook's steps are exactly as
machine-checkable as the scenario's own, and both share the run's ${vars.*} bindings
(runtime variables).
- name: sign up, then release the account
setup:
# the seed endpoint returns the new user's bare id as its response body
- http: { method: POST, url: "https://api.test/users", saveBody: userId }
steps:
- tap: { id: login.button }
- type: { text: "${vars.userId}", into: { id: login.username } }
cleanup:
- on: always
steps:
- tap: { id: session.logout }
- on: success
steps:
- http: { method: DELETE, url: "https://api.test/users/${vars.userId}" }
- on: failure
steps:
- http: { method: POST, url: "https://api.test/diagnostics", body: '{"failed":true}' }
(real file: demos/showcase/scenarios/setup_cleanup.yaml)
Before BE-0452,
the two fields were before and after, and the outcome word failure was error. BE-0452 gave a
target's hooks and a scenario's phases one pair of words. The loader reads no old name. A scenario
that still writes one fails to load, and the error names the replacement.
More than one rule may carry the same on value, and rules composing that way run in declaration
order — the same way two capturePolicy rules may share a trigger. A rule whose own steps fail does
not stop the phase: the remaining rules still run, because skipping the rest of the cleanup is the
outcome teardown exists to avoid. What that failure does to the run's verdict depends on where the
run already stood. On a run that was passing, the failing rule becomes the failure
(cleanup: step 0 (tap): …). On a run that had already failed, the failing rule is appended behind
the original failure instead of replacing it, so the reason a reader sees first is still the original
cause rather than a symptom of the cleanup it triggered.
A cancelled run (SIGTERM, the serve Web UI's Cancel button) reaches the phase too, dispatching
cleanup as a failure outcome, and the cleanup rules get a bounded slice of the cancellation grace
window to run in. Once that slice is spent, the remaining rules are abandoned so the shutdown tail
that writes the report still fits inside the window.
The target's hooks: hooks.setup and hooks.cleanup¶
targets.<name>.hooks.setup takes the shape of a scenario's setup as an app-wide default.
targets.<name>.hooks.cleanup takes the shape of a scenario's cleanup. Each merges with the scenario's own field,
in opposite orders:
| Field | Merge order | Why |
|---|---|---|
setup |
config (hooks.setup), then scenario |
The app-wide setup seeds the state this scenario's own setup then builds on — the same config-then-scenario layering interrupts follows |
cleanup |
scenario, then config (hooks.cleanup) |
This scenario releases what it created before the app-wide teardown closes around it, the last-acquired-first-released order a fixture-based teardown pair gives |
With use:, a hook step calls a component. A login flow that every
scenario of a target shares then lives in one file.
BE-0452
removed the setup prelude file, at the target level and as preconditions.setup alike. A
scenario that must not run its targets' setup sets preconditions.inheritSetup: false. Their
hooks.cleanup still runs, since cleanup runs on every path out of a scenario.
When to reach for which¶
Three fields sit near this ground, and each answers a different question:
| Field | Runs | Reported as | For |
|---|---|---|---|
setup / cleanup |
as its own phase, before steps / after the verdict |
its own setup / cleanup block | setup and teardown the reader must be able to tell apart from the scenario under test |
use (a component) |
expanded in place, where the step calls it | more numbered steps | a reusable flow shared by several scenarios, where no separation is wanted |
capturePolicy |
throughout the step loop, per step | evidence attached to a step | capturing extra evidence when a step fails, not running steps |
capturePolicy's on: { result: failure } trigger and a cleanup rule's on: failure share the
word failure for the same idea, at two scales: a capturePolicy trigger fires for one failed step,
wherever in the run it happened, while a cleanup rule fires once, for the whole scenario's verdict.
What codegen emits¶
setup needs no framework construct: codegen emits its steps inline at the top of the generated
test body under a // setup divider, which is exactly the phase's meaning — they run first, and a
failure aborts what follows. cleanup needs one, and each target reaches it differently. Playwright and
UI Automator wrap the test body in try / catch / finally, since an assertion on either target
throws, so the catch sees the very failure the verdict would have been. XCUITest registers a single
addTeardownBlock instead, because XCTAssert records a failure rather than throwing, and reads the
outcome from testRun?.hasSucceeded (codegen).
Selectors (addressing an element)¶
A selector identifies which element to act on or assert against. Provide one or more fields; multiple fields are AND-ed (all must hold), and at least one is required. How a selector resolves to exactly one element, and why an ambiguous selector fails instead of picking the first match, is covered in selectors. The formal shape is in dsl-grammar.
| Field | Type | Description |
|---|---|---|
id |
str | list[str] | Exact accessibilityIdentifier — first choice (stable, non-localized). A list is an OR of candidates: the element's id must equal any one |
idMatches |
str | list[str] | Glob over the id (e.g. "list.row.*"; assumes multiple matches). A list matches if the id matches any glob |
label |
str | Exact accessibilityLabel (visible text) — auxiliary / disambiguation |
labelMatches |
str | Regex / substring over the label (re.search) |
traits |
list[str] | Narrow by accessibility trait (subset test, e.g. [button]) |
value |
str | Exact accessibility value |
within |
Selector | Scope to a container — the match must sit inside an element the nested selector resolves to (nestable) |
index |
int | Pick the k-th of multiple matches (negatives allowed) — last resort, order-sensitive |
- tap: { id: counter.increment } # by id (recommended)
- tap: { id: [stable.refresh, stable_refresh] } # OR of id candidates (see below)
- tap: { label: "Delete" } # by visible label (e.g. an alert button)
- tap: { id: row.action, within: { id: list.row.3 } } # scoped to a container's subtree
- tap: { labelMatches: "^Item ", traits: [button], index: 0 } # first matching button, fields AND-ed
Prefer
id. For a set of elements (count / existence) useidMatches. Useindexonly as a last resort — it breaks when order changes. Full resolution semantics: selectors.
Cross-platform ids: a candidate list (BE-0221)¶
A scenario is shared across platforms only to the extent its selectors are by id, and the driver decides which app-side attribute satisfies that id. But some platforms can't reproduce the SPEC id verbatim: Android's android:id (the Views toolkit) allows neither . nor -, so stable.refresh surfaces as stable_refresh and search.results-empty as search_results_empty.
To keep one scenario running unchanged everywhere, give id / idMatches a list of candidates and the match becomes an OR over them:
- wait: { for: { id: [stable.refresh, stable_refresh] }, timeout: 10 }
- count: { sel: { idMatches: [stable.row.*, stable_row_*] }, equals: 5 }
The dotted form matches on iOS and Android Compose (which reproduce it verbatim); the underscore form matches on Android Views. Only one form is ever on screen for a given app, so the selection stays deterministic: if both candidate forms happened to be present at once, the selector is ambiguous and fails fast — an OR never turns a two-or-more match into a silent pick. The candidate list keeps the id convention explicit in the scenario, rather than a hidden driver-side .↔_ rewrite that could conflate distinct ids. The showcase's shared scenarios use this so showcase-swiftui / showcase-compose / showcase-views all run the same files.
List the underscore form only in a scenario a Views target actually runs: E2E_VIEWS_SCENARIOS in demos/showcase/android/Makefile, or a lane that drives showcase-views directly. In any other scenario no lane resolves that candidate, so a typo in it never fails and a reader cannot tell a needed candidate from a dead one; such a scenario carries the single dotted id.
Step grammar (steps)¶
Each step is exactly one action plus optional modifiers (capture: / name:). A step with two
or more actions fails validation (scenario/models/steps.py _one_action).
| Action | Form | Description |
|---|---|---|
tap |
tap: <Selector> |
requires unique resolution (fails if ambiguous) |
tapPoint |
tapPoint: { x: <frac>, y: <frac> } |
tap a normalized screen coordinate (0..1, top-left origin) instead of a selector — the bottom rung of the stability ladder, for a control the accessibility tree exposes as no addressable element (for example, a no-id tab-bar tab); record's vision path emits it, and run replays it against the current screen size |
doubleTap |
doubleTap: <Selector> |
two quick taps on the resolved element |
longPress |
longPress: { sel: <Selector>, duration: <sec> } |
long press |
type |
type: { text: "...", into?: <Selector>, submit?: <bool> } |
with into, focuses first |
clear |
clear: { into: <Selector> } |
focus the field and remove its entire current content; web context raises |
delete |
delete: { into: <Selector>, count: <int> } |
focus the field and delete count characters from the end (count > 0); web context raises |
select |
select: { into: <Selector>, mode?: "all" } |
focus the field and select its content (mode default all); the web context raises — the iOS (XCUITest) backend supports it natively, and codegen emits the native equivalent |
copy |
copy: {} |
copy the active selection to the clipboard; requires a prior select; the web context raises — the iOS (XCUITest) backend supports it natively |
paste |
paste: { into: <Selector> } |
focus the field and paste the clipboard at the caret that focusing tap leaves (on a field that already holds text, wherever the tap lands), replacing any selection; the clipboard holds what a copy or a setClipboard put there; verify the result with a value assertion; the web context raises. On iOS it taps the edit menu's Paste item (a hardware Cmd+V pastes nothing from a clipboard setClipboard wrote), accepting UIKit's label in any language because the menu follows the app's localization. Depending on the Simulator build that paste raises iOS's paste-consent prompt, so declare interrupts: [{ systemAlert: paste, choice: grant }] to have the guard answer it |
selectOption |
selectOption: { sel: <Selector>, option: "..." } |
set a web <select> to the option with this value; web only (iOS / Android raise) |
selectPhotos |
selectPhotos: { indices: [0, 1], timeout: 10 } |
pick the cells at these positions (0 = the first cell, newest image first) from the system photo picker the app has already opened, then confirm: PHPickerViewController on iOS (XCUITest), the Android 13+ photo picker on adb. A single-selection picker closes on its one tap; an index beyond the visible grid fails |
setPickerValue |
setPickerValue: { sel: <Selector>, value: "..." } |
move a wheel-style picker (UIPickerView or a wheel-mode UIDatePicker on iOS; NumberPicker or a spinner-mode DatePicker on Android) to the row with this value (below); iOS (XCUITest) and Android (adb). sel addresses one wheel — a multi-component picker's siblings are separated by within / traits / index, one step each |
swipe |
swipe: { on: <Selector>, direction: up\|down\|left\|right } or swipe: { from: [x,y], to: [x,y] } |
selector form and coordinate form cannot mix; the directional form scrolls |
drag |
drag: { on: <Selector>, direction: up\|down\|left\|right, amount?: <frac> } |
a real pointer drag of the element (a handle / divider / slider), not a scroll |
scroll |
scroll: { to: <Selector>, direction?: up\|down\|left\|right, within?: <Selector>, amount?: <frac>, maxScrolls?: <int> } |
scroll (non-inertially) until to is on-screen, or fail at a bound; direction is scroll direction (default down), the inverse of swipe's |
back |
back: {} |
navigate back one level, each backend using its platform-correct primitive — the Android system back key, the iOS OS-provided back button, or web history (BE-0210) |
navigate |
navigate: "<url>" |
load a URL in the current page, resolved against the target's baseUrl (below); web only (iOS / Android fail in preflight) |
openUrl |
openUrl: "<url>" |
hand an absolute URL, such as a deep link, to the operating system for the app the scenario drives at that point, and wait until that app is in front (below); iOS (16.4 or later) and Android (the web backend fails in preflight) |
pinch |
pinch: { sel: <Selector>, scale: <num> } |
two-finger magnify; scale > 0 (>1 zooms in, <1 out) |
rotate |
rotate: { sel: <Selector>, radians: <num> } |
two-finger rotation; >0 is clockwise |
handleSystemAlert |
handleSystemAlert: { sel: <Selector>, timeout: <sec> } |
tap a button on an iOS SpringBoard permission prompt, deterministically (below); iOS (XCUITest) only. sel accepts only label / labelMatches / index, and resolves against the system language the run pins the Simulator to. In place of sel, prompt: notifications\|tracking\|paste + choice: grant\|deny names the button by meaning and lets the run resolve its label (BE-0320), or its position under a language the label table does not cover (BE-0445) |
wait |
wait: { for\|until: ..., timeout: <sec> } |
condition wait (below) |
sleep |
sleep: { seconds: <sec>, reason: "..." } |
fixed pause for a delay no condition can observe; at most 30s, reason mandatory (below) |
assert |
assert: [ <Assertion>... ] |
mid-step verification |
relaunch |
relaunch: { env?: {...}, args?: [...] } |
terminate + relaunch the app (re-applying launch env/args, plus the given overrides), then wait until ready |
setLocation |
setLocation: { lat: <num>, lon: <num> } |
override the simulated GPS location (simctl location set) |
push |
push: { payload: {...} } |
deliver a simulated push notification (simctl push) with this APNs (Apple Push Notification service) payload |
http |
http: { method?, url, headers?, body?, status?, saveBody?, extractBody?: [{ var, path }] } |
issue an HTTP request (test-data setup / webhook / API); checks status, stores the body as ${vars.<saveBody>}, and/or pulls named JSON fields into ${vars.<var>} by path (below) |
totp |
totp: { secret, into?: { var }, name? } |
generate an RFC 6238 time-based one-time password (2FA) locally into ${vars.<var>}; name also registers the secret for later otp steps |
otp |
otp: { from, into: { var }, minRemaining? } |
write the current code of the generator a totp step registered as from into ${vars.<var>} (below) |
email |
email: { match: { to?, subject?, subjectMatches? }, extract: { var, bodyMatches }, timeout } |
poll the configured mailbox until a matching message arrives, extract a code into ${vars.<var>} |
generate |
generate: { random\|datetime: {...}, into: { var } } |
compute a random or current-datetime value at run time into ${vars.<var>} (below) |
eval |
eval: { from, ops: [...], into: { var } \| { vars: [...] } } |
split, join, filter, and otherwise transform a value the scenario already holds, into ${vars.*} (below) |
manual |
manual: { label: "...", bypass?: "..." } |
a human takeover recorded during record (BE-0185); has no deterministic run-time equivalent, so it fails loudly at run time — never a silent pass |
background |
background: {} |
send the app to the background (Home button) |
foreground |
foreground: {} |
bring the step's target app to the front without terminating it; an app that is not running launches the way relaunch would, minus the terminate, then waits until ready. iOS and Android |
installApp |
installApp: { from: <target>, keepData?: bool } |
install a later device-group member's build onto the step's device, over an existing build of the same identifier (keepData, default true) or after uninstalling it (false); launches nothing (device groups) |
setPrimaryTarget |
setPrimaryTarget: { target: <target> } |
from this step on, a step, interrupts entry, or top-level expect entry that omits target runs on target; top-level steps only (device groups) |
clearKeychain |
clearKeychain: {} |
reset the Simulator keychain (saved passwords / certificates) |
clearClipboard |
clearClipboard: {} |
clear the device clipboard (iOS Simulator; Android with BajutsuAndroid embedded) |
setClipboard |
setClipboard: { text: "..." } |
seed the device clipboard for a paste flow (iOS Simulator; Android with BajutsuAndroid embedded) |
setMocks |
setMocks: [ { match, respond }... ] |
replace the app's whole stub table mid-scenario over the in-app control channel (below); [] removes every stub. iOS (XCUITest) with network collection on (BE-0365) |
overrideStatusBar |
overrideStatusBar: { time?, batteryLevel?, batteryState?, cellularBars?, wifiBars? } |
override the status bar for deterministic screenshots |
clearStatusBar |
clearStatusBar: {} |
remove status-bar overrides (restore the live bar) |
use |
use: { component: <file>, with?: {...} } |
expand a reusable component's steps — a compile-time macro (reuse); takes no modifier but target — capture / extract / name / from are rejected, and a target is stamped onto every step the component expands to (multi-target) |
group |
group: { name: <str>, steps: [...] } |
name a run of consecutive steps — a compile-time macro, folded together in report.html (below) |
web |
web: { within: <Selector>, steps: [...] } |
enter a WebView's DOM: within resolves the host WebView natively, and the nested steps address its normalized DOM instead of the native tree (below) |
Modifiers (a use step takes target alone — see the table row above):
capture: [<token>...]— evidence for this step only (evidence).name: <str>— the step id (the evidence output directory name · report label). Defaults tostep<i>.from: <str>— provenance (below): the phrase this step was recorded from. Authoring metadata;runignores it.target: <str>— which of the scenario's owntargetsthis step runs against.
tap¶
- tap: { id: counter.increment } # exact id (recommended)
- tap: { label: "Delete" } # exact label (for an in-app alert etc. with no id)
type¶
- type: { text: "a@b.com", into: { id: auth.email } } # focus, then type
- type: { text: "hello", submit: true } # submit appends a newline / confirm (uses current focus)
Internally, when
intois given,_do_action(orchestrator/actions/)taps the target before it callstype_text.
selectOption¶
- selectOption: { sel: { id: nav.theme-picker }, option: midnight } # set the <select> to the option whose value is "midnight"
The dropdown of a native HTML <select> is not part of the page's element tree, so a coordinate tap
cannot switch the value deterministically. selectOption resolves the <select> through the same
unique-match core every action uses. It then sets the option by its value (not its visible label)
and fires a change event, so the page reacts exactly as it would to a user's pick. The value
matches what a value assertion reads back from the <select>, so a selection is directly
assertable.
selectOption is a web-only action. A <select> has no native counterpart on iOS or Android, so
those backends fail the step with a clear "unsupported action" reason rather than doing nothing.
navigate¶
- navigate: /settings # resolves against the target's baseUrl
- navigate: https://idp.example.com/login # an absolute URL is used as written
- navigate: "/items/${vars.itemId}" # ${...} tokens are expanded like any step field
navigate opens a URL in the middle of a scenario. Typical uses are a page that no control links to
(a link from an email, a shared-item URL), a deep page a UI walk would otherwise have to reach, and
a redirect to exercise (BE-0464).
The value is a plain string, not a mapping.
The value resolves against the target's baseUrl the way a browser resolves a link (urljoin).
/settings replaces any path in baseUrl, settings resolves against that path, and an absolute
http / https URL is used as written, even on another origin. Any other scheme (javascript:,
file:, data:) fails. The loader refuses a literal value when the scenario loads. A value with a
${...} token fails the step once the run has substituted it.
The step returns once the page's load event fires. It does not read the response status, so a 404
page loads and the step passes; the next assert or wait decides the verdict. Cookies and storage
persist, so navigate continues the session rather than resetting it. A failure the URL itself
causes, such as an unresolvable host or a timeout, fails the step with the browser's message.
navigate needs the urlNavigation capability, which only the web backend declares. A scenario that
uses it on iOS or Android fails in preflight, before any device is leased, including a navigate
nested inside a web: block.
openUrl¶
- use: signIn
- openUrl: "showcase://tabs/settings" # the app under test receives the URL
- assert: [{ exists: { id: settings.title } }]
- openUrl: "showcase://items/${vars.itemId}" # ${...} tokens are expanded like any step field
- app:
bundleId: ${vars.browserId} # the browser receives the URL
steps:
- openUrl: "https://example.com/promo"
- tap: { label: "Open in app" }
openUrl hands a URL to the operating system in the middle of a mobile scenario
(BE-0487).
The launch deep link (subject.launch.deeplink) opens before any step runs.
So it cannot reach a screen that needs a signed-in user, and openUrl can.
The value is a plain string, not a mapping, and must be an absolute URL such as myapp://path.
A mobile backend has no baseUrl, so a relative value such as /settings fails the check.
A javascript:, data:, or file: URL fails it too, since none of them names a place in an app.
The scenario loader checks a literal value.
The run checks a value with a ${...} token after substitution, and fails the step on a bad one.
The URL goes to the app the scenario drives at that point.
Outside any app: block, the app under test receives it.
Inside an app: block, the app the block names receives it.
To open a URL in a browser, wrap the step in an app: block that names the browser.
Scoping the URL to one app keeps the step deterministic.
On Android, an unscoped intent can raise the app chooser when two installed apps claim one URL.
The step passes when the operating system accepts the URL and the receiving app comes to the front.
The wait for the front uses the bound of the app: block on that backend.
That bound is about 10 seconds on iOS and 30 seconds on Android.
The step waits for no screen content: a following wait or assert waits for the new screen.
Each backend decides acceptance its own way:
| Backend | How the URL is handed over | When the step fails |
|---|---|---|
| iOS (XCUITest) | the runner calls XCUIApplication.open(_:) on the receiving app |
the call raises or records an XCTest issue, whose message becomes the reason; or the device runs an iOS before 16.4 |
| Android (adb) | am start -W --activity-single-top -a android.intent.action.VIEW -d <url> <package> |
am start prints an Error line, an exception, or a status other than ok; or it prints an Activity not started warning other than the one saying the running activity received the intent |
On iOS, the receiving app is in front already mid-scenario, so the foreground wait proves nothing.
The step then rests on what open(_:) raises or records.
If open(_:) reports nothing for a URL that no app handles, the step passes.
The assertion that follows then catches the wrong screen.
On Android, --activity-single-top lets the step open the URL the launch deep link used.
The activity on top of its task receives the URL through onNewIntent.
The app must handle a deep link there, as most apps that accept deep links already do.
A URL may raise a system confirmation dialog, such as a browser asking to open another app.
The step leaves that dialog up, since it does not answer a prompt the real user would see.
A systemAlert entry in interrupts or a following handleSystemAlert step clears it.
openUrl needs the openUrl capability, which the iOS and Android backends declare.
On the web backend, a scenario with openUrl fails in preflight; that backend uses navigate.
The scenario loader refuses an openUrl inside a web: block.
A step there runs against the page, which has no operating system to hand a URL to.
setPickerValue¶
- setPickerValue: # move the wheel to the "大学" row
sel: { within: { id: form.school }, traits: [pickerWheel] }
value: "大学"
A wheel-style picker is an ordinary form control on both mobile platforms. No step but
setPickerValue can set one. On iOS the picker is a UIPickerView, or a UIDatePicker switched
to a wheel-only mode. On Android it is an android.widget.NumberPicker. A DatePicker or
TimePicker in spinner mode counts too, since its fields are NumberPicker wheels. The wheel's rows
are not separately addressable elements, so tap, which addresses a resolved handle, cannot land on
a specific row.
The coordinate-driven steps fare no better. swipe, drag, and scroll are bounded or directional
drags. They can spin a wheel roughly toward a value but cannot guarantee stopping on it. tapPoint
can only hit whatever row the wheel already shows. Asserting the result of any of those steps would
depend on a drag distance matching the row height by chance. Bajutsu rules out that approximate
action everywhere else.
On iOS, setPickerValue instead calls XCUITest's own adjust(toPickerWheelValue:) on the element
the selector already resolved. The step is handle-based the way tap is, and not coordinate-based
the way swipe is.
On Android, the adb backend resolves the wheel the way a tap resolves its target. It then hands
the wheel's identity to the resident UI Automator server, with no coordinate. The server steps the
wheel one row at a time. Each step is the accessibility scroll action NumberPicker itself provides.
After each step, the server reads the selected row back.
When the shown row and the value both parse as integers, the walk starts toward the value, so a year wheel takes the short way. Otherwise the walk goes forward first. On a wheel with ends, the walk turns back at the last row. A wrapping wheel stops once it comes back to the row it started on. A cap of 400 rows and a deadline bound the walk. The step needs the resident server: without it, the step fails instead of dragging the wheel by coordinates.
sel must resolve the wheel itself, which is seldom the element carrying the identifier. A
UIPickerView publishes its identifier on the picker and exposes the wheel as a separate child. A
selector naming that identifier alone resolves the parent instead, and the step fails, because
adjust(toPickerWheelValue:) raises on any element that is not itself a wheel. Pairing the
identifier with the pickerWheel trait, as above, reaches the child.
On Android, a NumberPicker's own node carries no resource-id at all. Put the identifier on a
container around the wheel instead. The same within plus pickerWheel selector then reaches the
wheel inside. The adb backend gives a NumberPicker node the pickerWheel trait, next to the
numberPicker trait its class name yields.
A value the wheel does not carry fails the step, naming the value, rather than leaving the wheel wherever it stopped. The following assertion then tests the app, not the gesture.
A multi-component picker (a year wheel beside a month wheel) exposes each component as its own
pickerWheel element, and sel always addresses exactly one of them. Use the within / traits /
index fields every selector already carries, one step per component. Both the component order and
the row labels follow the locale the run pins. The example below assumes ja_JP, whose wheels read
year | month | day. Under the config default en_US, the wheels read month | day | year, with rows
May and 2016. demos/showcase/scenarios/picker_wheel.yaml pins no locale and so reaches that
second layout:
- setPickerValue:
sel: { within: { id: form.birthdate }, traits: [pickerWheel], index: 0 } # the year wheel
value: "2016年"
- setPickerValue:
sel: { within: { id: form.birthdate }, traits: [pickerWheel], index: 1 } # the month wheel
value: "5月"
within scopes by frame containment, so its container must be large enough to hold the wheels. A
wheel-mode UIDatePicker fails that test on its own: iOS lays the components out at their intrinsic
height and then clips them to the picker, so every component reports a frame taller than the picker
publishing it. A within naming the date picker then matches nothing at all. Put the identifier on
a surrounding container whose frame covers the components instead. The showcase screen takes that
route (demos/showcase/ios/swiftui/Sources/PickerView.swift) and groups the caption, the wheel, and
the mirror text under one identifier.
This route also sidesteps the datePicker classification gap (selectors).
A UIDatePicker's own container element falls to other, but the step addresses the wheel children
underneath it, which are classified pickerWheel, so the gap never bites here.
setPickerValue runs on iOS (XCUITest) and Android (adb). On Android it drives any NumberPicker,
including one that a Jetpack Compose screen embeds through AndroidView. A wheel that Compose draws
itself stays out of scope: Compose ships no standard wheel, so each app's own wheel exposes a
different accessibility shape.
A web page has no picker wheel. A <select> expresses the same intent, and
selectOption sets it. The web backend thus lacks the pickerWheel capability, and
preflight stops the scenario there before any work starts, naming the step's location.
web (entering a WebView's DOM)¶
- web:
within: { id: checkout.webview }
steps:
- tap: { id: pay.submit }
- wait: { for: { id: pay.confirmation }, timeout: 10 }
A WebView is a browser view that a native app embeds. web runs its nested steps against the
Document Object Model (DOM) of such a view. The steps address the page's normalized DOM, not the
app's native accessibility tree: data-testid becomes Element.identifier. The block serves a
hybrid screen, where part of the UI is web content inside the app. iOS support came with
BE-0037, and Android
support with BE-0482. The
same block runs unchanged on both platforms. Control returns to the native driver once the block's
steps finish.
The nested steps share the enclosing scenario's vars.* bindings, the same as if's and
forEach's do. The capture / extract modifiers are not allowed on the web step itself.
How the block names its WebView. within is an ordinary selector, and it must carry an id.
The run resolves the whole selector against the native tree to one element, the WebView itself.
Zero matches or two or more fail the step, as for any selector. Each platform then finds the page
behind that element in its own way:
| Platform | How the run reaches the page | Which page it picks |
|---|---|---|
| iOS | The BajutsuKit WebView bridge, a Hypertext Transfer Protocol (HTTP) server inside the app, on the port the run passes as BAJUTSU_WEBVIEW_PORT. |
The WKWebView whose accessibility identifier equals the first id of within. |
| Android | The Chrome DevTools Protocol (CDP), on the WebView's debugging socket, which the run forwards with adb for the length of the block. |
The one attached, visible page whose rectangle contains the frame within resolved to. Zero matches or two or more fail the step as ambiguous. |
Android matches by containment, not equality, because the two rectangles measure different areas. The native tree clips a view's bounds to the screen and to its ancestors. CDP reports the page's full rectangle. A WebView partly hidden under a toolbar thus still matches.
What the app must provide. A block whose app is missing one of these pieces fails the step with a reason that names what is missing.
- iOS: the app links BajutsuKit and calls
BajutsuNet.startIfEnabled(). That call starts the bridge when the run passes the port. TheWKWebViewcarries an accessibility identifier. - Android: a debuggable build calls BajutsuAndroid's
Bajutsu.enableWebViewDebugging(context)inApplication.onCreate, or right before it creates its first WebView. Every debug build type setsandroid:debuggable. The helper does nothing in a non-debuggable build, so a release build never opens the socket. The machine running bajutsu needs theandroidextra (websockets).scripts/install.shinstalls it when the configured backends include Android.
What a block supports. Inside the block, tap / tapPoint / doubleTap / type / wait /
assert run. longPress / swipe / drag / clear / delete / select / copy /
paste / selectOption / scroll / back / navigate / pinch / rotate / handleSystemAlert /
setPickerValue are not reachable there, and each fails with a clear "not supported in web context"
reason. The block also leaves these cases out:
| Case | What happens |
|---|---|
Content inside an <iframe> |
The DOM walk skips a frame's document, so a selector cannot name an element inside it. |
A file chooser (<input type="file">) |
A tap reaches the input, but the chooser it opens is a system screen outside the page, which the block cannot drive. |
| Two WebViews that overlap, on Android | Both rectangles contain the frame, so the step fails as ambiguous rather than guessing. |
| An app that restarts mid-block, on Android | The debugging socket closes, and the step fails with the transport's reason. |
| Another debugger already attached to the page, on Android | The page offers no second connection, and the step fails. |
WebView content tapped by its native label, with no web: block |
Not supported. The native tree often shows a WebView's content too coarsely to select by, and the block is the supported route. |
| A browser tab that the system shows over the app | Not supported. The view belongs to a separate browser app, so neither the bridge nor the app's debugging socket reaches it. |
app¶
- app:
bundleId: com.apple.mobilesafari
steps:
- assert:
- exists: { id: TabBarItemTitle }
app drives another app's UI. It activates the named app by bundle id (any installed app, including
one the test target never launched; a running app is brought forward) and runs the nested steps
against that app's own accessibility tree, not the test target's. The feasibility findings behind
this behavior are recorded in docs/specs/ios-cross-app-ui-control-feasibility.md.
Control returns to whatever was active before the block once its steps finish, the same enter/leave
contract web uses. Nesting an app block inside another returns to the immediate parent, not
unconditionally to the test target. bundleId is a plain string, not a value drawn from a fixed
set: any installed app can be named, including one the test target has no way to open itself. iOS
(XCUITest) and Android (adb) support the step; a scenario using it against the web backend fails
preflight before any device work.
bundleId must name an app already installed on the device. An app that is slow to foreground (a
cold launch, a permission prompt) fails the step with ElementNotFound. On iOS, an app that is not
installed at all is not guaranteed to fail this cleanly.
On Android, bundleId is the package name (com.android.settings, say). The adb driver starts the
package's launcher activity with am start and waits until that package holds the resumed activity.
A package with no launcher activity, including one that is not installed, fails the step with
ElementNotFound before any launch. Leaving brings the app beneath back the same way, and the app
under test keeps its process meanwhile, so the handoff never reads as a crash. As on iOS, an
entered app is not reset: it keeps whatever state an earlier block or run left it in.
One Android limit applies on the way back: when an app's launcher activity is singleTask or singleInstance
(or sets clearTaskOnLaunch), starting it again destroys every activity its task pushed above it, so
that app comes back at its root screen rather than the screen it left.
Driving the Home Screen (com.apple.springboard)¶
SpringBoard, the process that draws the Home Screen, is one more bundle id to app. A scenario can
long-press into edit mode, open the widget gallery, and add a widget with no new step and no
runner change. demos/showcase/scenarios/widget.yaml does that, asserts the new widget, and then
taps the widget to open its host app. The measurements behind it are in
docs/specs/ios-home-screen-widget-feasibility.md: each screen's labels in en_US and ja_JP, and
every gap below.
- background: {} # hand the screen to the Home Screen first
- app:
bundleId: com.apple.springboard
steps:
- longPress: { sel: { id: "Home screen icons" }, duration: 2 } # empty wallpaper → edit mode
- tap: { label: "Edit", traits: [button] }
- tap: { label: "Add Widget", traits: [button] }
The constraints come from driving an operating-system screen the scenario does not own:
- Measured on the iOS Simulator, iOS 26.5. We checked no real device and no other runtime.
- Pin the layout with
preconditions.subject.reinstall: erase. The long press lands on the icon grid's centre, which must be empty wallpaper. After an erase and reinstall, SpringBoard opens on the page that holds the test apps, and that page leaves its lower half free. Erasing also removes the widgets an earlier run added, so the assertion on the new widget stays unambiguous. - Pin the language with
preconditions.device.ios.locale. Every label is SpringBoard's own text. Even an icon'sidentifieris its localized name:Calendarin English,カレンダーin Japanese. - A search result row is absent from the tree. Once the gallery's search field holds text,
SpringBoard's snapshot reports the result list as an empty container, although the row still
shows on screen. Tap the first row with
tapPointat its fixed place under the search field. On an iPhone 17 Pro, that place is{ x: 0.5, y: 0.29 }. - Pick a size with
drag, notswipe. The size carousel pages on a flick alone. The non-inertial scroll behindswipesettles back onto the page it started from. - A host app's first launch can raise prompts. Calendar asks for location and notifications
access, in either order or both at once. Two prompts leave no single button to tap, so
widget.yamlleaves them up; its next run'seraseclears them.
swipe¶
- swipe: { on: { id: comp.swipearea }, direction: left } # frame center → a screen fraction in a direction (default 0.125)
- swipe: { from: [100, 400], to: [100, 200] } # raw coordinates (last resort)
A swipe takes exactly one or the other of {on,direction} and {from,to}. Mixing the two forms
or omitting a side is a validation error.
The directional form means "scroll", and each backend realizes it with the primitive that
actually scrolls. iOS and Android use a real OS drag. On web, a mouse drag does not scroll a page, so
the backend uses a wheel event (desktop) or a touch drag (a mobile
runsOn.browser.emulate) instead (BE-0227). The coordinate form is
a literal pointer drag for its own sake (a canvas / map pan / drag handle), the same raw-drag last
resort on every backend.
drag¶
- drag: { on: { id: replay.divider }, direction: right } # drag a grabbed handle
- drag: { on: { id: volume.slider }, direction: up, amount: 0.3 } # ... a fraction of the screen
drag is an element-anchored pointer drag. It grabs the element and moves it in a direction,
which suits a resize divider, a slider thumb, a reorder handle, or any other control you drag rather
than scroll. It shares swipe's directional geometry (amount is a fraction of the screen,
0 < amount ≤ 1; omitted, a small default). Where a directional swipe scrolls, drag
performs a genuine pointer drag.
The distinction only bites on web, where a directional swipe is a wheel scroll that would leave a
grabbed handle unmoved. Use drag there. On iOS and Android, a real OS drag both scrolls and moves
handles, so the two coincide.
scroll¶
- scroll: { to: { id: notice.row.20 } } # scroll down until the row appears, then …
- tap: { id: notice.row.20 }
- scroll: { to: { label: "Log out", traits: [button] }, # scroll a specific container …
within: { id: settings.list }, maxScrolls: 25 }
- scroll: { to: { id: chart.point.7 }, amount: 0.2 } # … in finer steps than the default
scroll brings an off-screen element into view: it scrolls one step, re-queries the tree, and stops
the moment to resolves with its frame's center on-screen — the point a following tap aims at,
so a target taller than the viewport still succeeds once its center is reached.
scroll reports the end of the content only on evidence. The end of the content means the target is
not in the scrolled region, which is the whole screen or the container within names. scroll
reports it only once two consecutive reads show the content standing still, by any of these signs:
- An element the loop watched move is still there, has stopped, and belongs to the scrolling region rather than to chrome above it.
- The region's bounds cut nothing off, so no frame can be hiding motion.
- Where the tree can show neither, the screen as drawn did not change across the step either.
A tree of plain rows meets the second sign on the first step, so a typo in to fails at once there.
A tree that reports a window or root view spanning the screen never meets it, so on those backends
the screen as drawn is what fails fast, one step later. Where no evidence is available at all,
scroll keeps stepping and reports at maxScrolls (default 15) that it could not observe whether
the region moved. That failure makes a different claim from the list having ended, and the
difference is real: Android reports an element's bounds clipped to the part of it that is visible,
so a row taller than the screen reports the same frame while content scrolls behind it.
A step after which nothing that had been in view is on screen at all is the opposite error, because
the step may have carried the target past the viewport. Partly on screen counts as in view, so an
ordinary step on a screen showing one card at a time is not mistaken for an overshoot. On an
overshoot, scroll halves the step and scrolls back once to read the span that passed. When even the
smallest step it will take still leaves nothing behind, scroll fails naming the overshoot rather
than reporting the target absent.
amount sets how far one step travels, as a fraction of the viewport, greater than 0 and at most 1.
This is the same unit and range swipe and drag take for their own amount. Omitted, a step
covers 0.6 of the viewport. Lower amount for a screen the default step crosses too coarsely to land
on the target. Raise it for a screen that reveals so little per step that maxScrolls runs out
before the target appears.
amount decides where the loop starts and nothing else. The halving above still shrinks the step
from wherever amount put it, and still stops at the same floor, which does not move with amount.
An amount at or below that floor leaves the halving nothing to shrink, so the first step that
overshoots fails the call outright, naming the step it took.
A re-read, not a single query, settles whether a step moved the region. The re-read matters on
Android, where the accessibility tree arrives after the gesture has already moved the list. A read
taken meanwhile describes the pre-scroll screen, which looks like the end of the content. Confirming
costs Android's declared read budget, on the step that ends a failing scroll. Web and iOS pay
nothing, because their reads do not lag. within scopes the gesture (and every decision above) to one
scrollable container; omitted, the whole screen scrolls.
Use scroll to reveal a target, swipe for a fixed gesture, and drag to move a grabbed
handle. Each step is non-inertial: it advances a bounded, screen-relative distance and leaves no
momentum. The same scenario reaches a target identically on a fast device and a slow CI emulator,
which is the determinism a hand-tuned swipe chain cannot guarantee.
The distance a step travels is the distance it asked for, on every backend. The driver conformance suite measures one step's realized travel and fails a backend whose content carries on past the gesture's own endpoints. The look-back above catches a step that overshoots anyway, rather than assuming it away.
scroll'sdirectionis the direction the content moves, not the finger — the inverse ofswipe.scroll: { direction: down }reveals below-the-fold content (the driver swipes the finger up);swipe: { direction: up }is the finger going up. An author reaching forscrollthinks "scroll down the list", soscrollnames that;swipenames the finger.
doubleTap / pinch / rotate (gestures)¶
- doubleTap: { id: gest.doubletap } # two quick taps
- pinch: { sel: { id: gest.pinch }, scale: 2.0 } # >1 zooms in, 0<scale<1 zooms out
- rotate: { sel: { id: gest.rotate }, radians: 1.57 } # >0 clockwise (radians)
scale must be > 0; otherwise validation fails. pinch / rotate require multi-touch, which both
the iOS (XCUITest) backend and the generated XCUITest (pinch(withScale:) / rotate(_:)) provide.
A backend without multi-touch fails with a "needs multiTouch" reason. doubleTap runs everywhere
(two taps).
Real files: demos/showcase/scenarios/gestures.yaml for
doubleTap / longPress, and
demos/showcase/scenarios/gestures_multitouch.yaml
for pinch / rotate.
wait (condition wait)¶
A wait never sleeps for a fixed time (the one fixed pause is the separate sleep step). timeout is mandatory (no infinite waits).
- wait: { for: { id: home.title }, timeout: 5 } # until an element appears
- wait: { until: { gone: { id: home.spinner } }, timeout: 15 } # until an element disappears
- wait: { until: screenChanged, timeout: 5 } # until query() changes
- wait: { until: settled, timeout: 3 } # until the screen stops changing
- wait: { until: { request: { method: GET, path: /items, status: 200 } }, timeout: 8 } # until a matching request is observed
for and until are mutually exclusive. until is screenChanged / settled /
{ gone: <Selector> } / { request: <RequestMatch> }.
The request form polls the network collector (evidence, the --network run flag)
until at least one observed exchange matches. The matcher is the same as the
request assertion's: method / url / urlMatches / path /
pathMatches / status / bodyMatches, all AND-ed, with count raising the threshold. The
endpoint is pinned by url (exact full URL), by urlMatches (regex/substring), or by path alone.
Timeout handling differs by kind (run-loop). For
for / gone / screenChanged / request, a timeout is a step failure. settled is a
stabilization hint, so a timeout merely proceeds with the current screen and does not fail.
sleep (fixed pause)¶
A wait never sleeps for a fixed time. For a rare delay that nothing on screen or in the network log
shows, use sleep. Typical cases are a server-side retry throttle, or an animation that outlives
until: settled.
- sleep: { seconds: 2, reason: "the server throttles a retry for two seconds" }
secondsmust be above 0 and at most 30 seconds. A longer pause almost always hides a missing condition, so loading refuses it and points atwait.reasonis mandatory and must not be blank. It states why nowaitcondition can express the pause. The progress log andreport.htmlshow the reason, and generated code keeps it as a comment.- The step reads no tree and sends no input. It runs on every backend, and a cancelled run stops the pause within a quarter of a second.
bajutsu auditlists everysleepas afixed-sleepfinding. More than 10 seconds of fixed pause in one scenario grades itModerate.recordandcrawlnever offersleepto the model, andtriage --aiflags a fix that adds one. Scenarios authored through the MCP server are not guarded, so review asleepthere by hand.
assert (mid-step verification)¶
assert verifies mid-step. The DSL is the same as expect (next section), visual included. A
step's own assert can carry a visual entry, which takes its own fresh, single-shot screenshot
right there. The capture is scoped to that step's own evidence directory, so it never collides with
the scenario's expect-phase capture or with another step's. responseSchema is the one kind a
step's assert cannot evaluate: it always fails there with "no schema context", so keep it in
expect.
- assert:
- disabled: { id: auth.submit }
- tap: { id: modal.open }
- assert:
- visual: { baseline: modal.png } # verify the modal right after it opens
- tap: { id: modal.dismiss }
A step-level check's first baseline is not yet reachable from bajutsu approve or the serve UI's
Approve button. Both read merely the scenario's expect results today. Until that catches up,
promote the baseline by hand. The failing step's own assertion_results records a
visual.actual path in the run's manifest.json (the report's raw JSON view shows it too). Copy the
file at that path into the baselines directory, under the name the baseline: field gives.
setLocation / push (device control)¶
- setLocation: { lat: 35.681, lon: 139.767 } # simctl location set
- push: { payload: { aps: { alert: "You have mail" } } } # simctl push (APNs payload)
Both drive the Simulator via simctl and need a per-device control channel, so they are unavailable
on the fake driver and in parallel runs. There the step fails cleanly rather than crashing. push
delivers its payload as the APNs JSON to the app under test.
http (request, for test-data setup)¶
- http: { method: POST, url: "https://api.test/seed", body: '{"n":1}', status: 200 } # fails if status != 200
- http: { url: "https://api.test/token", saveBody: token } # vars.token ← response body text
- http:
method: POST
url: "${secrets.API}/login"
body: '{"user": "e2e"}'
status: 200
extractBody:
- { var: token, path: "data.token" } # vars.token ← the nested field, not the whole body
- { var: userId, path: "data.user.id" }
- assert:
- exists: { id: home.title }
http issues the request from the runner over HTTP, and does not go through the UI driver. A
status mismatch fails the step, and saveBody stores the response body text as
${vars.<name>} for later steps. Because it touches no device, http is the one device-independent
action here.
extractBody (BE-0440) sits beside saveBody on the same step, and a step may set either, both, or
neither. Each entry parses the response body as JSON, walks a path, and stores the value it finds
as vars.<var>. A path walks object keys and [n] (zero-based) array indexes:
- An opening key carries no leading dot (
data.token). - An index may follow a key or another index (
items[0].id,rows[0][1]). - A path may open on an index when the body is itself a JSON array (
[0].id).
A string value lands as-is. Any other JSON value (number/boolean/null/object/array) renders as
the same compact JSON text a captured event body field would (42, true, null, {"id":42}), so
a later ${vars.*} comparison reads the shape the API returned.
These cases fail the step:
- A response body that fails to parse as JSON fails with the JSON parser's own message.
- A
paththat does not resolve fails with an error naming thevarand thepath. A path does not resolve when it hits a missing key, an out-of-range index, a key applied to anything but an object, or an index applied to anything but an array. - A malformed
path(a negative or non-numeric index, an empty segment, a trailing dot) fails the same way.
path and var may themselves use ${vars.*} / ${secrets.*} substitution
(path: "items[${vars.i}].id"). A value substituted into a path must supply exactly one path
segment. An empty segment, or one carrying ., [, or ], fails the step, so a field name arriving
from a response body can never redirect the walk to a different field.
Two extractBody entries sharing one var, or an entry whose var equals saveBody, is a scenario
load error. When only a substitution reveals the collision, it is a step failure at run time. A
collision is never a silent overwrite decided by write order.
totp (two-factor one-time password)¶
- totp: { secret: "${secrets.TOTP_SEED}", into: { var: code } } # vars.code ← current 6-digit OTP
- type: { text: "${vars.code}", into: { id: auth.code } }
totp computes an RFC 6238 time-based one-time
password locally, from the shared secret (base32; keep it in ${secrets.*}, not in the YAML) and
the current time. It stores the current code in ${vars.<var>} for a later type / assert. This
automates a 2FA sign-in without a scripting escape hatch or an LLM, because the value is a
deterministic function of the secret and the clock
(BE-0046).
A code is valid for one 30-second window of the clock, so a totp step that lands in the last
seconds of a window can hand the app a code that expires before the following type and tap
submit it. Set minRemaining to the seconds those steps need:
- totp: { secret: "${secrets.TOTP_SEED}", into: { var: code }, minRemaining: 10 }
When the current window has fewer than minRemaining seconds left, the step waits for the next
window to start and then generates the code
(BE-0474). The value
is an integer from 1 to 30. Omitting it means no wait.
The wait ends on the window boundary the clock decides, never on an author-chosen pause. One window
bounds the wait, and the determinism audit does not list it as a fixed-sleep. The
live log labels the step totp (≥10s left), and the report shows the time the wait took as the
step's duration. If the wall clock jumps during the wait (a Network Time Protocol (NTP) correction,
say), the step still passes and records a warning on its report row that states the jump and the
seconds left. A later sign-in that fails on an expired code then shows its cause.
Fetch a fresh code later with otp¶
A scenario that enters a code more than once, or long after it starts, registers the secret once
and fetches each code at the point it needs one. Give the totp step a name, then name that
generator in an otp step's from:
- totp: { secret: "${secrets.TOTP_SEED}", name: main } # registers the generator; writes no code
- tap: { id: auth.change_password }
# ... any number of steps ...
- otp: { from: main, into: { var: code }, minRemaining: 10 } # vars.code ← the code current now
- type: { text: "${vars.code}", into: { id: auth.code } }
A totp step needs name, into, or both, so a step that only registers a generator omits
into. Registering a different secret under a name already in use fails the step. minRemaining
needs into on a totp step, since a step that produces no code has nothing to keep fresh. The
totp step checks the secret when it registers it, so a malformed secret fails that step rather
than a later otp.
The otp step reads the clock when it runs, so each otp writes the code of the window current at
that step. It takes the same minRemaining window wait as totp, and the live log labels it
otp (≥10s left). Registration follows execution order: an otp whose generator has not been
registered yet fails the step, as does one whose totp sat in an if branch that did not run. A
generator lasts for the rest of the scenario run, if, forEach, and group bodies included.
The run keeps a registered secret out of reach of every ${...} token, so no scenario can read it
back into a typed field. The otp step itself carries no secret, so its evidence shows only the
generator's name.
email (poll a mailbox for a received code)¶
- email:
match: { to: "test@example.com", subjectMatches: "verification" } # which message to wait for
extract: { var: code, bodyMatches: "[0-9]{6}" } # vars.code ← first capture group
timeout: 30
- type: { text: "${vars.code}", into: { id: auth.otp } }
email waits for a 2FA / verification code delivered by email. It polls a generic HTTP mailbox
(configured under targets.<name>.services.mailbox, see configuration)
until a message that arrived after the step started satisfies match. It then extracts the value
from the message body by the bodyMatches regex (first capturing group, or the whole match) into
${vars.<var>}.
The wait is a condition wait with a mandatory timeout (no fixed sleep). A timeout, a matched
message whose body the regex can't hit, or an unreachable / non-2xx mailbox is a clean step failure,
never a silent wrong value. Only mail newer than the step's start counts (keyed on message id, so a
stale code from an earlier run is never matched), and among new matches the newest wins. The step is
deterministic and LLM-free. The endpoint and credentials live in config-referenced ${secrets.*}, so
the scenario stays app-agnostic (BE-0046).
generate (a value computed at run time)¶
- generate: { random: { string: { length: 8, charset: alnum } }, into: { var: username } }
- type: { text: "${vars.username}", into: { id: signup.username } }
- generate: { random: { uuid: {} }, into: { var: orderRef } } # a version-4 UUID
- generate: { random: { int: { min: 1, max: 100 } }, into: { var: quantity } }
- generate: { random: { float: { min: 0, max: 50, precision: 2 } }, into: { var: amount } } # e.g. "12.30"
- generate: { datetime: { format: "%Y-%m-%d", offsetDays: 1 }, into: { var: tomorrow } }
- type: { text: "${vars.tomorrow}", into: { id: booking.date } }
generate computes a value in the runner and stores it as ${vars.<var>}. A scenario can then supply
an input its author could not write as a literal: a username no earlier run has taken, tomorrow's
date on a booking form, or a reference that collides with no other scenario's. Data-driven rows
(reuse) supply a fixed table chosen in advance, and extract captures a
value the app already displays; neither invents a value the scenario did not already have
(BE-0377).
Exactly one generator kind produces the value:
randomdraws astring(alengthof characters from acharset:alnumby default, oralpha/numeric/hex), anintin the inclusive range[min, max], afloatin[min, max]rounded to an optionalprecisionof decimal places, or a version-4uuid.datetimerenders the current time as text.formattakes astrftimepattern (ISO 8601 to the second when omitted). The signedoffsetSeconds/offsetMinutes/offsetHours/offsetDaysfields add together to shift the time.timezonetakes an Internet Assigned Numbers Authority (IANA) zone name such asAmerica/Los_Angeles.
The default zone is UTC, so a scenario whose input must match a date the app renders in the device's own zone names that zone explicitly. Pinning the device to a zone is a separate concern (BE-0158).
The flow is deterministic even though the value is not. A generate step the loader accepted always
executes and always succeeds, because the step draws from a generator or reads the clock, with no
network and no model. The produced value alone differs between runs, the same way totp's time-derived code already
does. A format that cannot be rendered and a timezone that does not resolve fail the load, so no
run ever substitutes a different value for one mid-flight.
The run records each produced value in the manifest and the report, so a later failure shows which
value the run actually used. A scenario that must check a specific value captures it through
${vars.*} and compares against that capture, not against a literal it could not have known in
advance. Every codegen target renders generate as a labeled // TODO, because the step runs in the
runner rather than the app.
eval (transform a value held in a variable)¶
- tap: { id: cart.open }
extract:
items: { sel: { id: cart.summary }, prop: label } # "Apple, banana ,Cherry"
- eval:
from: "${vars.items}"
ops:
- split: { by: "\\s*,\\s*", regex: true }
- filter: { notEmpty: true }
- sort
into: { var: sortedItems } # ["Apple","Cherry","banana"]
- eval: { from: "${vars.sortedItems}", ops: [json, { pick: first }], into: { var: firstItem } }
- eval:
from: "Taro Yamada"
ops: [{ split: " " }]
into: { vars: [givenName, familyName] } # givenName=Taro, familyName=Yamada
- eval:
from: "${secrets.API_TOKEN}"
ops: [{ substring: { start: 0, end: 8 } }]
into: { var: tokenPrefix } # masked in every later artifact
eval transforms a value the scenario already holds and stores the result under ${vars.*}. The
values a target shows are often not the values a later step needs. A cart label may need to become
sorted items. A banner reading Order #A-1042 confirmed may need to become A-1042. Without eval,
the author asserts the whole string or moves the logic out of the scenario
(BE-0472).
The name does not mean the step runs code. ops draws from a fixed vocabulary, and a scenario cannot
add an operation. from is a string with the usual ${...} interpolation. Joining two variables
needs no operation: from: "${vars.a}-${vars.b}" builds the joined string. Each entry of ops runs
on the result of the one before it. An operation without arguments is a bare name. Every other
operation is a mapping with one key.
| Operation | Form | Input → output | Behavior |
|---|---|---|---|
split |
split: "," or split: { by, regex? } |
string → list | splits on a literal separator, or on a regular expression when regex is true; capture groups in the pattern are not kept |
join |
join: "-" |
list → string | joins the elements; an empty separator is allowed |
filter |
filter: { matches \| equals \| contains \| notEmpty, not? } |
list → list | keeps the elements that satisfy exactly one predicate; not: true keeps the others |
pick |
pick: first, last, or an index |
list → string | takes one element; an index counts from zero |
count |
count |
list → string | the number of elements, as decimal text |
sort |
sort, sort: desc, or sort: { order, numeric? } |
list → list | orders by Unicode code point, or by decimal value with numeric: true; desc equals sort followed by reverse |
unique |
unique |
list → list | drops repeats and keeps the first of each |
reverse |
reverse |
list → list | reverses the order |
json |
json |
string → list | parses a JSON array; a non-string element renders as compact JSON text |
regex |
regex: { pattern, group? } |
string → string | capture group (default 0) of the first match |
replace |
replace: { find, with, regex? } |
string → string | replaces every occurrence |
substring |
substring: { start, end? } |
string → string | the characters from start up to end, both counted from zero |
trim |
trim |
string → string | removes leading and trailing spaces, tabs, and line breaks |
case |
case: lower or upper |
string → string | changes letter case |
The result goes to into:
into: { var }stores one value. The run stores a string as it stands. The run stores a list as compact JSON array text such as["a","b"]. A laterevalturns that text back into a list withjson.into: { vars: [...] }spreads a list result over the named variables in order. The first element goes into the first name, the second into the second, and so on. A list longer thanvarsleaves its extra elements unassigned. A shorter list stops the step: a variable left unset would surface later as an unrelated mismatch.
The loader checks the step before the run starts. It tracks the string-or-list type from from to the
end. An operation applied to the wrong kind stops the load with an error naming the operation. The
loader also refuses these forms:
- an empty
opsor an emptysplitseparator - a
filterwith no predicate or with two or more - a negative index or an unknown
sortorder into.varson a string result- a regular expression that does not compile
A free-text or index argument may hold a ${vars.*} or ${secrets.*} token. The loader cannot check
such an argument, so the run applies the same checks to it after substitution. The fixed-choice
arguments (sort's order and case) and the flags regex, not, and numeric take literal
values only.
The step stops at run time, with no fallback value, in these cases:
- a
regexfinds no match, or the pattern lacks thegroup - an index or a
substringbound falls outside the value jsonreads text that is not a JSON arraynumericsorting meets an element that is not a number- a
${...}token has no value bound - a value exceeds 1,048,576 characters or 10,000 elements
The message names the operation's position and never quotes a value. A step that stops this way writes
no variable. The run records the produced value in the manifest and the report, as for generate.
A result derived from a secret is itself a secret. An input counts as secret when it holds a
${secrets.*} token. A ${vars.*} token that an earlier eval marked secret counts too. The run
masks the result in that scenario's evidence from the step on, its own report row included. The run
also marks the target variables secret. count is the one exception: an element count carries no character of its input.
Evidence written before the step stays as written. The step's failure reasons and produced values
are masked too, so the manifest, the reports, and CI annotations never show the value.
Every codegen target renders eval as a labeled // TODO, because the step runs in the runner.
manual¶
A human takeover recorded during record.
- manual: { label: "solve the login CAPTCHA" } # no deterministic equivalent (a real CAPTCHA)
- manual: { label: "grant Face ID", bypass: "device-control biometric match (BE-0052)" } # names the bridge an author could wire
record emits a manual step when a blocker is an operation the AI cannot perform: a CAPTCHA, a
biometric prompt, or a gesture the agent repeatedly fails to resolve. The human operates the live
device and hands control back (the acted handoff,
recording). The step records a marker of the
observed transition, not the raw gesture.
bypass, when set, names the test-build flag or the device-control / device-state primitive
(BE-0035 / BE-0052) an author could wire to make the step replayable. Omitted, it marks a takeover
with no such equivalent (a real CAPTCHA). Every codegen target renders the step as a labeled
// TODO.
A manual step is never a silent pass. It has no deterministic run-time equivalent, so at run
time it fails loudly with ManualStepRequired, surfacing label and the bypass hint (directives 1
and 2). The author's path to a replayable scenario is to wire the named bypass and then replace the
manual step with the deterministic action
(BE-0185).
Device & system control¶
- background: {} # Home button (backgrounds via SpringBoard, no terminate)
- foreground: {} # bring the app to the front, launching it if it is not running
- clearKeychain: {} # reset saved passwords / certificates
- clearClipboard: {} # clear the pasteboard
- setClipboard: { text: "COUPON123" } # seed the pasteboard (paste flows)
- overrideStatusBar: { time: "9:41", batteryLevel: 100, wifiBars: 3 } # freeze the status bar
- clearStatusBar: {} # restore the live status bar
On iOS, these steps drive the Simulator through simctl, like setLocation / push. They need a
per-device control channel and fail cleanly on the fake driver / in parallel runs. On Android, every
step in this list runs except clearKeychain, because Android has no keychain.
Use overrideStatusBar right before a screenshot or a visual assertion. It freezes the clock and
signal bars for a stable image.
On Android, adb drives these steps with no code inside the app (BE-0478):
| Step | Android mechanism |
|---|---|
background |
presses Home (input keyevent KEYCODE_HOME); the app keeps its process |
overrideStatusBar |
enters the System UI demo mode and sends one command per set field |
clearStatusBar |
leaves the demo mode, restoring the live status bar |
background waits, on a condition, until the app no longer holds the resumed activity. When the app
stays in front, the step fails. After background, the launcher is in front. A selector for the app
under test then finds nothing, as on iOS while the Home Screen is in front. Inside a device group, a
read fails with AppNotInFront instead, until a foreground step brings the app back.
The demo mode differs from simctl status_bar in three ways:
- Unset fields freeze too. Entering the demo mode freezes every icon. A field the step leaves unset takes the demo mode's own fixed value, not the live one. The step also hides the notification icons, which the iOS status bar never shows.
- Values are narrower. The table below lists what Android accepts. Preflight fails any other value before device work and names the step. System UI would ignore such a value, and the broadcast would still report success.
- Wi-Fi bars scale. Android shows Wi-Fi levels 0–4, so a full iOS signal must stay full. The table below gives the mapping.
| Field | Android accepts | Mapping |
|---|---|---|
time |
H:MM or HH:MM |
the clock as four digits (9:41 becomes 0941) |
batteryLevel |
0–100 | unchanged |
batteryState |
charging, charged, discharging |
plugged is true for the first two |
cellularBars |
0–4 | unchanged |
wifiBars |
0–3 | levels 0, 1, 3, and 4 for 0, 1, 2, and 3 |
On both platforms the override persists on the device until a clearStatusBar step runs.
background / foreground are the two halves of a background/foreground transition.
foreground terminates nothing. Whether foreground waits depends on the case:
- On iOS outside a device group, a running app resumes with no wait at all. Wait for a concrete element afterward when a step needs one.
- An app that is not running, as after an
installApp, launches the wayrelaunchwould launch it.foregroundthen waits on a condition until the app is ready. - On Android, and on iOS inside a device group, a running app gets the same condition wait. When the
app never reaches the front, the step fails with
AppNotInFront. The wait looks for the app's own screen, not the target'ssubject.startWhen, becausesubject.startWhennames the screen a launch lands on, and a resumed app comes back on whatever screen it left.
setClipboard seeds the pasteboard for a paste flow (BE-0052).
targets / target (multi-target scenarios, BE-0428)¶
A scenario's top-level targets field names every target it
drives. For example, an iOS target and a web target may be two clients of the same service. One
scenario can act on one target and then check the result on another, and the two interleave freely
in a single deterministic run with one verdict. Each entry names a targets.<name> config unit, the
same unit --target already resolves. Naming the same target twice is a load error. A scenario that
declares no targets (the default) keeps today's behavior unchanged, and every step in it must
still omit target.
bajutsu run launches every declared target before the scenario's first step and tears them all
down after the last one. No step waits for its target to launch, so the interleaving above costs no
extra startup. Each declared name resolves against the one config file the invocation loads, so both
targets must appear side by side as targets.<name> entries there. Merging two config files is not
supported (configuration).
Under two or more targets and no primaryTarget (below), every step must set its own target.
That includes an if / forEach / web wrapper, not merely a leaf action. Every top-level
expect entry must set one too, each naming one of the declared targets:
- name: liking a post on the app shows up on the web
targets: [showcase-app, showcase-web]
steps:
- target: showcase-app
tap: { id: post.like }
extract:
postId: { sel: { id: post.id } }
- target: showcase-web
wait: { for: { id: "post.${vars.postId}.likeCount" }, timeout: 10 }
expect:
- target: showcase-web
value: { sel: { id: "post.${vars.postId}.likeCount" }, equals: "1" }
A scenario built around one target can declare that target as its primaryTarget (BE-0436).
A step or top-level expect entry may then omit target, and it runs against the primary. A step
names a target when it acts on another one. A reader spots those steps at a glance:
- name: liking a post on the app shows up on the web
targets: [showcase-app, showcase-web]
primaryTarget: showcase-app
steps:
- tap: { id: post.like } # target omitted: runs on showcase-app
extract:
postId: { sel: { id: post.id } }
- target: showcase-web
wait: { for: { id: "post.${vars.postId}.likeCount" }, timeout: 10 }
expect:
- target: showcase-web
value: { sel: { id: "post.${vars.postId}.likeCount" }, equals: "1" }
primaryTarget must name the first entry of targets. The runner already treats that entry as
the primary for leasing, recovery, and evidence. Pinning the field to it keeps the file and the
runner agreeing on one primary. Reordering targets without updating primaryTarget fails at load
time. Setting primaryTarget on a scenario that declares no targets fails at load time too.
With one declared target, primaryTarget may name that target and changes nothing.
The default applies at every nesting depth. A nested step inside an if or forEach may omit
target, and it then runs against the primary, whatever target its wrapper names. A step may still
name the primary explicitly, and both spellings behave the same way. The loader never writes the
resolved name back into the step, so the serve editor and a run's scenario.yaml snapshot keep the
step as terse as its author wrote it.
A self-declaring scenario resolves its own targets, so bajutsu run --scenario <file> needs no
--target at all. An explicit --target passed beside such a scenario must name a target the
scenario declares. A stale flag left over from editing the file stops the run and never picks a
target the file dropped. The command-line reference has the full
rule, including which invocations still require --target.
With one declared target, a step may omit target. A step may also name that target directly, and
both spellings behave the same way.
A scenario declares nothing about the machine that runs it. Bajutsu computes what the scenario
needs from the scenario itself. The scenario needs the driver each declared target resolves to. It
also holds one device per declared target, counted across every driver. One Simulator target and
one browser target count two. Before any device work, bajutsu run compares those needs with the
machine's worker capability file. It also compares
them with the host operating system
(BE-0450).
A scenario this machine cannot run fails with a reason that names the driver or the limit. That
reason starts with worker-capability-unsupported:. The other scenarios run.
${vars.*} carries across every declared target in one run. These are the same
runtime variables extract populates. A value one target's step captures
is readable from an assertion against another target. A cross-platform check can thus name the
record each side touched, not merely a changed count.
A step nested inside a web or app block is the one exception. It must omit target
outright. It always runs against the device the enclosing block already resolved. That is web's
own WebView bridge, or app's unchanged native driver.
Target groups: naming one target once for a run of steps (BE-0437)¶
The target rule above repeats the same value across a whole run of steps that act on one target.
A target group removes that repetition. It sets target once. It lists every nested action
under its own steps, instead of stamping target on each one by hand:
- name: favorite a horse on the iOS showcase, then carry what it captured into the web demo
targets: [showcase-swiftui, web]
steps:
- target: showcase-swiftui
steps:
- wait: { for: { id: [stable.row.1, stable_row_1] }, timeout: 10 }
- tap: { id: [stable.row.1, stable_row_1] }
- wait: { for: { id: [horse.favorite, horse_favorite] }, timeout: 5 }
- tap: { id: [horse.favorite, horse_favorite] }
extract:
favorited: { sel: { id: [horse.favorite.value, horse_favorite_value] } }
- target: web
steps:
- tap: { id: onboarding.start }
- type: { text: "favorited-${vars.favorited}@example.com", into: { id: auth.email } }
- type: { text: "pw", into: { id: auth.password } }
- tap: { id: auth.submit }
- wait: { for: { id: home.title }, timeout: 5 }
- tap: { id: counter.increment }
expect:
- target: showcase-swiftui
value: { sel: { id: [horse.favorite.value, horse_favorite_value] }, equals: "on" }
- target: web
value: { sel: { id: counter.value }, equals: "1" }
Written the flat way, this same scenario needs ten target: lines to name the same two runs. Each
group above names its target once. No target: line drifts out of sync with the one above it.
A target group is pure authoring sugar. A load-time pass expands it before the rest of this
section's rules run. It produces the same flat, per-step target: form a hand-written scenario
already uses. bajutsu run, the report, and the CLI never see a target group. They see the
expanded steps.
A target group always requires its own target, whatever the scenario declares. This holds even
for a scenario declaring zero or one target, and even for one declaring a primaryTarget (above). A
leaf action differs: it may omit target and run against the primary. A group's purpose is fixing a
target for its own nested steps, so it always needs one.
A target group also refuses capture, extract, name, and from. Each of those four reads off
the one step the runner executes, and a target group never reaches the runner: expansion first
replaces it with its own nested steps. Expansion would otherwise drop any of those four with no
warning.
Every step nested directly inside a group's own steps must omit target. This covers a leaf
action, an if, a forEach, and a web/app block alike. The group already fixed it. A child
that sets one anyway hits a load-time error, not a silent override. A target group nested directly
inside another one hits the same error. A nested group always sets its own target. An immediate
child may never do that.
The omission is shallow, not recursive. An if or forEach nested directly inside a group
inherits the group's target for itself. A leaf action inherits it the same way. But the steps
inside its own then, else, or body form a fresh scope. Each still needs its own explicit
target or a target group of its own. That holds the same as outside any group. A web or app
block nested directly inside a group inherits the group's target the same way. It opens its bridge
against that target. The steps nested inside that block keep omitting target under the existing
rule above. That rule has nothing to do with the group.
A web: or app: block's own nested steps refuse a target group outright. Every step there
already runs against one target. The block itself already resolved that target. A group there
would merely repeat that target for no purpose. Or it would claim a different one with no driver
open for it.
What a multi-target run does with the rest of a target's config¶
Each declared target keeps its own config, not the primary target's. Resolution per target:
| Reads that target's own config | Why |
|---|---|
hooks.setup / hooks.cleanup lifecycle steps |
Each target's own hooks fold in, in declared order, stamped with that target's name — so an app-side teardown and a web-side one each run against the right driver |
services.mailbox |
An email or totp step polls the inbox its own target configured |
runsOn.ios.locale, subject.launch.env, evidence.capture, hooks.interrupts |
Each is a property of the app under test, not of the run |
paths.baselines / paths.schemas / paths.goldens |
A visual or golden assertion compares against its own target's directory when that target configures one, else the run's |
| Backend capabilities | Each target's steps are checked against its own backend before any device is leased, so a construct only one platform supports fails the right one |
A nested if / forEach step from a target's own hooks.setup / hooks.cleanup list omits target
too. It resolves to that config's own target, never to the scenario's primaryTarget. The flat,
every-nesting-depth default above applies to a scenario's own steps alone.
Two run-wide values stay shared. The redact secret set unions every declared target's own
evidence.secrets. That union scrubs every target's evidence, since scrubbing too widely is the safer error.
The run directory is a run-level artifact, never per target.
The web engine flags still apply to a run's single web target. Those flags are --headed,
--browser, and --browsers. A scenario naming two web-platform targets refuses all three. Which
of the two they mean has no answer. Extending the matrix across a multi-target run is separate
work.
Device groups: several apps on one device (BE-0447)¶
Each declared target holds a device of its own by default. Some journeys need two apps on one device instead:
- An app update installs a new build over an old one. The scenario then checks that the old data survived.
- A companion app runs beside the app under test and hands it a code. A multi-factor authentication (MFA) app is one example.
A device group expresses both. A device group is an entry of targets that holds two or more
names. Its members share one device.
The config does not change. Two builds of one app become two targets. They share a subject.id (a
bundle identifier or an Android package), and each has its own subject.path. A companion app is a
target of its own:
targets:
showcase-previous: # the old build the scenario starts on
platform: ios
subject: { id: com.example.showcase, path: build/Showcase-1.app }
showcase: # the current build
platform: ios
subject: { id: com.example.showcase, path: build/Showcase-2.app }
A bare name in targets stays a group of one. A scenario that writes no array runs as before. The
loader enforces three rules on the list:
- A name appears once across the whole list, nested or not.
- An array holds two or more names. Write a single target as a bare name.
primaryTargetnames the first member of the first group.
Which members start: installs¶
Not every member of a group belongs on the device at the start. An update scenario starts on the old
build and holds the new one back. The top-level installs list names the members that start beside
the primary:
| Member | At the start |
|---|---|
| The primary (the first member of the first group) | Installs its subject.path and launches. Listing it in installs is optional |
| A group of one (a bare name) | Installs and launches, as in any multi-target scenario |
A member listed in installs |
Installs and launches |
| Any other member of a group of two or more | A later member: nothing installs or launches until an installApp step and a foreground step bring it up |
A group outside the primary's must list at least one of its members in installs. The loader then
never guesses which member starts.
Device-wide preparation runs once per group. That covers the erase, the system-locale pin, and the seeded photos. A second starting member installs without repeating the erase, so it never wipes the first.
Installing a later member: installApp¶
installApp: { from: <target> } installs the build of the later member from. The build lands on
the device of the step's own target. A step that omits target installs on the primary's device. The step
first terminates any running app with the same identifier. Then the build installs:
- With
keepData: true, the default, it installs over the existing build. The data container survives. - With
keepData: false, the step uninstalls the identifier first.
The step launches nothing. A foreground step addressed to the member launches the build.
The loader refuses an installApp in these cases:
fromnames a starting member.frombelongs to a group other than the one holding the step's device.- A second top-level
installAppnames the same member. A member installs once per scenario. - The step sits inside a
web:orapp:block, which never picks a device. - The step sits in an
interruptsrecovery and names notarget, on the step or on its entry. This case applies once the scenario contains asetPrimaryTarget.
A loop or a recovery that reaches an installApp a second time fails at run time by name. The
scenario names a target, never a path, so the build artifact stays in config.
Moving the default target: setPrimaryTarget¶
setPrimaryTarget: { target: <target> } moves the default target. From that step on, a step that
omits target runs on the named target. The same holds for an interrupts entry and a top-level
expect entry.
The step changes routing and nothing else. Leasing, the evidence directories, and crash recovery stay
with the declared primary. In the setup and cleanup phases, an omitted target still resolves
to the declared primary, since teardown has to run no matter where the run stopped.
setPrimaryTarget belongs among a scenario's top-level steps alone. The loader refuses it inside
these constructs:
iforforEach- a target group,
web:, orapp: - an
interruptsentry setuporcleanup
That placement lets the loader follow the current primary through the steps in order. A component
may contain a setPrimaryTarget when its use: or group: call is a top-level step. Expansion never
stamps a caller's target onto it.
Switching between apps, and retired members¶
A device shows one app at a time, so the scenario switches explicitly. A foreground step addressed
to a member brings that member's app to the front. When the app is not running yet, foreground
launches the app and waits until the app is ready. The runner never switches apps on its own between steps.
A device can hold two or more members. Each member's driver then checks that its own app is in
front. A step for a member whose app sits behind another fails with AppNotInFront. That error names
the cause and points at foreground. The step never resolves a selector against another app's tree.
A later member moves through three states. A step addressed to it fails by name unless the state allows that step:
| State | Steps that may address the member |
|---|---|
| Not installed | installApp and setPrimaryTarget |
| Installed, not launched | foreground and setPrimaryTarget |
| Running | Any step |
An install retires every member that shares the build's identifier. In an update scenario, the old build's member retires when the new build installs over it. A step addressed to a retired member fails by name and never reaches the new build. A companion app has another identifier. Its install adds a second app beside the first and retires nothing.
What run checks before leasing a device¶
run checks every device group against the config before it acquires any device. On a refusal,
run exits with status 2. The check refuses these cases:
- Members of one group differ in platform, backend, device route, or effective
locale. The device route is the device provider (anappiumenvironment named by--worker-config) andrunsOn.ios.kind. - A web target sits in a group of two or more, since a browser has no device to share.
- Two starting members of one group share an identifier. The second install would replace the first.
- An
installApp.fromtarget has nosubject.path, or itssubject.pathdoes not exist. A Git-sourced config builds the binary on demand, as it does for the primary. - A
setPrimaryTargetnames a member that an earlierinstallAppretired.
Some device providers hand a device over with the app preinstalled. Once it reserves such a device,
run refuses a group whose member comes from that provider. The provider holds the binary, so no
local build can install beside it.
The iOS Simulator (XCUITest) and Android (adb) backends support device groups. A physical iPhone
does not. Code generation emits a labeled // TODO for installApp and setPrimaryTarget.
Two examples¶
An update scenario starts on the old build and creates data. It installs the new build over the old one and checks that the data survived:
- name: notes survive the update
targets: [[showcase-previous, showcase]]
primaryTarget: showcase-previous
steps:
- tap: { id: notes.add }
- type: { text: hello, into: { id: notes.field } }
- installApp: { from: showcase } # retires showcase-previous
- setPrimaryTarget: { target: showcase }
- foreground: {} # launches showcase, the primary now
- wait: { for: { id: notes.item }, timeout: 10 }
expect:
- exists: { id: notes.item, label: hello } # resolves to showcase
A companion scenario starts the authenticator beside the app on one device. A web client keeps a device of its own:
- name: read the code in the authenticator, enter it in the app
targets:
- [showcase, authenticator] # one device holding both apps
- showcase-web # a device of its own
primaryTarget: showcase
installs: [authenticator]
steps:
- target: authenticator
foreground: {}
- target: authenticator
tap: { id: code.show }
extract:
code: { sel: { id: code.value } }
- foreground: {} # back to showcase
- type: { text: "${vars.code}", into: { id: login.otp } }
- target: showcase-web
wait: { for: { id: session.active }, timeout: 10 }
The showcase demo runs both journeys on iOS and on Android (real files:
demos/showcase/scenarios/device_group_update.yaml,
demos/showcase/scenarios/device_group_companion.yaml).
There, the update reads back a note saved on a screen that persists it across installs, and the
companion is the other toolkit's showcase app, which has a bundle id of its own.
Limits¶
Assertion.target follows a narrower rule than Step.target. A top-level expect entry is the sole
place it may appear. An assertion reached through a step's inline assert: list already has a
target. So does one reached through an if's condition. The enclosing step's own target fixed
it in both cases. Setting target there would restate that value, or contradict it outright. The
loader refuses an interrupts entry's condition the same way. The entry's own target fixes the
tree that condition polls. The loader refuses target in all three places at load time.
Two targets on the same backend share one device pool. A scenario holds one device per device group
for its whole length. A bare name counts as a group of one. Two ungrouped iOS targets thus need
two devices: pass them with --udid a,b, one device per group. run refuses the invocation up
front when a pool has fewer devices than a scenario needs. Blocking on a queue that will never free
one would be the alternative. run also caps --workers, keeping concurrent scenarios from starving each other the same way.
What the report shows¶
Each step's row in the report names the target that ran it. A scenario declaring no targets leaves
that label empty, as before. manifest.json carries the same name per step. It also carries one
target_devices row per declared target. Each row holds that target's backend, device, model, and
OS version.
Four singular fields stay empty on a multi-target run. They are backend, device,
device_name, and device_runtime. None of them describes the whole scenario any more. That matches the "empty means
not applicable" rule engine uses. A single-target run's reader sees all four as before.
Assertion DSL¶
Shared by expect (final verification) and assert (mid-step). Items in the list are all
AND-ed; one failure fails the step. The evaluation mechanics (element resolution, comparison)
are in selectors.
| Assertion | Meaning | Example |
|---|---|---|
exists |
a matching element exists (negate: true checks absence) |
exists: { id: home.title } / exists: { id: settings.banner, negate: true } |
value |
accessibility value match | value: { sel: { id: counter.value }, equals: "2" } |
label |
label exact / substring / regex | label: { sel: { id: settings.status }, contains: "done" } |
count |
number of matching elements | count: { sel: { idMatches: "list.row.*" }, equals: 5 } |
enabled / disabled |
actionable or not (the notEnabled trait) |
disabled: { id: auth.submit } |
selected |
selected / toggled state (the selected trait) |
selected: { id: tab.home } |
request |
a matching network exchange was observed (needs --network) |
request: { method: POST, path: /login, status: 200, count: 1 } |
event |
an analytics / telemetry event was sent — endpoint + JSON body fields, with a count (needs --network) |
event: { url: "https://t.example.com/track", body: { name: purchase_completed }, count: { equals: 1 } } |
requestSequence |
matchers were observed in this order (needs --network) |
requestSequence: [ { urlMatches: "/auth/refresh" }, { urlMatches: "/api/account" } ] |
responseSchema |
a captured response body conforms to a JSON Schema (needs --network) |
responseSchema: { request: { urlMatches: "/api/items" }, schema: items.json } |
visual |
the screen matches a baseline image (visual regression) | visual: { baseline: home.png, threshold: 0.02 } |
clipboard |
the device clipboard matches (read back via simctl pbpaste on iOS, the BajutsuAndroid receiver on Android) |
clipboard: { equals: "COUPON123" } / clipboard: { matches: "\\d{6}" } |
attribute |
an element's HTML attribute matches, or is absent (web only) | attribute: { sel: { id: docs.link }, name: href, equals: /docs } / attribute: { sel: { id: panel }, name: aria-busy, absent: true } |
existswrites its selector inline ({ id: ... }directly).negateis optional.value/labeltakesel:+ exactly one ofequals/contains/matches.counttakessel:+ exactly one ofequals/atLeast/atMost.enabled/disabled/selectedtake a selector inline.requestmatches an observed network exchange (details below); needs the--networkrun flag.eventmatches an analytics / telemetry event the app sent (details below); needs the--networkrun flag.requestSequencechecks a list of request matchers were observed in order (details below); needs the--networkrun flag.responseSchemavalidates a captured response body against a JSON Schema (details below); needs the--networkrun flag.visualpixel-compares a screenshot against a baseline image (details below).clipboardreads the device clipboard (simctl pbpasteon iOS; on Android, the BajutsuAndroid in-app receiver, BE-0233) and checks exactly one ofequals/matches(regex). The assertion is the read-back half ofsetClipboard, for verifying a "copy" action. It needs the per-device control channel, so it is unavailable on the fake driver and in parallel runs, and fails cleanly there (BE-0052).attributetakessel:+name:+ exactly one ofequals/contains/matches/absent: true. See details below and BE-0466. The web (Playwright) backend reads the attribute on demand. Any other backend fails it before the device lease.
Locale caveat: string comparisons on
label/valueand assertions that look at visible text break under translation. Write these against the target's fixedrunsOn.ios.locale, and write the selector itself byid.
request (network assertion)¶
request asserts that the run's network collector observed a matching HTTP exchange (needs the
--network run flag and BajutsuKit in the app). The same matcher backs the until: { request: ... }
wait and mocks (below). At least one match field is required; the listed fields are AND-ed.
| Field | Type | Description |
|---|---|---|
method |
str | HTTP method (GET, POST, …) |
url |
str | Exact full URL (the endpoint) |
urlMatches |
str | Regex / substring over the full URL |
path |
str | Exact path (query ignored) |
pathMatches |
str | Regex over the path |
query |
map | Key/value dictionary over the URL's query parameters (below) |
body |
map | Key/value dictionary over the JSON request body (below) |
status |
int | Response status code |
bodyMatches |
str | Regex / substring over the raw request body |
count |
int | Number of matching exchanges — exact for the assertion, a lower bound for the wait |
- assert:
- request: { method: POST, path: /login, status: 200, count: 1 }
- request: { urlMatches: "/search", bodyMatches: "apple" } # match on the request body
countis not a match field — at least one ofmethod/url/urlMatches/path/pathMatches/query/body/status/bodyMatchesmust be present. (real file:demos/showcase/scenarios/network_mock.yaml)
Key/value dictionaries (query and body)¶
A regex over the URL or the body breaks on a harmless change. Reordered parameters break
urlMatches: "page=2&sort=asc". A pretty-printed JavaScript Object Notation (JSON) body breaks
bodyMatches: '"id":\s*42'. The query and body dictionaries name keys instead, so order and
formatting no longer matter
(BE-0463).
expect:
- request:
method: POST
path: /api/orders
query: { page: "2", sort: { $matches: "^(asc|desc)$" } }
body:
user: { id: "42", email: { $exists: true } }
items: [ { sku: A-1, qty: "1" } ]
coupon: { $exists: false }
querychecks the URL's query string. Keys and values are URL-decoded (+reads as a space). A key with an empty or missing value (?debug,?page=) holds the value"". A repeated key matches when any occurrence satisfies the value.bodychecks the request body parsed as JSON, recursing into nested maps and lists. A non-JSON, non-object, or absent body fails everybodycriterion.
Each value takes one of four forms:
| Form | Example | Holds when |
|---|---|---|
| Scalar | id: "42" |
The text form of the JSON value equals the string ("42" matches the number 42; booleans and null read as true / false / null) |
| Operator map | id: { $matches: "^4" } |
Every operator holds: $matches searches the text form with a regex, and $exists: true / false requires the key present / absent |
| Plain map | user: { id: "42" } |
The JSON value is an object, and each listed key holds; unlisted keys are ignored |
| List | tags: [a, b] |
The JSON value is an array of the same length, each element holding against the matcher at its index |
query takes the scalar and operator-map forms alone, because a query string has no nesting.
The scenario fails to load when a dictionary breaks the grammar:
- An unquoted scalar. The loader refuses a value such as
qty: 1orok: true. Quote it (qty: "1"). An unquoted number would never equal the text it is compared with. - An empty dictionary.
query: {}orbody: {}would match every exchange. - A mixed or unknown operator map. The loader refuses a map that mixes
$keys with plain keys. It also refuses any operator other than$matchesor$exists. - A bare operator at the top. The top-level keys of
queryandbodyalways name fields.
To check a field whose own name starts with $, double the prefix: $$ref checks the field $ref.
Suppose the assertion does not hold, and one exchange alone satisfies the endpoint and method
criteria. The message then names the first key that did not hold, such as
query.page: expected "2", got "3". Otherwise the message keeps the plain count. That holds with no
candidate, with more than one, and with count set.
query and body work wherever this matcher does. That covers request, event,
requestSequence, responseSchema.request, and until: { request }. Values accept ${vars.*} /
${secrets.*} tokens. mocks refuse both fields for now. The in-app matcher would ignore them and
stub more traffic than the scenario says.
event (analytics event assertion)¶
event asserts on a behavior the screen never shows: an analytics / telemetry event the app sent
(BE-0048).
The check is pure, over the same observed exchanges request reads (needs the --network run flag),
so the verdict stays machine-only, with no LLM. The assertion filters the timeline by the event's endpoint (the
same method / url / urlMatches / path / pathMatches matcher as request), then by its
query / body dictionaries, and checks how many exchanges survive against a count operator.
| Field | Type | Description |
|---|---|---|
method / url / urlMatches / path / pathMatches |
str | Endpoint matcher (AND-ed), same meaning as request |
query |
map | Key/value dictionary over the URL's query parameters, the same grammar as request (dictionaries) |
body |
map | Key/value dictionary over the JSON request body, the same grammar as request. The flat form compares top-level fields as text (so amount: "300" matches the JSON number 300); nested maps, lists, $matches, and $exists work too (dictionaries) |
count |
object | Expected multiplicity — exactly one of equals / atLeast / atMost. Omitted means at least one |
expect:
# the purchase event fired exactly once with the right amount
- event:
url: "https://t.example.com/track"
body: { name: purchase_completed, amount: "300" }
count: { equals: 1 }
At least one of an endpoint field,
query, orbodymust be present, so an event always pins something. A non-JSON, non-object, or absent request body matches nobodycriterion (it fails rather than guessing). Dictionary values support${vars.*}/${secrets.*}tokens like the rest of the DSL. Write a top-levelbodykey that starts with$as$$, since a single$marks an operator.
requestSequence (ordered requests)¶
requestSequence asserts that several requests happened in a given order, for example a token refresh
before the protected call (BE-0048).
The check is pure, over the observed timeline (needs the --network run flag), so the verdict stays
machine-only. The assertion takes a non-empty list of request matchers (the same
fields) and matches them as an ordered subsequence: each matcher must match a distinct exchange at
a strictly later position than the previous one. Unrelated traffic may interleave between them, so
the check is robust to noise; listing the same matcher twice requires two occurrences in order.
expect:
- requestSequence:
- { method: POST, urlMatches: ".*/auth/refresh" }
- { method: GET, urlMatches: ".*/api/account" }
Each matcher uses the same fields as
request(method/url/urlMatches/path/pathMatches/status/bodyMatches); a matcher's owncountis ignored here, since the sequence's job is order. For a pure multiplicity check, userequestwithcount.
responseSchema (JSON Schema of a response)¶
responseSchema asserts that a captured response body conforms to a JSON Schema — a contract
check the screen cannot express (BE-0048).
The check is pure and deterministic, over the observed timeline plus a stored schema file (needs the
--network run flag), so the verdict stays machine-only. request selects the exchange (the same
matcher fields) whose response is validated; schema is a file path resolved within the target's
schemas directory (--schemas flag, config targets.<name>.paths.schemas, or schemas/ beside the
scenario). Validation uses the jsonschema library — install the schema extra
(pip install bajutsu[schema]).
expect:
- responseSchema:
request: { method: GET, urlMatches: ".*/api/items" }
schema: items.json # resolved within the schemas dir
The assertion validates the first matching exchange's response. It fails (rather than guessing) when no exchange matches, the schema file is missing, the response has no body or is not JSON, or the body does not conform. The schemas dir resolves like
--baselinesforvisual.
visual (visual regression)¶
- assert:
- visual: { baseline: "home.png", threshold: 0.02, exclude: [{ x: 0, y: 0, w: 390, h: 47 }] }
- visual: { baseline: "detail.png", compare: pixelmatch, colorTolerance: 0.1, antialiasing: true }
- visual: { baseline: "summary-card.png", element: { id: "summary-card" } } # one element only
- visual: { baseline: "home.png", exclude: [{ selector: { label: "last updated" } }] } # mask by element
visual captures a screenshot and compares it against baseline (a PNG resolved inside the run's
baselines dir — --baselines, or baselines/ beside the scenario).
The comparison engine is selectable via compare (BE-0165):
| Engine | Description | Default |
|---|---|---|
exact |
Pixel-perfect — any channel difference counts as a changed pixel. | Yes (backward-compatible) |
pixelmatch |
Perceptual YIQ color distance with anti-aliasing detection. Tolerates sub-pixel rendering noise and one-pixel edge shifts. | No |
When compare is omitted, the engine falls back to the target's evidence.visualCompare config
(under defaults.evidence or targets.<name>.evidence), and then to exact.
threshold is the allowed percentage of differing pixels (default 0.0 = exact match), shared
by all engines. colorTolerance (0–1, default 0.1) sets the per-pixel perceptual color
tolerance for pixelmatch; antialiasing (default true) discounts anti-aliased pixels from
the diff. exclude masks regions before comparing, such as a status bar or a clock. Each entry is
either a rectangle in screenshot pixels ({ x, y, w, h }) or a { selector: <Selector> }
that names an element to mask (BE-0171); the element is resolved to its frame at evaluation time.
A baseline is created or updated with the approve command
(cli) or the serve UI; a missing baseline fails the assertion. Pair it with
overrideStatusBar to keep the clock / battery deterministic. Diffs are surfaced in
report.html; for pixelmatch, only the surviving (non-discounted) pixels appear in the diff.
approve and the report's diff strip read merely the scenario's trailing expect results. A
visual entry inside a step's own assert needs its first baseline promoted by hand instead (see
assert above).
Element-scoped comparison (BE-0171). By default visual compares the whole screen, so any
unrelated change (a banner, a list that grew a row) fails the check and churns the baseline. Give
element: <Selector> to compare only that element: the screenshot is cropped to the element's
frame and the baseline is that crop, so the check ignores everything outside it. The selector is
resolved with the usual unique-resolution rules — an ambiguous selector fails immediately
rather than cropping the first match, and a selector matching nothing fails too. approve promotes
an element-scoped baseline exactly as it does a whole-screen one (the baseline is simply a smaller
image).
Selector-based masking (BE-0171). A pixel rectangle in exclude drifts the moment the layout
reflows or the device resolution changes. Naming the element instead — { selector: { label:
"last updated" } } — is stable across those changes: the element is resolved to its frame and
masked exactly as a rectangle is. A mask selector that matches nothing is a no-op (there is nothing
on screen to hide); an ambiguous one fails, consistent with the determinism rule. Selector masks
and rectangles can be mixed in one exclude list, and both work with an element-scoped comparison
(a mask inside the cropped element is translated into the crop's coordinates).
attribute (HTML attribute of an element)¶
- assert:
- attribute: { sel: { id: docs.link }, name: href, equals: /docs }
- attribute: { sel: { id: hero.image }, name: alt, contains: Bajutsu }
- attribute: { sel: { id: panel }, name: data-state, matches: "^(ready|idle)$" }
- attribute: { sel: { id: panel }, name: aria-busy, absent: true }
attribute checks one HTML attribute of one element on the web (Playwright) backend
(BE-0466). It
covers what the element tree does not carry. Examples are a link's href, an image's alt, and a
data-* or aria-* state attribute.
selresolves the same way it does forvalue. No match fails the assertion, and two or more matches fail it as ambiguous. The element must be one the web query lists: one with adata-testid, arole, or an interactive tag, or a text element addressed by itslabel. A bare<img>carries no text, so it needs adata-testid. Two content-identical elements whose attribute values differ also fail as ambiguous, since nothing else tells them apart.namemust be a valid HTML attribute name. An empty name fails to load. A name holding a space, a quote,>,/,=, or a control character fails too. Bajutsu never lowercases the name. It reaches the DOM'sgetAttributeas written. HTML elements match it case-insensitively. SVG and MathML elements match it case-sensitively.- The operator is one of four, and a scenario names one alone.
equals,contains, andmatches(a regular expression) follow the rules ofvalueandlabel.absent: truepasses when the element has no attribute by that name. The scenario loader refusesabsent: false. - Presence differs from emptiness. A present attribute with an empty value reads as
"". Soequals: ""passes andabsent: truefails. A missing attribute failsequals,contains, andmatches. The reason says so:expected equals='x' but attribute 'href' is absent. A mismatch prints the actual value:expected equals='/docs' but actual='/about'. That reason passes the artifact redaction boundary like every other reason. ${...}tokens in the expected string expand like any other assertion field.
attribute works in a step's assert, in expect, in an if condition, and in an interrupts
condition. A wait step cannot use it. The runner reads the attribute on demand, right after each
query that feeds the block. A step's assert re-reads it on every tick of the condition wait. The poll stops once the attribute matches or the lane's wait floor runs out.
The floor comes from BAJUTSU_MIN_WAIT_TIMEOUT. Element snapshots, goldens, crawl output, and reports gain no
field. A report shows merely the attributes an assertion names.
The assertion needs the elementAttribute capability, and the web backend alone declares it. On
any other backend, the capability preflight fails the scenario before any device lease. The check
finds an attribute anywhere, the iOS WebView context of a web: block included. Config
validation also refuses an attribute in a non-web target's hooks.interrupts at load time. attribute
reads attributes, not DOM properties: the value assertion keeps covering a form control's live
value.
Network mocks (deterministic stubs)¶
mocks makes a test independent of a live server: when an outgoing request matches, BajutsuKit returns
a canned response instead of hitting the network. Each mock is { match, respond }.
matchreuses the request-side fields of the request matcher (method/url/urlMatches/path/pathMatches/bodyMatches).status/countdo not apply to a mock'smatch.respondis the canned reply:status(default200),headers(default{}),body(a string),delayMs(artificial latency). Omittingrespondreturns an empty200.
- name: GET answered by a mock stub
mocks:
- match: { method: GET, urlMatches: "example.com" }
respond:
status: 418 # real example.com returns 200; 418 proves the stub served it
headers: { Content-Type: text/plain }
body: "stubbed by bajutsu"
steps:
- tap: { id: net.fetch }
- wait: { until: { request: { method: GET, urlMatches: "example.com", status: 418 } }, timeout: 6 }
expect:
- request: { method: GET, urlMatches: "example.com", status: 418 }
Mocks are handed to BajutsuKit via the BAJUTSU_MOCKS env (dump_mocks, scenario/serialize.py). The formal
shape is in dsl-grammar.
Changing the mocks mid-scenario (setMocks)¶
The app reads the scenario-level mocks once, at launch. Some journeys need a different response
later on: the profile loads, then the next refresh fails. A setMocks step covers that case. The
step replaces the app's whole stub table with its own list of { match, respond } entries. The new
table does not extend the old one, and setMocks: [] removes every stub.
- name: profile refresh fails after a first success
mocks:
- match: { path: /api/profile }
respond: { status: 200, body: '{"name":"A"}' }
steps:
- tap: { id: profile.refresh }
- wait: { until: { request: { path: /api/profile, status: 200 } }, timeout: 6 }
- setMocks:
- match: { path: /api/profile }
respond: { status: 500 }
- tap: { id: profile.refresh }
- wait: { until: { request: { path: /api/profile, status: 500 } }, timeout: 6 }
The new table travels to the running app over the in-app control channel (BE-0365). The step then
waits until the app confirms it installed the table. No later step runs against the
old stubs. The step fails with a reason when the app refuses the table. It also fails when the app
stays silent for five seconds. A relaunch step restarts the app, and the app then reads the
launch-time mocks again.
setMocks has three requirements. run checks the first two before it touches a device.
- The run uses the
xcuitestbackend with network collection on. Web, Android, andfakecannot carry the channel. The step addresses the scenario's primary app. It cannot sit inside aweb:orapp:block, and it cannot name another target. - Neither the scenario nor its target pins
BAJUTSU_CONTROL_CHANNELto a value other than"1".runwrites"1"itself. - The app's build passes
-DBAJUTSU_ENABLE_CONTROL_CHANNELto BajutsuKit. Without that flag the app never answers, and the step fails on its timeout.
Reuse, data, and tags¶
A small templating and macro layer wraps the core grammar. It runs at load time, before the deterministic run, so the runner only ever sees plain, fully-expanded scenarios. The normative rules (expansion order, ${ns.key} interpolation, depth limits) are in dsl-grammar. This section covers the authoring perspective.
Components (use → reusable steps)¶
A component is a list of params and a list of steps that reference them as ${params.<name>}. A use step invokes it, binding params via with. use is a compile-time macro: expand_components (scenario/expand.py) replaces it with the component's substituted steps before the run. Expansion is recursive: a component may itself use another, up to depth 25. Expansion raises an error on a missing or unknown param, a residual ${params.*} referencing something undeclared, or a reference cycle. No use step survives into the run, so determinism is unaffected.
Expansion reaches a scenario's own steps and the recovery steps of each interrupts entry. It also reaches every if / else branch and forEach body inside them. It reaches every web: / app: block too. A use in any of these bodies expands at load time. A broken use in a branch the run never takes still fails the load.
A use step takes one modifier alone: target. The loader refuses a use step that also sets any of capture / extract / name / from. Expansion replaces the whole step, so it would otherwise drop those fields with no warning. Expansion carries target forward instead, as the multi-target rule below describes.
A component lives in a file of its own, reusable across the whole suite:
# login.component.yaml — a component file (a single mapping, loaded separately)
params: [user, pass]
steps:
- type: { text: "${params.user}", into: { id: auth.user } }
- type: { text: "${params.pass}", into: { id: auth.pass } }
- tap: { id: auth.submit }
# in a scenario — expands to the three steps above with params substituted
steps:
- use: { component: login.component.yaml, with: { user: alice, pass: hunter2 } }
- tap: { id: home.tab }
File-scoped components (components:)¶
A block reused inside one file alone needs no file of its own (BE-0422). Declare it under components: at the top of the scenario file. The scenarios below it then call it by name. The component sits beside the scenarios that use it. The suite's directory listing gains no entry:
# one scenario file — no separate component file
components:
search:
params: [query]
steps:
- type: { text: "${params.query}", into: { id: home.search }, submit: true }
scenarios:
- name: search returns dogs
steps:
- use: { component: search, with: { query: dog } }
expect:
- label: { sel: { id: home.status }, equals: "1 result" }
- name: search returns cats
steps:
- use: { component: search, with: { query: cat } }
expect:
- label: { sel: { id: home.status }, equals: "2 results" }
A scenario file carries components: in its {description, scenarios} mapping form. A file written as a bare list of scenarios takes the mapping form first.
The ref's own shape decides how use resolves. One component: field serves both kinds. Every existing scenario keeps working unchanged:
| Ref | Resolves as |
|---|---|
holds a /, or ends in .yaml / .yml |
a component file, resolved against the scenario file's own directory and confined to the suite root |
| anything else (a bare name) | an entry in this file's own components: |
A bare name the map does not define is an error naming the ref. No fallback opens a file. The map is scoped to one file. The loader reads it per file and never merges it across a suite directory. A name declared in one file stays invisible from its siblings. Reuse that spans files stays the component file's job. Crossing into a component file drops the map entirely. A component file declares no components:, so a bare use inside one is always undefined. A file-scoped component's own steps expand in the declaring file's scope. One may use another by bare name, or use a file by path.
A target's hooks may call a component too (above). Such a use resolves when the config loads. It resolves against the config file's directory, confined to the suite root. A config declares no components: map, so a hook's use names a component file by path. Its with values substitute at load, and ${secrets.*} and ${vars.*} still resolve when the step runs. A hook may not use group: or setMocks, directly or through a component.
Components in a multi-target scenario¶
A scenario declaring two or more targets calls components the same way (BE-0446). Each step a component expands to needs a target, and it receives one from one of two sources:
Caller (use: step) |
Expanded step omits target |
Expanded step names target |
|---|---|---|
Names target: X |
Stamped with X |
Accepted when it equals X; refused at load time otherwise |
Omits target |
Resolves as a hand-written step in the same position would: through primaryTarget if declared, otherwise a load-time error |
Kept as written, then checked against the declared targets |
The first row suits a single-target component. A sign-in is one example: it knows nothing about the device that calls it. The second row suits a cross-target component, whose own steps route across the scenario's targets:
components:
web-sign-in:
params: [email]
steps:
- tap: { id: onboarding.start }
- type: { text: "${params.email}", into: { id: auth.email } }
- tap: { id: auth.submit }
like-and-verify:
steps:
- target: showcase-swiftui
tap: { id: horse.favorite }
- target: web
wait: { for: { id: favorites.updated }, timeout: 5 }
scenarios:
- name: sign in on the web, then like on the app
targets: [showcase-swiftui, web]
steps:
# The caller fixes the target: every step of `web-sign-in` runs against `web`.
- use: { component: web-sign-in, with: { email: a@b.com } }
target: web
# The component routes its own steps across both targets.
- use: { component: like-and-verify }
A conflicting target fails the load rather than letting either side win. Overriding the component would send a step to a device its author never wrote it for. Letting the component win would break the caller's request without notice. The error names the component chain and the step:
use: web-sign-in > step 'submit': target 'showcase-swiftui' conflicts with the caller's target 'web'
A caller's target reaches every step the expansion produces, at any depth. It descends into if /
else branches, forEach bodies, and nested use calls. It stops at a web: or app: block. The
block itself receives the target. The steps inside the block keep omitting target. A component
can also take its target as a param (target: ${params.device}). Param substitution runs before
any target check. A use step inside a target group
receives the group's target as its caller target.
An interrupts entry's recovery use follows the same table, with one difference. Consider an
expanded recovery step that omits target. It stays on the runner whose interrupt guard fired. It
never resolves through primaryTarget.
Grouping steps (group: → folded in report.html)¶
A group: step names a run of consecutive steps. expand_components replaces it with its own
steps, the same way it replaces a use: step. run never sees either — both are compile-time
macros:
steps:
- group:
name: ログイン
steps:
- tap: { id: auth.open }
- type: { text: "${vars.user}", into: { id: auth.user } }
- tap: { id: auth.submit }
- tap: { id: home.tab }
Unlike use:, group: takes no params and names no separate file. The group is a purely local,
one-off label for organizing one scenario's own steps:. It does not reuse steps across
scenarios. A named use: / components: call is the right tool once the same run of steps is
called from more than one place.
report.html folds each group's steps into a collapsed section. A heading names the group and
counts its steps. A group whose steps all pass stays collapsed. A group holding a failing step
opens automatically. The existing "expand all" / "collapse all" controls open and close every
group too. A network request/response row, or a step that never ran, can split one group:
invocation into more than one fold in the report. Each fragment then gets its own heading, rather
than one continuous fold.
group: does not nest. A group: step inside another group:'s own steps: fails at load time.
So does one inside an if / forEach / web / app step's nested steps:. A group: that a
component carries into such a body through use fails the same way.
A group: step's own target works as on a use: step, in a scenario declaring two or more
targets. Expansion stamps it onto every step
the group produces, with the same depth and conflict rule.
A group: step that omits target leaves each child to resolve as a hand-written step would.
Data-driven scenarios (data / dataFile)¶
A scenario with data (inline rows) or dataFile (a CSV path — the two are mutually exclusive) is expanded into one scenario per row, substituting ${row.<column>} (expand_data, scenario/expand.py). Each derived scenario is renamed "<name> [row N: col=val, …]" and keeps the original preconditions, so every row reinstalls the app fresh and inherits the template's subject.reinstall.
- name: search returns a result
data:
- { q: dog, expect: "1 result" }
- { q: cat, expect: "2 results" }
steps:
- type: { text: "${row.q}", into: { id: search.field }, submit: true }
expect:
- label: { sel: { id: home.status }, equals: "${row.expect}" }
A string that is exactly one token (
"${row.qty}") takes the raw value (a number stays a number); a token embedded in a larger string is spliced in as text ("item-${row.id}").
A CSV dataFile has a header row naming the columns; each subsequent row becomes one scenario.
Refs stay inside the suite. A
usecomponent and adataFilepath resolve relative to the scenario file, and the resolved file must stay within the suite root (the scenarios dir the load started from). A ref that leaves it — an absolute path, a../chain that escapes the root, or a symlink pointing outside — is rejected with a clear error and never read, so a scenario cannot make the loader open a file outside its own tree (BE-0174). A relative ref that stays inside the root keeps working — a siblingcomponents/shared.yaml, or, from a scenario in a subdirectory, a../shared.yamlthat climbs no higher than the root.
Tags and selection¶
tags label a scenario; the CLI --tag / --exclude flags pick which scenarios run. A scenario is kept
when it carries at least one --tag (or none was given) and none of the --exclude tags —
--exclude wins over --tag (select_scenarios, scenario/select.py). Both flags accept a comma list.
- name: checkout smoke
tags: [smoke, checkout]
steps:
- tap: { id: cart.checkout }
uv run bajutsu run --target showcase-swiftui --tag smoke --exclude wip # run @smoke, skip anything @wip (across the app's scenarios dir)
Secrets (${secrets.X})¶
Declare secret environment-variable names in config (evidence: { secrets: [API_TOKEN, ...] }). Each declared name X is resolved from the environment and substituted into the executed step at action time as ${secrets.X}. The scenario file stores the token, never the value, and literal values are auto-masked in evidence, making secrets safe to commit and review. Unlike ${params.*} / ${row.*} (load-time expansion), this namespace is resolved by the run loop.
# config declares: evidence: { secrets: [API_TOKEN] }
steps:
- type: { text: "${secrets.API_TOKEN}", into: { id: auth.token } } # real value typed; token kept in the report
Runtime variables (${vars.*})¶
A step's extract modifier captures a UI element's property into vars.* after the step
executes. Subsequent steps (and scenario-level expect) can reference the captured value
via ${vars.<name>}.
steps:
- tap: { id: counter.inc }
extract:
count: { sel: { id: counter.value } } # vars.count ← element's value (default)
heading: { sel: { id: header }, prop: label } # vars.heading ← element's label
- assert:
- value: { sel: { id: other.field }, equals: "${vars.count}" }
Each extract entry specifies a sel (selector, resolved via resolve_unique) and an
optional prop (value | label | identifier; default value). If the selector
cannot be uniquely resolved or the property is None, the step fails.
Conditional steps (if)¶
A step can evaluate a condition (using the same assertion DSL) and branch:
steps:
- if:
condition: { exists: { id: dialog.alert } }
then:
- tap: { id: dialog.dismiss }
else:
- tap: { id: home.start }
The condition is evaluated against the current element tree (with ${...} interpolation).
If it passes, then steps run; otherwise else steps run (or nothing if else is omitted).
Nested steps share the same vars.* bindings as the enclosing scenario. capture and
extract modifiers are not allowed on if steps.
Iterating over elements (forEach)¶
A step can iterate over all elements matching a selector:
steps:
- forEach:
sel: { idMatches: "item.*" }
as: current
steps:
- tap: { id: "${vars.current}" }
The element list is snapshotted once at loop start. Each matched element's identifier is
stored as vars.<as> for the nested steps. An element with no identifier fails the step.
Zero matches is a no-op (success). The selector supports ${...} interpolation. capture
and extract modifiers are not allowed on forEach steps.
capture token grammar¶
Shared by capture: (per-step) and capturePolicy[].capture (rules). The form is
<kind>[.<modifier>].
- Kinds:
screenshot/elements/actionLog/deviceLog/network/video/appTrace/rawTree - Modifiers:
before/after/around/onError
Validation is over the set of kinds and modifiers (scenario/models/_base.py _validate_capture). The
acquisition timing per kind, and which are captured, are in
evidence.
YAML caveat¶
PyYAML (YAML 1.1) resolves on/off/yes/no to booleans. To prevent the capturePolicy
trigger key on: from becoming True, Bajutsu's YAML loader (common/_yaml.py) treats only
true/false as booleans and keeps on/off/yes/no as strings.
from (provenance)¶
from: records which natural-language phrase a construct was recorded from (BE-0044). It is an
optional string attached at four levels — the scenario (the original goal), each step but use
(whose one modifier is target), each expect assertion, and each capturePolicy rule — so a reviewer
can see why each part exists and judge whether record normalized the intent faithfully.
- name: open settings and reindex
from: "Open settings, reindex, and confirm the normalization setting is gone" # the original goal
steps:
- tap: { id: settings.open }
from: "Open settings"
expect:
- exists: { label: "Normalization setting changed", negate: true }
from: "The normalization setting is gone"
capturePolicy:
- on: { action: tap, idMatches: "*.submit" }
capture: [screenshot.after, network]
from: "Capture a screenshot and network log on every submit"
record(Tier 1, AI) is the only writer. It fillsfrom:while normalizing the goal into the structured scenario; a hand-authored scenario simply omits it (and a dumped scenario stays clean —from:is pruned when unset).run(Tier 2) ignores it entirely — provenance is authoring metadata, never read by the orchestrator, so it adds no AI to the gate and cannot affect pass/fail.- Grouping is emergent: when one utterance produces several steps, they carry the same
from:string; there is no span syntax.lintreports an advisory provenance-coverage figure (how many steps carryfrom:); it never fails a run. - Shown in
traceand the report.bajutsu traceprints each step's phrase inline (← "<phrase>") andreport.htmlshows it under the step, collapsing a run of the same phrase into one label — turning the timeline into a natural-language ↔ action map. - The phrase is kept verbatim in whatever language the author wrote (not translated).
Round-trip (load ⇄ dump)¶
load_scenarios(text) -> list[Scenario]: YAML string → validated models.dump_scenarios(scenarios) -> str: models → YAML, dropping only fields the author never set.
record's output goes through this dump_scenarios. The generated YAML reloads cleanly via
load_scenarios.