English · 日本語
CLI reference¶
Implementation:
bajutsu/cli/, built on Typer. Each command lives beside its owning feature, such asrun/cli.pyorrecord/cli.py.cli/commands/holds only the commands that belong to no feature:doctor,lint,schema,config,report, andrunner. The entry point isbajutsu = "bajutsu.cli:app"inpyproject.toml. Every command in this command-line interface (CLI) selects one app with--target <name>and points at config with--config(defaultbajutsu.config.yaml). App-specific differences live in config (configuration).
Related: run-loop · recording · codegen · configuration
Common¶
- Every command loads
.envfirst (_bootstrap, below). - A missing config, an undefined app, or no available actuator prints a message and exits with code 2.
--backendis a check, not a choice. A target names oneplatform, which decides the actuator (drivers). The flag accepts three values:- the target's platform, such as
ios; - that platform's actuator name:
xcuitest,adb, orplaywright; fake.
Any other value exits 2, and so does a comma list. Empty uses the target's own platform
(BE-0452).
- --udid takes no URL. A command that drives a device through an Appium grid names the grid in an
appium environment of its --worker-config file instead (see run).
run¶
Runs a scenario deterministically. Pass/fail is machine-only, and since BE-0402 no flag of run reaches a model at all.
The alert guard is on unless --no-system-alert-handling turns it off. The guard answers each OS prompt that a systemAlert interrupts entry names, in the scenario or its target. It answers through the native SpringBoard path (iOS XCUITest, no model). Any other prompt stays untouched, and the failure of the blocked step names it. See the systemAlert interrupts entries.
bajutsu run --target <name> [--scenario <file.yaml>] [options]
bajutsu run --scenario <self-declaring.yaml> [options] # --target optional
By default run loads every *.yaml in the app's configured scenarios dir
(targets.<name>.paths.scenarios, see configuration), so the config alone is enough
to run. Pass --scenario <file> to run one file instead, or repeat the flag to run several
specific files in one process, sharing a single warm runner.
The scenarios run in a child process of run
(BE-0449). run passes
its flags to that process through the one-shot path of bajutsu worker --once. The command line
and a worker thus share one implementation. The child process inherits the terminal's standard
input, output, and error. The PASS|FAIL line stays on stdout and progress on stderr. --step
reads the terminal as before.
The child process starts in a session of its own, so a Ctrl-C reaches run alone. run turns a
SIGINT, a SIGTERM, or a SIGHUP into the cancel that a serve job receives. The run then stops at its
next safe boundary and still writes its manifest and report
(BE-0370). A signal
that the caller already ignores stays ignored, as SIGHUP does under nohup.
Running the child in its own session changes three more things at the terminal:
- Ctrl-C twice. The first Ctrl-C asks the run to stop, which can take up to the grace window
(90 seconds by default,
BAJUTSU_CANCEL_GRACE) when a driver call is in flight. Press Ctrl-C again to stop the run immediately, without its manifest. - Ctrl-Z.
runstops the run's process group and then itself.fgorbgcontinues both. - A background
--steprun. The run does not belong to the terminal's job control, so a--stepprompt in a run started with&still reads the terminal instead of stopping the job. Run--stepin the foreground.
If run itself dies without a signal that it can catch (kill -9, an out-of-memory kill), the run
notices that its caller is gone and cancels itself the same way.
On Windows, where the web backend runs, run executes the run body in the run process itself,
with no child process, as it did before BE-0449. The child process, the signal relay, and the three
terminal behaviors above apply on macOS and Linux only.
| Exit code | When |
|---|---|
| 0 | every scenario passed; a not-applicable record beside them does not change this |
| 1 | a scenario failed, or every record was not applicable (no run executed) |
| 2 | a usage error, such as an unknown flag or an unloadable config |
| 128 + the signal number | run received SIGINT (130), SIGTERM (143), or SIGHUP (129), whatever the cancelled run reported |
When --target is optional¶
A scenario that declares its own
targets resolves every target it
needs from the config. --target then has nothing left to decide. Four rules follow:
| Rule | What happens |
|---|---|
--scenario becomes required in its place |
The whole-suite shorthand globs one target's configured scenarios dir. A scenario naming two or more targets belongs to no one such dir, so it names its files explicitly |
An explicit --target is checked, never ignored |
Passed beside a self-declaring scenario, it must name one of that scenario's own declared targets. A stale flag left over from editing the file exits 2, instead of picking a target the file dropped |
| The scenarios dir refuses a self-declaring file | A file carrying its own targets inside a target's configured dir fails before the run starts. The error names the file and points at --scenario. The check applies after --tag / --exclude, so a suite that selects the file away runs the rest |
A batch mixing both shapes still needs --target |
A file declaring no targets has nothing but the invocation's --target to resolve one from. Omitting the flag with even one such file in the batch fails, naming that file |
Passing --target alongside a mix of both shapes keeps working. Each file declaring no targets
uses it as always. Each self-declaring file has it checked for membership instead.
| Option | Default | Description |
|---|---|---|
--target |
(required, unless every --scenario file declares its own targets) |
the target app (config's targets.<name>); see when --target is optional |
--scenario |
config's paths.scenarios dir |
run these *.yaml files instead of the app's whole scenarios dir; repeat the flag to run several in one process, sharing one warm runner |
--backend |
the target's platform |
a check against the target's platform: the platform, its actuator name, or fake (see Common) |
--tag |
"" | comma list; run only scenarios carrying any of these tags |
--exclude |
"" | comma list; skip scenarios carrying any of these tags |
--udid |
booted |
the target Simulator (comma list = a device pool for --workers); a URL exits 2 |
--erase / --no-erase |
scenario › config › clean |
move every scenario's effective subject.reinstall along its ladder (overwrite < clean < erase). --erase makes it erase (wipe the simulator first); --no-erase lowers an erase to clean and leaves overwrite and clean alone. Omit both and each scenario's preconditions.subject.reinstall applies, then the target's subject.reinstall, then clean (BE-0473). A target that states erase therefore uninstalls under --no-erase; a scenario that must keep the app's data writes preconditions.subject.reinstall: overwrite |
--system-alert-handling / --no-system-alert-handling |
on | turn the system-alert guard on or off for every scenario. On, the guard answers the systemAlert entries of the scenario's interrupts and of the target's hooks.interrupts — the reactive guard that dismisses system alerts the iOS backend cannot see, natively on XCUITest (no model, BE-0315). No scenario or target switches the whole guard off: one that handles a prompt itself writes choice: leave on that prompt's entry (BE-0473). An alert no entry names is left alone. A guarded wait names what the guard saw in its own timeout; a blocked step outside one names it when the native query enumerated the alert, and on a backend with no native query fails as it always did (BE-0402). No AI credential is needed (recording) |
--ios-tipkit-handling / --no-ios-tipkit-handling |
scenario › config › off | dismiss a blocking Apple TipKit tip (the framework-owned popover), or keep it: the flag sets every scenario's tipKit interrupts entry to dismiss or leave, over the scenario's own entry and the target's, so that no scenario has to hand-author the same recovery. The guard recognizes a tip by its dismiss scrim (PopoverDismissRegion) and its own container (TipView) together, because a confirmationDialog installs an identical scrim and must be left alone; an author who does write a screen entry in interrupts (or in a target's hooks.interrupts) for a tip keys it on TipView for the same reason. iOS only (inert on other backends) and off unless asked for, since a tip is sometimes the assertion's own subject; omit and each scenario's own tipKit entry applies, then the target's, then off. Needs no credential (scenarios) |
--alert-poll-interval |
unset | seconds between the native system-alert presence queries. A scenario's own systemAlert entry's pollInterval wins over this flag, and this flag wins over the target's; within one layer the smallest value applies. The entries themselves carry no flag: an entry pairs a prompt with a choice, which one flag value cannot carry legibly, so it stays a scenario-file and target-config declaration (BE-0406) |
--log-predicate |
"" | an NSPredicate narrowing the deviceLog stream (e.g. subsystem) |
--log-subsystem |
"" | the os_log subsystem for appTrace (defaults to the app's subject.id) |
--network / --no-network |
config › on | collect the app's network exchanges for request assertions; omit and it resolves the target's evidence.collectNetwork config, then on (BE-0177). iOS needs BajutsuKit in the app; web observes natively via Playwright, and stubs scenario mocks in-process |
--workers |
1 | parallel scenarios over a device pool. A scenario declaring several targets holds one device per target for its whole length, so run caps this to what the pools can serve without starving a worker. On iOS, needs --udid u1,u2,… and is capped to that pool size. On web, --workers N alone is N parallel browser-context lanes — no --udid needed (BE-0054). Each lane carries its own network collector, interval recordings, and (iOS) device control, so network / video / setLocation / push work the same as a single-device run |
--baselines |
config's paths.baselines, then baselines/ beside the scenario |
override the target's paths.baselines: the directory of baseline images for visual assertions; baseline: home.png resolves inside it |
--schemas |
config's paths.schemas, then schemas/ beside the scenario |
override the target's paths.schemas: the directory of JSON Schema files for responseSchema assertions; schema: items.json resolves inside it (needs the schema extra) |
--goldens |
config's paths.goldens, then goldens/ beside the scenario |
override the target's paths.goldens: the directory of golden JSON files for golden assertions; golden: response.json resolves inside it |
--headed / --no-headed |
target driver.headless (headless) |
web backend: show the run in a visible, slow-motion Chromium window instead of headless, so you can watch each step (the window opens on the machine running the command). Omit to use the target's driver.headless config; iOS ignores it |
--browser |
target runsOn.browser.kind (chromium) |
web backend: the browser to drive for this run — chromium / chrome / edge / firefox / webkit; omit to use the target's runsOn.browser.kind config. The flag replaces that field, so a browser list the target declares stops counting as a device condition. On a target that lists its browsers, the flag names one of them, and any other browser exits 2 (configuration, BE-0076, BE-0468) |
--browsers |
"" | web backend: run the cross-browser matrix — a comma list of browser kinds (e.g. chromium,firefox,webkit); each scenario runs once per browser and the run is green only if every browser passes (configuration). On a target that lists its browsers, every named browser must be listed, or the flag exits 2. A single engine is exactly --browser |
--kind |
every kind in the target's runsOn.ios.kind |
iOS: run only these kinds — simulator, device, or a comma list of both; one run per named kind. Each must be a kind the target declares: a kind its runsOn.ios.kind list leaves out exits 2 and names the target's kinds, and a target with a single kind accepts only that kind. A named kind keeps its coordinate, so --kind device keys the same history as the device run of the full list. The flag applies to the command's --target, so a run whose scenarios declare a second iOS target exits 2 (BE-0468) |
--progress / --no-progress |
off | stream per-scenario / per-step progress lines to stderr (the serve UI consumes these) |
--score / --no-score |
off | print the app's entry-screen convention score (doctor's Ready/Partial/Blocked grade) to stderr, computed from this run's own first launch — so CI reads the tell without a separate doctor step that cold-spawns a second runner. Diagnostic only; never affects pass/fail |
--trace-driver |
off | write <sid>/driver_trace.json per scenario, recording every Python↔driver call — the driver method invoked, its host-device round trips, and (on Android) which of those fell back to a subprocess — with a start time and an elapsed duration, attributed to the step it happened during (BE-0415). Diagnostic only; never affects pass/fail |
--step |
off | debug one scenario: stop before each step at a step> prompt (see Stepping through a scenario). Needs a terminal, exactly one scenario, one engine, and one worker |
--break |
— | stop at a step, by its name or its index as a failure reason shows it (step 3 (tap): … is index 3), and run freely until then. Repeatable; implies the prompt |
--break-on-fail |
off | stop at the prompt when a step fails, while its screen is still up; implies the prompt |
--zip |
off | after the run, also write runs/<id>.zip — one portable artifact (report + evidence) for CI upload or sharing. Runs after the verdict, so it can't affect pass/fail; see export |
--runs-dir |
runs |
directory to write the run tree into. Lets a caller run from one working directory but persist the run elsewhere — serve uses it when the active config is bound from a different tree (a Git checkout or an uploaded bundle) to run from that tree while keeping the run in serve's store (BE-0073) |
--evidence-store |
"" (also BAJUTSU_EVIDENCE_STORE) |
after the run, upload the whole run tree to object storage at this URI — s3://bucket/prefix (AWS / R2 / MinIO) or gs://bucket/prefix (Google Cloud Storage). The remote layout mirrors the local one under the prefix (<prefix><runId>/…), so the upload path selects the cloud lifecycle policy (retain main-branch evidence, expire feature-branch evidence). Runs after the verdict, so an upload failure is reported as a warning and can't affect pass/fail. Needs the s3 or gcs extra (BE-0110) |
--config |
bajutsu.config.yaml |
the config file |
--label |
the config's own name | tag this run's history entry with a short free-text label, so the runs of two configs stay readable apart in the run list, Replay, and the run-stats dashboard (BE-0404). Opaque to the tool: never parsed, never matched against config, never consulted by authorization. A value longer than 120 characters is refused rather than shortened |
--config-offline |
off | for a Git --config (configuration): use the local cache and never touch the network — needs a pinned @<sha>, since a branch can't be resolved offline |
--require-pinned-config |
off | for a Git --config: fail unless it pins a commit SHA — a branch or tag can move under a gate, so only a SHA is accepted |
--worker-config |
the shipped default (bajutsu/common/capability/worker.yaml) |
the worker capability file run checks each scenario against; it replaces the default and never merges with it (BE-0450); see what this machine can run. run drives devices on its own host or through an Appium grid, so the file's environment is local or appium; a device-cloud environment such as devicefarm exits 2. The run inside a Device Farm package takes devicefarm-host, whose device Device Farm booted and installed the app on, so the run launches that build instead of installing app.path (BE-0465). An appium file names the grid's endpoint and its one platform (today ios): targets of that platform take the endpoint as their device, the scenario's other targets stay on this machine, and a scenario with two grid targets is refused (BE-0452). record, crawl, repl, audit, doctor, and bajutsu mcp take the same flag |
- The run writes evidence to
FileSink(runs/<runId>, udid=..., log_predicate=...)(evidence). runIdisYYYYMMDD-HHMMSS.- Output:
PASS|FAIL runs/<runId>/manifest.json. The command exits 0 if every scenario passes and 1 on failure. - Worker capability (BE-0450). Before
runresolves any target, it checks each scenario against the worker capability file and the host operating system. The file is--worker-config, or the shipped default. A scenario fails when it needs a driver this machine cannot run. It also fails when it declares more targets thanmaxTargetsPerJob. Its reason starts withworker-capability-unsupported:. The run's results and report record the scenario like any other scenario that did not pass.runacquires no device for that scenario and runs the rest. When no scenario can run,runexits 2 and lists each reason. - Device conditions form runs (BE-0452). A target may declare a device condition:
runsOn.ios.model,runsOn.ios.os,runsOn.ios.family,runsOn.android.avd,runsOn.android.apiLevel,runsOn.browser.version, or a list onkind,model,avd, orbrowser.kind. A scenario declares no condition of its own (BE-0469).runreads every device of the--udidpool, makes one run per listed value and per major version of a range, and runs each on its own eligible devices (configuration). Each run's evidence lands underruns/<id>/run-NN/. Its JUnit and Common Test Report Format (CTRF) test names carry its coordinates. A run that no device fits fails. A run an open range left without a device is "not applicable": the console, the manifest, and the reports list it apart (reporting). The line before the verdict counts passes, failures, and not-applicable records. When every record is not applicable,runexits with code 1 and printsno run executed.audit --repeatand an MCP run form runs the same way.record,crawl, andrepltake the first run alone.doctorlists the runs (below).--backend fakematches none, with one notice.--browsersbeside a remaining condition such asbrowser.versionexits 2, and--stepexits 2 when the conditions form more than one run. - Preconditions come first (BE-0452). Before any device work,
runandauditcheck each scenario. They check it against the targets it drives. Some fields exist on one platform alone, such assubject.launch.argsor atipKitinterrupts entry. When no driven target's platform has such a field, the command exits 2 and names it. So does a merged iOSseedPhotoswithout an effectivesubject.reinstall: erase(scenarios). - No
AI usage:line:runspends no tokens under any flag (BE-0402), so it installs no usage ledger and reports none.record,crawl, andtriage --aistill do.
bajutsu run --target showcase-swiftui --udid <UDID> --backend ios --no-erase # the app's whole scenarios dir
bajutsu run --scenario demos/showcase/scenarios/smoke.yaml --target showcase-swiftui --no-erase # one file
bajutsu run --scenario cross-platform.yaml --config both-targets.yaml # the file names its own targets
bajutsu run --target showcase-swiftui --worker-config worker.yaml # iOS targets on an Appium grid
The worker.yaml behind the last line names the grid in an appium environment:
version: 1
environment: appium
appium:
endpoint: https://grid.example.com/wd/hub
platform: ios # the one value today
drivers: [xcuitest, adb, playwright, fake]
Stepping through a scenario¶
These flags hold the step loop at a step> prompt, so an operator can look at the app between steps.
The prompt goes to stderr, and stdout stays the PASS|FAIL line. --step stops before every step.
--break runs freely until it reaches the named step. --break-on-fail stops once, on the step that
fails; a scenario-level expect mismatch does not stop.
| Command | Effect |
|---|---|
next (or n, or an empty line) |
run the step, then stop before the next one |
continue (or c) |
run on until the next breakpoint, or the end |
quit (or q, exit, or end of input) |
end the run here; the scenario is reported as cancelled and the run exits 1. At a failure pause, the failure keeps its own reason |
| Ctrl-C | ends the run like quit, and the run exits 130 (see run) |
tree, find, screenshot |
read the live app, exactly as in repl |
tap, type, scroll, back, step <yaml> |
act on the app, exactly as in repl |
A command that acts on the app makes the run hand-driven. Whatever its assertions say, the scenario
then fails with interactive: <verb>, naming the first such command's verb. result.json records
that verb under interactive; the rest of the line is never kept, since it can hold a typed value. Assertions judge a run that reads or steps, as usual. A long pause can still change the outcome for an app that reacts to time, such as a session timeout. Steps an interrupts rule runs never stop at the prompt. The prompt needs a terminal on
stdin and stderr, so a piped or CI run exits 2. A run with more than one scenario, a --browsers
matrix, or --workers above 1 exits 2 as well.
doctor¶
A runnability gate plus the convention score for the current screen (AI-independent; configuration).
bajutsu doctor --target <name> [--udid booted] [--backend ...] [--config ...] [--scenario <file.yaml>] [--environment-only] [--worker-config <worker.yaml>]
doctorfirst runs the environment gate (preflight.py). The gate checks the required CLIs for the actuator (xcrunandxcodebuildfor XCUITest) and a booted Simulator, and prints a ✓/✗ checklist. A missing check exits 1, failing fast with a fixable hint.--environment-onlystops right after that gate — render theenvironment:section, then exit 0 (a pass) or 1 (a✗), skipping the screen probe below (on iOS, the cost of bringing up a short-lived XCUITest runner). A CI lane that only needs to verify the environment gate against a real host (BE-0304) uses this to skip the probe's unrelated failure modes, the runner's own startup flakiness included.--scenario <file.yaml>runs a device-free capability preflight (BE-0024): before the screen probe, it checks whether the scenario uses constructs the chosen backend can't perform and exits 1 if any are unsupported.--worker-configtakes the same worker capability file asrun. Anappiumfile sends the screen probe of a target on the grid's platform to the grid.- A device condition adds a
runs:section. The conditions arekind,family,model, andoson iOS,avdandapiLevelon Android, andbrowser.*on the web. The conditions come from the target'srunsOn. The section lists the runs they form, for each scenario of the--scenariofile, or for the target itself without--scenario.doctorreads the--udidpool the wayrundoes and forms the same runs. Each line names the devices that fit the run:
runs:
checkout:
✓ iPhone SE (3rd generation) / iOS 17 [kind: simulator]: iOS 17.5 "iPhone SE (3rd generation)" (5A3F...)
✘ iPad Pro 13-inch (M4) / iOS 17 [kind: simulator]: fails: no available device fits this run
– one run [kind: simulator]: not applicable: runsOn.ios.os ">=19" available: ...
✓ marks a run with its eligible devices. ✘ marks a run that fails for want of a device. –
marks a not-applicable record. An iOS line names, in brackets, the kind and the family the run
took when its label leaves them out: a single kind or family adds no coordinate, so
family: ipad reads one run [kind: simulator, family: ipad]. A listed kind and a family
list are coordinates, so the label carries them, as in device / ipad (BE-0468). The section is
informational and never changes the exit code.
A target with no pool to read lists the declared runs alone, marked •. This covers a grid
target, a physical device under a condition other than kind and family, and a pool whose
devices cannot be read. An open range then reads as
"one run per available major version". --backend fake evaluates no condition and prints no
section.
- When no earlier check stops it, doctor calls query() through the actuator and renders
score(elements, subject.idNamespaces). Under a
device condition, the probe scores the device of the target's first run, the one
record takes. Exits 1 when the grade is Blocked, 0 otherwise.
audit¶
A static determinism score for a scenario: the device-free counterpart of doctor's convention
score (AI-independent; selectors; BE-0049).
audit reads a scenario file, expanding components and data like trace --explain. It reports, per
scenario, how reproducible the scenario is, without running it.
bajutsu audit <scenario.yaml> [--json]
- Grades each selector on the stability ladder (selectors): a unique
id/idMatchesis stable;label/labelMatches/traits/valueis moderate (auxiliary, no id); anindex(nth-of-many) is fragile. The audit also flags coordinate gestures (swipe {from,to}, which a stable id could replace), over-loose waits (until: screenChanged/settled, which wait for no concrete condition), and every fixed sleep (sleep, with its reason); more than 10 seconds of fixed sleep in one scenario grades itModerate. - Emits a per-scenario
grade(Stable/Moderate/Fragile), a stability fraction, and the located findings, as text or as--jsonfor tooling. - Advisory and read-only: it never runs the scenario, never edits it, and never gates CI — a successful audit exits 0 even with findings (only a missing / unreadable scenario file exits 2). A finding is something to harden, not a verdict — the opposite of retry-to-pass, which hides flakiness.
Two further modes prove determinism by observation instead of static grading:
bajutsu audit <scenario.yaml> --repeat K --target <name> # run K times, diff the outcomes (takes --worker-config, like run)
bajutsu audit --history <runs-dir> # mine past runs for flakiness
--repeat Kruns the scenarioKtimes under identical preconditions and reports anything whose outcome varied (deterministicvsflaky) — a divergence is a finding to fix, never a retry that turns red into green.--history <runs-dir>is the longitudinal view: it groups each scenario's accumulated runs by the run's scenario fingerprint (theprovenance.scenarioHasheachmanifest.jsoncarries) and classifies each scenario (flaky/deterministic/unproven). A verdict that flipped while the fingerprint stayed constant is true flakiness — not an edited scenario, since editing changes the hash and starts a fresh group. Keying by scenario name as well as fingerprint pins which scenario in a suite flaked. Runs with no fingerprint (pre-provenance) can't be grouped and are reported as skipped. Like the other modes it is read-only and exits 0 even when it finds flakiness (only a missing runs dir exits 2).- A history is per OS version
(BE-0358).
The group key also carries the device operating-system (OS) version the run happened on, parsed
from the
device_runtimelabel eachmanifest.jsonrecords per scenario. One scenario run on iOS 18.6 and on iOS 26.5 forms two histories, each classified on its own evidence. A verdict that differs because the versions differ then reads as what it is — a reproducible difference on one OS — rather than as flakiness. Two spellings of one version (iOS 18.6andiOS 18.6.1) are the same OS, so a history never splits across patch releases. A run whose label is missing or unrecognized (a web run, or one recorded before the label was mined) groups underunknown OSrather than joining any version's history, and the output states that count.
coverage¶
A static end-to-end (E2E) coverage map for a suite: the read-only counterpart of doctor's convention score
(AI-independent; BE-0050).
Where doctor grades the ids one screen exposes, this grades the ids a whole suite exercises:
it walks every *.yaml in the app's configured paths.scenarios dir (expanding components / data), groups
the stable ids they reference by namespace, and measures them against the app's declared
subject.idNamespaces (configuration) — without running anything.
bajutsu coverage --target <name> [--config ...] [--runs <dir>] [--crawl <screenmap>] [--json] [--html <path>]
- Reports the coverage fraction (declared namespaces the suite references / declared namespaces),
the per-namespace ids that touch each, the gap list (declared namespaces no scenario
references — what is untested), and off-namespace ids (referenced ids whose namespace was never
declared). As text, or
--jsonfor tooling. - A referenced id is any
id/idMatchesa scenario addresses — across steps, nested control flow,withinscopes, and assertions. A placeholder the run binds later is not one: the suite loader expands components (params.*) and data rows (row.*) before building the map, and a${vars.*}or${secrets.*}token surviving that expansion names no element, so the map excludes it rather than counting the token's own text as an id. --runs <dir>folds in two run-evidence dimensions:- Endpoint coverage: it reads every
network.jsonunder the runs dir (the union of observed exchanges) and measures how many observed endpoints (METHOD path) the suite's network assertions (request/event/requestSequence) cover. It reports the fraction asserted, the unasserted observed endpoints (traffic the suite never asserts on), and matchers declared but not observed in any run. - Observed-id coverage: it reads every per-step
elements.jsonunder the runs dir and collects the stable ids the runs actually rendered (each element'sidentifier; null and empty ids are dropped), grouping them by the declared namespaces — the run-evidence counterpart to the static id map. It reports the per-namespace observed ids, the namespaces observed in no run, and observed ids whose namespace was never declared (off-namespace).
Omit --runs for the static id-namespace map only.
- --crawl <screenmap> (with --runs) folds in a screens-visited dimension against an
autonomous crawl's discovered surface (BE-0038):
the denominator is the screens the crawl found (screenmap.json nodes; pass the file or its
run dir), the numerator is the screens the run set reached — each per-step elements.json is
fingerprinted with the same crawl.fingerprint, so a visited screen matches a discovered one.
It reports the fraction reached and the unvisited screens the crawl discovered but no run
touched. Needs --runs for the visited evidence; given --crawl without it, the dimension is
skipped with a warning.
- --html <path> also writes a self-contained HTML report of the same figures (inline CSS,
no JavaScript, no external asset — it opens straight from disk), with a coverage bar per dimension
and the gap / off-namespace / unvisited lists called out. The endpoint, observed-id, and
screens-visited sections render only when --runs (and, for screens, --crawl) supply them. The
text (or --json) output is unchanged; the path is confirmed on stderr.
- Advisory and read-only: it never runs a scenario, never edits anything, and never gates CI —
it exits 0 even with gaps (only a missing config / scenarios dir or an unreadable scenario exits
2). A gap is a namespace to cover, not a verdict.
impact¶
Test impact analysis finds, from a source change, the scenario steps that the change is likely to affect
(AI-independent; BE-0321).
impact is the reverse of coverage: where coverage walks the suite forward to the app surface it
exercises, impact inverts the same static analysis into a map from each
stable id, screen, and asserted endpoint to the (scenario, step)
pairs that reference it, then reads a change as a git diff and reports the steps a developer should
review. A developer sees exactly the steps at risk before pushing, and a continuous-integration (CI) pipeline can order those steps first
for fast feedback.
bajutsu impact --target <name> [--config ...] [--range HEAD~1..HEAD] [--diff <path>|-] [--repo <dir>] [--json]
- The change is a
gitdiff.--rangenames a revision range to diff (the defaultHEADdiffs the working tree againstHEAD);--repopicks the repository (default: the current directory). Alternatively--diff <path>reads a unified diff from a file, or--diff -from stdin — the form a CI pipeline that already has the diff feeds in. In working-tree mode (a range with no..), an untracked new file is folded in too — read for its referenced ids, or marked unattributable when it is binary — so a brand-new screen is not silently missed. - The match is a plain string search, over the stable ids (and screen names / endpoint paths) the suite already declares: an id is touched when its literal appears on an added or removed line of the diff. This needs no language-specific parsing and encodes no per-app knowledge, so it works identically for a Swift, Kotlin, or TypeScript diff.
- Reports the affected
(scenario, step)pairs, each carrying the touched literal that implicates it (so you see not just which step but why), as text or--jsonfor a CI consumer. - Soundness is bounded, and the bound is surfaced. A change the scan cannot vouch for is
unattributable: a change that edits no referenced literal (a handler's logic, a shared helper), or
one the string match can never see into (a binary file). When the change carries any such part, the
report is flagged incomplete and lists the unattributable files, so a CI narrowing must fall
back to the full suite on that signal (
--json'scompleteisfalse). Over-selection is the safe direction — a short or common id may widen the affected set, but no affected step is silently skipped. - Safe CI use. The affected set is a mechanism; a team chooses the policy. The safe default is additive: order the affected steps first, or flag them for a human, while still running the whole suite. Narrowing a pre-merge run down to the affected set is opt-in, and sound only with two safeguards — the conservative fallback above, and a full-suite safety cadence (the whole deterministic suite still runs on merge to the main branch, nightly, or at release), so a step the scan misses is caught before it ships. In every case the pass/fail verdict stays with the deterministic runner.
- Advisory and read-only: it never runs a scenario, never edits anything, and never gates CI —
it exits 0 whatever the affected set (only a missing config / scenarios dir, an unreadable
scenario, or a
git/ diff read failure exits 2). No model is consulted at any point.
stats¶
An aggregate run-stats dashboard — the read-only trend across many runs, complementing the
per-run report.html (AI-independent;
BE-0102). Where coverage
answers "what surface do we test?" and audit answers "is a scenario reproducible?", stats
answers "how is the whole suite doing over time?" It aggregates the manifest.json of every run
under a directory, with no device, no AI, and no verdict.
bajutsu stats --runs <dir> [--json] [--html <path>]
--runs <dir>is the directory of past runs to aggregate.statsreads each run'smanifest.jsonand skips an unreadable or malformed one. A missing directory exits 2.- Reports, as text or
--jsonfor tooling: - Pass-rate over time — overall and per day (the day is parsed from the run id's timestamp
prefix; a custom-labelled run id has no day and buckets under
(no date)). - Duration / performance — total wall-clock across the set (each run's duration is summed from
its scenarios'
duration_s) and the slowest scenarios by average duration. - Failure hotspots — the scenarios, steps (
scenario > action), and assertion kinds that fail most, each with its most frequent failure reason. - Flakiness — each scenario's classification (
flaky/deterministic/unproven) reused from the BE-0049 longitudinal audit. - Volume — run count per backend, the denominator the rates are read against.
- Scenario series are keyed by the BE-0049
(scenarioHash, name)identity, widened with the device OS version the run happened on (BE-0358), so a verdict that flips at a constant fingerprint on one OS is true flakiness while an edited scenario — or the same scenario on another OS version — starts a fresh series. Each series names its OS and averages durations within it, so two OS versions' timings never pool into one average that neither ran. A run without aprovenance.scenarioHashstill counts toward the run-level trend but can't join a scenario series (reported as skipped). --html <path>also writes a self-contained HTML dashboard of the same figures (inline CSS, a minimal inline-SVG trend line, no JavaScript, no external asset — it opens straight from disk). The text (or--json) output is unchanged; the path is confirmed on stderr.- Advisory and read-only: the command never re-runs a scenario, never recomputes a verdict, and never gates CI. An older manifest that lacks a newer field renders as "not captured", never as a failure. Any threshold a team sets on its numbers is that team's own informational check.
flakiness¶
A ranked flaky-scenario surface over the run history: the actionable counterpart to audit
--history (AI-independent;
BE-0220).
Where audit --history labels each scenario flaky / deterministic / unproven, flakiness
sorts the whole suite flaky-first and scores how badly each one flips, so a team sees its worst
offenders at a glance, with no device, no AI, and no verdict.
bajutsu flakiness --history <runs-dir> [--json] [--window N] # mine a directory of run manifests
bajutsu flakiness [--org <org>] [--json] [--window N] # read the serve database (BAJUTSU_DATABASE_URL)
- Two sources, one ranking.
--history <runs-dir>mines a directory of past runs'manifest.json(the CI / scripting form); omitting it reads the serve database (BAJUTSU_DATABASE_URL), grouping straight from theprovenance.scenarioHashstamped on each run row. A missing runs directory or an unconfigured database exits 2. - Runs are grouped by
provenance.scenarioHashand each scenario is scored by its verdict flip rate —2·min(passed, failed)/runs(0 when consistent, 1 at a 50/50 split) — reusing the BE-0049 classification. The output is ordered flaky-first, then by descending flip rate, then by run count. - The grouping is per OS version too, under the same rule as
audit --history(BE-0358). A fleet running one suite across a device matrix no longer scores a genuine OS difference in it as flakiness. This surface groups per run whereaudit --historygroups per scenario, so a run whose scenarios spanned two OS versions can speak for neither and joins the sameunknown OSgroup an unrecognized label gets. Reading the database, the OS comes from the column each run row carries. A run recorded before that column existed is repaired from its stored manifest each time theservepanel ranks it — in memory, never written back, since a write on a read path could re-insert a run an operator had purged. Where the manifest is gone the row staysunknown OS, and the report discloses that count. - The grouping is per target and per run coordinates as well
(BE-0452).
Device conditions can split a scenario into runs, such as one per listed model. Each run's
coordinates then join the key. A layout bug on one model thus never reads as a flaky scenario.
The target joins the key as well, so a scenario run against two targets keeps two histories. The
text output names the coordinates in brackets after the OS. A run with no condition keeps empty
coordinates. A history recorded before BE-0452 thus continues. One run can hold results on two or
more coordinates, since an invocation records every run its conditions formed under one run id.
Such a run groups under
*, apart from every other history. - Each entry carries the newest passing and newest failing run ids, so
--jsonconsumers (and the serve panel) can link straight to the representative evidence on both sides. --window Nkeeps only each history's newestNruns before scoring — per history, since that is the group being scored (0, the default, uses the whole history). A history holds one scenario's runs on one OS version, target, and set of run coordinates. Windowing per scenario instead would evict an older OS version's history once the newest runs had all moved to a newer one, hiding the very cross-version finding the grouping exists to surface.--org <org>picks the tenant whose runs the database read mines.- A run with no
provenance.scenarioHash(a pre-provenance run) or no recorded verdict can't be grouped and is reported as skipped, exactly as inaudit --history. - Advisory and read-only: the command only reports flakiness already recorded in history. It re-runs nothing, recomputes no verdict, and never gates CI.
export¶
Bundles a finished run into a single portable .zip. The archive holds report.html together with manifest.json,
junit.xml, ctrf.json (reporting), the executed scenario.yaml, and
all of its evidence (screenshots, video, network.json, …)
(BE-0060).
The whole runs/<id>/ tree is rooted under a single <id>/ folder, so report.html's relative
links resolve offline, so the report opens by double-click with no server.
bajutsu export <run-id | run-dir> [-o out.zip] [--force]
<run>is a run id (resolved under--runs, defaultruns/) or a path to a run directory.- Output defaults to
<id>.zipbeside the run dir;-o/--outputoverrides. It refuses to overwrite an existing file without--force(mirroring howrecordnever silently overwrites). exportis pure packaging of what the run already wrote, with no device, no AI, and no effect on any verdict. It archives strictly the run dir (never.env, config, or anything above it) and inherits the run's secret-scrubbing.bajutsu run --zipis the same archiver, run inline after the verdict;bajutsu serveoffers it as a Download button beside the embedded report.
trace¶
Inspects a finished run as a text timeline: per scenario, steps and observed network
exchanges interleaved chronologically, followed by expectations, app-trace intervals, and an evidence
summary. Read-only (reads the saved manifest.json / network.json / appTrace.json, and
scenario.yaml for provenance).
- Each step shows the natural-language phrase that it was recorded from, its
from:provenance (BE-0044), inline as← "<phrase>"; a run of steps sharing one phrase is labeled once. A hand-authored scenario or an older run without provenance simply shows none.
bajutsu trace [<run-dir>] [--scenario <substr>] [--runs runs]
bajutsu trace --explain <scenario.yaml> # pre-run dry run (no device)
- With no
<run-dir>,traceuses the latest run underruns/.--scenariofilters by name substring. - Exits 2 if no run is found.
--explain <scenario.yaml>is the pre-run counterpart: instead of reading a finished run, it previews how the scenario'scapturePolicywould fire (BE-0028). Action-triggered rules are counted exactly and their matching steps listed;event/resultrules are reported as runtime-conditional; and a heavy capture (video/deviceLog/appTrace/network) on a broadly-matching rule is flagged with ⚠ so you can tighten the match before a run pays for it. Read-only and deterministic — no device, no LLM. (Components and data rows are expanded; a target'shooks.setupis not included.) Exits 2 if the scenario file is missing.
report¶
Re-renders a finished run's report.html (and re-emits junit.xml and ctrf.json,
reporting) from its stored data, with
the current template, and needs no device, no LLM, and no re-run
(BE-0068). A
template improvement or a rendering-bug fix reaches past runs without re-executing them; the
verdict is read from the stored model, never recomputed.
bajutsu report <run-id | run-dir> # re-render one run
bajutsu report --all [--runs runs] # re-render every run dir (with a manifest.json) under runs/
- The render model is
manifest.json(a versioned, lossless render input —schemaVersion) plus the executedscenario.yaml; the renderer reads only the run dir. An older run renders without error, with any newer-only section shown as "not captured" rather than invented. reportre-presents recorded outcomes. It never re-evaluates an assertion or alters a verdict, so it sits inside the determinism contract.serveuses the same renderer on view: it rendersreport.htmlfresh from each run's stored model on every request (falling back to the baked file if the model can't be loaded), so upgradingserverefreshes every report with no re-bake.- Exits 2 if the run (or, with
--all, the runs root) has no readablemanifest.json.
triage¶
Diagnoses the first failed scenario in a run and suggests a minimal fix. The diagnosis is advisory: it
never judges pass/fail (the AI boundary). It assembles failure context (the failing step and its
reason, failed expectations, the element tree nearest the failure, the scenario) and runs a
TriageAgent. The default is rule-based (HeuristicTriageAgent, no API key): it categorizes the
failure (selector / timing / assertion) and, when a target id is absent but a similar id is on
screen, suggests "did you mean …?" (the classic renamed-id self-heal). --ai swaps in a
Claude-backed agent (needs ANTHROPIC_API_KEY) that reasons over the same context plus the
failure screenshot for richer diagnoses.
An agent may also return a structured fix the tool can apply: renameId, addIndex
(disambiguate an ambiguous match), or raiseTimeout. --apply <scenario-file> prints it as a
dry-run diff; --write applies it to the source; --rerun --target <name> then re-runs the
patched scenario (--no-erase) and reports whether it now passes. The boundary holds: a fix
is applied only when the user opts in after reviewing the diff, and a fragment that no longer matches the
source is a safe no-op.
bajutsu triage [<run-dir>] [--scenario <substr>] [--runs runs] [--ai]
bajutsu triage [<run-dir>] --ai --apply <scenario-file> [--write] \
[--rerun --target <name> [--backend ios] [--udid <udid>]]
- The default run is the latest one under
runs/. The command exits 0 when the run has no failed scenario. --rerunrequires--write(nothing to verify otherwise) and--target.- With
--ai, anAI usage:line of the tokens the diagnosis consumed is printed to stderr after the diagnosis. The rule-based default uses no AI, so it prints nothing. --json <path>also writes the diagnosis as JSON (category, summary, suggestions, and the structured fix); combined with--apply, it includes the fix's unified diff and patched text.--jsonitself only writes that JSON file and never touches the scenario source — patching the source still requires--write(as it always has). TheserveWeb UI uses--jsonto surface triage in the browser (BE-0147).
Cross-run flaky triage (--flaky)¶
Where the default triage explains one failed run, --flaky explains a pattern: why one
scenario intermittently passes and fails while its content fingerprint is unchanged
(BE-0220).
It is the fix-proposal counterpart to flakiness: flakiness ranks which
scenarios flip, and --flaky diagnoses why one of them does and proposes a fix. It reads the
evidence of both the passing and the failing runs of the scenario and reasons about the delta
between them (a selector that resolved in one run but was ambiguous in another, a wait that beat a
spinner most of the time, a network response that varied).
bajutsu triage --flaky --scenario <name> --history <runs-dir> --ai [--apply <file> [--write]] [--json <path>]
- Requires
--ai. Cross-run diagnosis has no rule-based agent. The delta reasoning is exactly the judgement-free investigator work that the LLM does, so--flakywithout--aiexits 2. It stays advisory: the runs already decided pass/fail; this only explains and proposes, never on the verdict path (prime directive 1). --scenario <name>picks the flaky scenario (matched as a substring of the run's scenario name, resolved to one exact name so two scenarios sharing a substring never mix), and--history <runs-dir>is a directory of past runs (each with amanifest.json) to contrast — the same directory shapebajutsu flakiness --historyandaudit --historyread. Both are required; a missing one exits 2.- The report shows the passing / failing run counts, the fingerprint, the intermittency category
(
selector-ambiguity/timing/network-variance/state-leak/unknown), and the proposed fix as a reviewable diff against the current scenario — never applied on its own. - Reuses the same structured fixes and flags as single-run triage:
--apply/--writepatch the scenario,--jsonwrites the machine-readable result, and any edit that would weaken an assertion is flagged as laxer (BE-0023) inline in both surfaces. A full YAML rewrite is deferred to a follow-up; the current fixes are the targeted structured edits. - Exits 0 when the scenario's history has no failing run or no passing run to contrast (nothing to diagnose as flaky). The mode is advisory, like the other read-only modes.
record¶
Explores toward a goal with AI and writes the recorded scenario (Tier 1; recording).
By default it auto-names a *.yaml under the app's configured scenarios dir
(targets.<name>.paths.scenarios); pass --out to write a specific path instead.
bajutsu record --target <name> --goal "<natural-language goal>" [--out <file.yaml>] [options]
| Option | Default | Description |
|---|---|---|
--target |
(required) | the target app |
--goal |
(required) | the goal to author (natural language) |
--out |
auto-named in config's paths.scenarios dir |
explicit output path (overrides the app's scenarios dir) |
--name |
(from the goal) | file name for the auto-named scenario (ignored when --out is given) |
--udid |
booted |
the target Simulator |
--backend |
the target's platform |
a check against the target's platform (see Common) |
--erase / --no-erase |
--erase |
erase before launch (the app must be installed); --erase raises the effective subject.reinstall to erase and --no-erase lowers an erase to clean, so the target's subject.reinstall: erase does not decide the wipe here |
--system-alert-handling / --no-system-alert-handling |
on | clear prompts during authoring (needs an API key) |
--max-steps |
30 |
cap the number of authoring turns (and therefore worst-case token spend) (BE-0194) |
--screenshot / --no-screenshot |
--screenshot |
send a screenshot each turn; --no-screenshot records elements-only (cheaper), for an app fully instrumented with ids (BE-0194) |
--headed / --no-headed |
target driver.headless |
web backend: author against a visible (headed, slow-motion) browser instead of headless; omit to use the target's driver.headless config |
--browser |
target runsOn.browser.kind (chromium) |
web backend: the browser to author against — chromium / chrome / edge / firefox / webkit; omit to use the target's runsOn.browser.kind config. On a target that lists its browsers, any other browser exits 2 |
--kind |
the first kind in the target's runsOn.ios.kind |
iOS: the one kind to author on — simulator or device. It must be a kind the target declares, or the flag exits 2; a comma list exits 2 too, because the command runs once (BE-0468) |
--alert-vision-instruction |
"" | free text the vision guard reads instead of dismissing the prompt |
--language |
config ai.language (auto) |
AI output language for the authored prose (from: provenance, reasoning) — ja / en / auto; overrides ai.language, auto follows the goal (BE-0188) |
--config |
bajutsu.config.yaml |
config |
--worker-config |
the shipped default | the worker capability file, as in run; an appium file authors a target of the grid's platform on the grid |
- Internally,
launch_driverfeedsrecord_loop(driver, goal, ClaudeAgent(), ...), whose result goes todump_scenarios. - Output:
recorded <N> steps -> <path>. NeedsANTHROPIC_API_KEY(ClaudeAgent). - A Git
--configis read-only input (BE-0063):recordreads the config from the fetched checkout, but the authored scenario goes local — with no--outit auto-names under the current directory (not the checkout'spaths.scenariosdir, which is the read-only SHA-keyed cache), and an--outinside the checkout is refused. Review the file and commit it to the repository through normal git. - An
AI usage:line with the tokens the authoring (and any alert-guard) AI consumed follows on stderr.
crawl¶
Explores the app breadth-first and writes a screen map of the reachable screens and the
transitions between them (Tier 1; BE-0038). record is
goal-directed: AI explores toward one natural-language goal and writes one scenario. crawl
is instead systematic discovery: it visits the screens it can reach and reports what it found. The
exploration engine is deterministic, since a screen's identity and the order in which candidate actions are
tried are pure functions of the element tree. The engine involves no AI, and crawl is never a
pass/fail gate.
bajutsu crawl --target <name> [--max-screens N] [--max-steps N] [--out <dir>] [options]
| Option | Default | Description |
|---|---|---|
--target |
(required) | the target app |
--max-screens |
50 |
stop after discovering this many distinct screens |
--max-steps |
200 |
stop after taking this many actions |
--prune-global / --no-prune-global |
on | explore a global control (a tab/nav reused across screens) once instead of from every screen that shows it; the Web UI lets you resume a pruned branch on demand (see --resume-src/--resume-key) |
--udid |
booted |
the target Simulator — or a comma list (A,B,C) for a parallel pool (see --workers) |
--workers |
1 |
crawl with this many workers at once, sharing one screen map: across this many simulators on iOS (BE-0064, capped to the --udid devices) or this many browser processes on web (BE-0077). 1 = single-worker crawl |
--backend |
the target's platform |
a check against the target's platform (see Common) |
--erase / --no-erase |
--erase |
erase before launch (the app must be installed); --erase raises the effective subject.reinstall to erase and --no-erase lowers an erase to clean, so the target's subject.reinstall: erase does not decide the wipe here |
--system-alert-handling / --no-system-alert-handling |
on | dismiss unexpected OS prompts while crawling (so they aren't read as crashes; uses the configured AI provider — ANTHROPIC_API_KEY, or AWS credentials for Bedrock) |
--alert-vision-instruction |
"" | free text the vision guard reads instead of dismissing the prompt |
--headed / --no-headed |
target driver.headless |
web backend: crawl a visible (headed, slow-motion) browser instead of headless; omit to use the target's driver.headless config |
--kind |
the first kind in the target's runsOn.ios.kind |
iOS: the one kind to crawl on — simulator or device. It must be a kind the target declares, or the flag exits 2; a comma list exits 2 too, because the command runs once (BE-0468) |
--language |
config ai.language (auto) |
AI output language for the guide's streamed reasoning — ja / en / auto; overrides ai.language, auto stays English for crawl (BE-0188) |
--out |
runs/<timestamp> |
run dir the screen map is written into |
--resume-src / --resume-key |
"" | resume one pruned branch (from the Web UI's "tap to resume"): --resume-src is the screen fingerprint it was pruned on, --resume-key its replay key, both with --out pointing at the existing run. Mutually exclusive with --continue |
--continue |
off | continue a past run (point --out at it) — re-explore every screen it left with untried operations, not one branch; raise --max-screens/--max-steps to go deeper, and --workers/--udid run the continuation in parallel (BE-0181). Mutually exclusive with --resume-src/--resume-key |
--config |
bajutsu.config.yaml |
config |
--worker-config |
the shipped default | the worker capability file, as in run; an appium file crawls a target of the grid's platform on the grid |
- A Git
--configis read-only input (BE-0063):crawlreads the config from the fetched checkout, but the screen map / screenshots go to the local--outrun dir (defaultruns/<timestamp>), never into the read-only SHA-keyed cache; an--outinside the checkout is refused. - Traversal is by deterministic replay, not in-place backtracking. To revisit a known screen,
the crawl relaunches the app to a clean start, replays the shortest recorded path to the screen,
and then takes the next untried action, the same way
runreaches any state. - Continue a past crawl (BE-0181):
a crawl stops on
--max-screens/--max-stepsfar more often than oncompleted, sobajutsu crawl --out <existing run> --continuepicks the run's map back up and re-explores every screen it left with untried operations — the whole remaining frontier — instead of re-crawling from the entry screen with a bigger budget. It needs no extra saved state: each such screen's recorded path (paths) is replayed and its still-untried operations re-derived by the deterministic guide (candidate_actions) and matched against the savedplan. Because the reconstruction is deterministic, it reconstructs only the deterministically derivable operations; AI-only operations are intentionally not reconstructed — for an AI-guided run, the richer operations the model proposed (a realisticfill/type, a vision-locatedtap_point) are not re-seeded, only the deterministic taps/inputs are. A frontier that reconstructs to nothing is left untouched (the saved map keeps itsplanand stop reason) rather than reported as completed. Raise--max-screens/--max-stepsto go deeper, and--workers/--udidrun the continuation across the same parallel pool a fresh crawl uses. This is distinct from the single-branch resume the Web UI's pruned-branch "tap to resume" fires (which re-explores one named pruned operation);--continueand that resume are mutually exclusive. - Disabled controls (
notEnabled) are reported per screen asblockedrather than tapped. To enumerate transitions the crawl explores the combinations of control states: it tries each empty text field (and toggles each switch, and switches each tab of a tab bar) independently, and — when several fields are empty — also a compound fill of them at once. The compound matters because a control can stay disabled until several fields are valid, and an intermediate single fill is often invisible (a masked password exposes no value), so filling one at a time can't reach the all-filled state. Crawl is AI-driven: it first inspects the screen deterministically, then hands those operations to Claude to reason about and combine, proposing realistic inputs (a valid email, a password meeting the rules, all of a form's fields in one fill) to enable controls whose precondition isn't obvious, plus operations on id-less elements, narrating its reasoning into the run log. The AI also handles a tab bar whose individual tabs the accessibility tree can't address — the iOS accessibility tree can surface a SwiftUI TabView as a single "Tab Bar" group with no per-tab identifiers, so the bar is visible but its tabs can't be tapped by selector. When that bar is present (and no tab is already addressable by id), it locates the tabs by vision — the same fallback the alert guard uses — and taps each by coordinate, still switching tabs before drilling in. (UIKit tab bars, whose tabs the accessibility tree exposes as individual elements, are a planned refinement — for now they fall back to the same vision path.) The AI only chooses what to try — screen identity, transitions, and crashes stay deterministic, so the crawl is never a verdict (it never gates CI). - Output:
<out>/screenmap.json, a JSON graph ofnodes(screens — fingerprint, kind, ids, candidate actions, plusblockeddisabled controls),edges(transitions),crashes(action paths that collapsed the app UI),alerts(OS prompts the guard dismissed mid-crawl — the triggering path + the button tapped),plan(the live frontier: still-untried operations per screen, refreshed as the crawl advances so a reader can see what it will try next), andstop_reason(completed/max_screens/max_steps), plus<out>/screens/<fingerprint>.png— a screenshot captured for each discovered screen (while the crawl is on it). The map is rewritten as the crawl advances, so a reader (the Crawl tab inserve) can draw it live: each screen is a label+info node, and screens that are the same UI in different states (a form empty vs filled) collapse into one node you can expand in place. Stops at the first of--max-screens/--max-steps. - On completion the crawl also writes
<out>/screenmap.html, a self-contained report (inline CSS, no JavaScript, no external asset) and the offline counterpart to the live Crawl tab: the screens laid out in BFS depth columns with their screenshots, the transitions drawn as a static inline SVG (amber, with a 🛡️ marker, where a step tapped through an OS alert), and the crash / dismissed-alert paths listed below. It sits besidescreenmap.jsonandscreens/, so it opens straight from the run dir — share or archive it without the web UI. Read-only and model-free, like the JSON. - On completion the crawl also writes one
<out>/crashes/crash-NNN.yamlper faithfully reproducible crash. Each file is a repro scenario built from the crash's recorded action path, directly runnable byrun, so a discovered crash becomes a committed Tier 2 regression after human review. The conversion is pure, deterministic, and model-free (tap/type/fillmap to their steps). A path that taps a normalized coordinate (a vision-located control) has no selector to address, so it emits no scenario rather than a lossy one. - Parallel pool (BE-0064):
with
--workers Nover a--udid A,B,Cpool the crawl runs across N booted simulators at once, all sharing the one screen map and frontier — independent branches explore concurrently and the AI guide's round-trips overlap across devices, so wall-clock time falls roughly with the device count. Only scheduling becomes concurrent: which worker reaches a screen first is timing-dependent, so for an app with its own non-determinism the recorded paths and discovery order can vary run to run, but screen identity, transitions, and crashes stay the same deterministic functions of the element tree (the crawl is still never a verdict). A wedged device drops its work and the others carry on. Default--workers 1is the single-worker crawl, unchanged. On web the same model runs N browser processes instead of simulators (BE-0077): the worker count alone sizes the lane set (the web has no devices), each reset is a fresh browser context (the clean-stateerase), and a browser that wedges is torn down and relaunched so one bad browser can't sink the crawl.
How a screen is identified (the fingerprint)¶
Every screen is reduced to a fingerprint, a short, stable identity that lets the crawl tell a
revisit from a genuinely new screen. The reduction is a pure, deterministic function of the element tree (no
AI, no screenshot pixels), which is what keeps the screen map reproducible: the same screen always
hashes to the same value. Each node records both the fingerprint and its kind.
- Identifier fingerprint (
kind: "id", the normal case). The sorted set of accessibility identifiers present on the screen, hashed. Keying on which identifiers are present — not their on-screen text — makes the identity stable across locales and data changes: a list showing different rows, or a label in another language, is still the same screen. Each identifier is then tagged with its interactive state, but only when that state departs from the default, so the same layout in a different state hashes differently and the crawl explores the combinations: !— a disabled control (notEnabled),=— a filled text input (a field that now holds a value),+— a selected / toggled control (a switch on, a tab selected).
An enabled, empty, unselected screen contributes just the bare identifiers, so a screen with no
such state hashes exactly as its plain identifier set would. This is why filling a form or
flipping a switch yields a new node: it is a distinct, separately-explorable state, which is how
the crawl discovers an action that only becomes available once a precondition is met (e.g. a
submit button enabled after every field is filled).
- Structural fingerprint (kind: "structural", the fallback). When a screen carries fewer than
two accessibility identifiers, the id set is too thin to identify it reliably, so the crawl hashes
the actionable elements' traits bucketed by coarse on-screen position instead. This is less
stable — layout jitter can shift it, and unrelated screens with a similar control skeleton can
collide — so it is flagged as structural to signal the identity is approximate. Raising
accessibility-identifier coverage (see doctor) moves a screen back onto the stable id scheme.
- Display grouping is a separate, view-only step. The fingerprint above is the exact, per-state
identity stored in the map. The Crawl tab additionally groups nodes that are the same screen
in a different transient state — an empty vs filled form, a toggle, or an alert/overlay that adds a
few elements — into one expandable unit, so the graph stays readable. That grouping never changes
the fingerprints or what the crawl explores; it only collapses the drawing.
Web backend (--backend web)¶
The crawl runs against a web app the same way it runs against the Simulator — the exploration
engine is platform-neutral, so bajutsu crawl --target <web-app> --backend web produces the same
screenmap.json, screenshots, and crash list. The web app is identified by subject.url (not
subject.id), and because a browser needs no Mac or emulator a web crawl runs on the Linux
make check / CI gate. Three things differ from iOS, and all stay deterministic
(BE-0066):
- Clean start = re-navigate. There is no app process to relaunch; returning to a clean state
is
page.goto(subject.url)against a fresh browser context (theeraseequivalent, ~free), the same lifecycle seamrunuses. - Crash detection. The iOS signal (the accessibility tree collapsing) does not exist on the
web, so the crawl uses the browser's own deterministic signals instead: an uncaught JS exception
(
pageerror), a 4xx/5xx main-frame navigation, or a blank document. Each is a machine fact (an event, a status number, an empty element set) — no model judges whether the page "looks broken". - Dialogs instead of OS alerts. The web has no OS prompts; it has JS dialogs (
alert/confirm/beforeunload). They are auto-handled by a fixed, model-free policy (dismiss) and recorded inalerts, replacing the iOS vision alert guard.--system-alert-handlingand the vision path are iOS-only;--headedapplies (watch the crawl in a visible browser).
repl¶
Opens a manual shell against the running app. In the shell, you read the current element tree, act on one of its
ids, and read the tree again (BE-0423).
Beside record (goal-directed AI authoring) and crawl (autonomous exploration), repl is a third way to reach a target through the same driver
interface. Of the three, only repl asks no model anything and writes no scenario.
Use it to answer, in seconds, the question that otherwise costs a whole run-and-read-the-report
cycle: what does this screen actually expose, and does that id resolve?
bajutsu repl --target <name> [options]
| Option | Default | Description |
|---|---|---|
--target |
(required) | the target app |
--udid |
booted |
the target Simulator; a URL exits 2 (an Appium grid comes from --worker-config) |
--backend |
the target's platform |
a check against the target's platform (see Common) |
--erase / --no-erase |
erase, except on the live route | erase before launch (the app must be installed); --erase raises the effective subject.reinstall to erase and --no-erase lowers an erase to clean, so the target's subject.reinstall: erase does not decide the wipe here; the live route to an appium grid rejects erase outright, so the default there is off, and an explicit --erase on that route exits with a CLI error instead |
--headed / --no-headed |
target driver.headless |
web backend: inspect a visible (headed, slow-motion) browser instead of headless; omit to use the target's driver.headless config |
--browser |
target runsOn.browser.kind (chromium) |
web backend: the browser to inspect — chromium / chrome / edge / firefox / webkit; omit to use the target's runsOn.browser.kind config. On a target that lists its browsers, any other browser exits 2 |
--kind |
the first kind in the target's runsOn.ios.kind |
iOS: the one kind to inspect on — simulator or device. It must be a kind the target declares, or the flag exits 2; a comma list exits 2 too, because the command runs once (BE-0468) |
--attach |
off | local iOS Simulator (xcuitest) alone: connect to the app already running on the booted Simulator and leave it as it is — no erase, install, or relaunch (see Attaching to a running app, below) |
--config |
bajutsu.config.yaml |
config |
--worker-config |
the shipped default | the worker capability file, as in run; an appium file opens the shell on the grid (the live route) for a target of the grid's platform |
Once the app is up, the shell prompts with bajutsu>:
| Command | What it does |
|---|---|
tree |
the current element tree, as an id / label / traits / value / frame table |
tree --json |
the same tree verbatim, as JSON — for piping, diffing, or reading a frame exactly |
find <substring> |
the same tree, filtered to rows whose id or label contains <substring> (case-sensitive, like every selector match) |
tap <target> |
tap the element <target> resolves to, or a raw coordinate (see Targets, below) |
type <target> <text> |
focus <target>, then type <text> |
scroll @<x1>,<y1> @<x2>,<y2> |
a raw one-shot drag/scroll gesture from one coordinate to another (Driver.scroll) |
back |
navigate back one level, each backend using its platform-correct primitive |
screenshot [path] |
write a screenshot; auto-named repl-<UTC timestamp>.png in the current directory when the path is omitted |
step <yaml> |
run one scenario step verbatim — any one-shot action a scenario's own steps accept (swipe, pinch, rotate, setPickerValue, selectOption, and more), none of which has its own shortcut command (see Steps, below) |
help |
list the commands above |
exit / quit |
leave the shell |
clear |
wipe the scroll pane's transcript — the ncurses shell only; the plain fallback (below) has no persistent transcript to wipe, so it reports clear as an unknown command |
Targets (tap/type): the shell reaches the same id / label / index vocabulary a
scenario selector does, plus a raw coordinate that tap alone accepts, bypassing selector
resolution entirely, plus (--sel) the full selector grammar for anything the shortcut forms
below cannot express:
| Form | Matches |
|---|---|
<id> |
the element carrying that id, verbatim — may contain a space for tap (its whole remainder is the id); type needs a quoted target (below) for that |
<id>#<index> |
the <index>-th (0-based; negative counts from the end) of several elements sharing that id |
label:<text> |
an element carrying no id, addressed by its exact label |
label:<text>#<index> |
the <index>-th of several elements sharing that label |
@<x>,<y> |
tap only — a raw pixel coordinate, bypassing the element tree entirely (Driver.tap_point) |
"<target with a space>" |
type only — quote a multi-word label: target so it can be told apart from the text that follows |
--sel <yaml> |
a full selector — --sel {idMatches: row.*, index: 1}, --sel {label: Sign in, within: {id: form.login}} — the same id / idMatches / label / labelMatches / traits / value / within / index fields run accepts, parsed with the scenario's own Selector model. Must be one flow-style {...} mapping (block YAML needs newlines a single typed line cannot hold); the closing } is what tells type's target from its text, the way a quote does for "<target with a space>" |
The shortcut forms above cannot reach an id or label that itself ends in a literal #<digits>,
or starts with @ or label:, and none of them reaches idMatches, labelMatches, traits,
value, or within — --sel is the escape hatch for all of these; the shortcuts exist because
--sel {id: ...} is more to type than tap <id> for the common case.
- The columns are the fields a selector matches against,
normalized by the backend — not a platform inspector's own vocabulary. An id read off a
treerow is the idrunwill resolve. - Nothing is guessed. A
tapwhose target matches no element fails withElementNotFound, and one matching several fails withAmbiguousSelector— immediately, with the same messagerunwould raise (add#<index>to disambiguate, or read the tree again). The shell prints the failure and reads the next line. A raw@<x>,<y>coordinate tap skips this entirely, since it resolves no selector at all. tapdoes not scroll a cover away.runfirst retries a bounded scroll when another element obstructs the target;replsurfaces the driver's ownElementNotTappable, which names the covering element — the more useful answer while diagnosing a selector. The two can therefore disagree on a covered target; write the explicitscrollstep in the scenario to get the recovery.scrollis a raw gesture, not "scroll until visible." A scenario's ownscrollstep always names a target and stops once it is on screen; this shortcut has no target, no stopping condition, and no selector to fail on — it is the same one-shot primitiveDriver.scrollis, nothing more. Reach the target-seeking form, or any other gesture (swipe,pinch,rotate) or platform-specific action (setPickerValue,selectOption), throughstepinstead.- Leaving the shell leaves the app running. A Simulator- or device-backed app stays where you
left it, so you can keep inspecting it by hand. The two sessions the command itself owns are
closed: the web backend's browser, and the WebDriver session on the live route to an
appiumgrid, which would otherwise stay reserved on the grid until it expires. - A target declaring
services.serverhas its server started before the shell opens and stopped when it exits — without that, a web target would open the browser on a host that is not listening and everytreewould read the error page.
Steps (step <yaml>): one flow-style {...} mapping, the exact Step model a scenario file's
own steps validate against — step {swipe: {on: {id: card}, direction: up}},
step {pinch: {sel: {id: map}, scale: 0.5}}. Runs through the same dispatcher run uses for a
one-shot action (tap, type, scroll, swipe, drag, pinch, rotate, setPickerValue,
selectOption, select, copy, paste, handleSystemAlert, and more), so a target it resolves fails the
same way tap's target does (ElementNotFound / AmbiguousSelector). wait, assert, and
control flow (if / forEach / web / app) have no one-shot handler to run through — step
reports the gap by name rather than attempting them; only a whole scenario run has the loop those
need.
The shell itself is an ncurses-style screen when both stdin and stdout are a real terminal —
the command line stays pinned at the top; every command's answer accumulates in a scrollable pane
below it, each answer closed by its own -- EOL -- line so a long scrollback still shows where one
command's output ends and the next begins. A banner on the line right below the input names the
current mode; in the input line and the scroll pane it also spells out the Tab binding, so the
mode switch stays visible without checking this table. The mouse wheel also scrolls the pane, in
every mode, on a terminal that reports it — a keyboard-free alternative to the rows below. And
because that same pane keeps drawing in every mode, a command's answer can land below the current
view unnoticed — while scrolled up in the pane, or typing the next command without having Tab'd
back down — so the banner grows a [new output ↓] suffix (in whichever mode's own text) the moment
that happens, clearing again as soon as the view returns to the bottom:
| Key | In the input line | In the scroll pane |
|---|---|---|
Tab |
switch to the scroll pane | switch back to the input line |
↑ / ↓ |
recall the previous/next command from history | scroll the output up/down one line |
k / j |
typed as ordinary characters | scroll the output up/down one line (vim-style, alongside ↑/↓) |
PgUp / PgDn |
— | scroll the output up/down a full pane |
← / → / t / b |
move the cursor left/right (t/b type as ordinary characters) |
jump straight to the top/bottom |
/ |
— | open a filter prompt; Enter sets it (a case-insensitive substring over the whole transcript), an empty pattern clears it, Esc abandons the edit |
Ctrl-C |
abandon the half-typed line | — |
Piped stdin/stdout (a script, a test harness, bajutsu repl < commands.txt) falls back to a plain
line-at-a-time shell instead — every command above except clear behaves identically either way,
and exit / quit leave the shell in both (Ctrl-D also does, in the plain fallback only — curses
has no Ctrl-D/EOF signal to read).
Attaching to a running app¶
--attach connects the shell to the target app on a booted Simulator, as the operator left it. BE-0455
records the design. Without the flag, the shell starts by launching the app. That launch replaces a
running process with a fresh one. The operator loses the screen they reached and the logged-in state.
A debugger attached from Xcode detaches too. With --attach, the XCUITest runner drives the existing
process. The first tree prints the screen already in front. The app keeps its process id.
The attach start touches nothing device-wide:
- No device prep. The shell never erases, boots, or installs anything. It also skips the locale
pin, the permission grants, and the deeplink.
--attachturns--eraseoff by default.--attach --eraseexits 2. - No readiness wait. The app sits wherever the operator left it. The shell does not wait for the
target's
subject.startWhenscreen. - No retry. The runner starts once, with no recovery ladder. A retry or a repair would end the app or reboot the Simulator. A runner that does not come up exits 2 with its log tail.
- No app teardown. Leaving the shell stops the runner and leaves the app running.
Before the runner starts, the shell checks the device. Before it serves, the runner checks the app:
| Situation | Result |
|---|---|
No Simulator is booted, or the named --udid is shut down |
exit 2: attach never boots a device |
| The target's bundle id has a live process | the runner attaches to that process |
| The app is running, but in the background | exit 2 with the runner's log tail; bring the app to the front first |
| The app is installed but not running | the shell prints <bundle id> was not running; launching it. The runner launches the app with the target's launch env and arguments, without erase or install |
| The app is not installed | exit 2, naming the bundle id |
On every other route, --attach exits 2 before touching any device. Those routes are:
- the
playwrightandadbbackends; - the live route to an
appiumgrid; - a target with
runsOn.ios.kind: device.
--target stays required. The bundle id and the id namespaces come from the target's config.
codegen¶
Generates a native test from a scenario (AI-independent, structural mapping; see codegen): XCUITest (Swift, iOS), Playwright (TypeScript, web), or UI Automator (Kotlin, Android).
bajutsu codegen <scenario.yaml> --target <name> [--emit xcuitest | playwright | uiautomator] [-o <out>] [--config ...]
| Option | Default | Description |
|---|---|---|
--emit |
xcuitest |
output format: xcuitest, playwright, or uiautomator (others exit with code 2) |
-o, --out |
- |
output file. - for stdout |
- Config's
subject.launch.envgoes into the generated test:app.launchEnvironment(XCUITest) or seededlocalStorage(Playwright). --emit playwrightrequires a web target (platform: web, withsubject.url);--emit uiautomatorrequires an Android target (platform: android, withsubject.id); otherwise it exits with code 2.- On file output:
wrote <N> scenario(s) -> <out>.
approve¶
Promotes a run's captured screenshots into visual baselines, the second half of the
visual-regression loop (run, review, approve, re-run). approve reads manifest.json, so it needs
no Simulator and runs headless in CI. It is the CLI twin of the WebUI's Approve button.
bajutsu approve [<run_dir>] --baselines <dir> [--scenario <id>] [--all] [--runs runs]
| Option | Default | Description |
|---|---|---|
<run_dir> |
latest under runs/ |
the run to approve from |
--baselines |
(required) | where to write the promoted baseline PNGs |
--scenario |
"" | only this scenario id (e.g. 00-home, as in the run dir) |
--all |
off | also refresh baselines whose comparison already passed (default: only failing / missing) |
--runs |
runs |
runs root, used when <run_dir> is omitted |
- Copies each visual check's captured
visual-actual.pngto<dir>/<baseline>. Exits 0 when at least one baseline was promoted and 1 when there was nothing to approve.
serve¶
A local web UI to author, run, and explore. It is a Tier-1 convenience and not part of the CI
gate. Four top-level tabs over the CLI: Record authors a scenario from a goal
(python -m bajutsu record ...), Replay runs a scenario and shows its report
(python -m bajutsu.run.internal ..., the body of bajutsu run), Crawl explores the app and draws its screen map live
(python -m bajutsu crawl ...), and Stats shows the aggregate run-stats dashboard across the
run history (bajutsu stats). All of these tabs run against the active config, which you open from
the file browser, a Git repository, or an uploaded .zip bundle (see "Open config" below).
Each request queues a job. A bajutsu worker runs it, since the server runs none itself
(BE-0449). The server streams the job's output. It serves the produced runs/<id>/ tree. The
report's relative asset links resolve against it, as does the crawl's screenmap.json. The server
binds 127.0.0.1.
bajutsu serve [--port 8765] [--config bajutsu.config.yaml] [--root .] [--runs runs] [--baselines <dir>]
[--host 127.0.0.1] [--token <t>] [--max-concurrent-runs 4] [--evidence-store <uri>]
Authentication and security headers¶
--token(or$BAJUTSU_SERVE_TOKEN) enables authentication (BE-0051). With a token set, every request must authenticate: API clients sendAuthorization: Bearer <token>; the browser exchanges the token once viaPOST /api/login(the UI prompts on a 401), which sets an HttpOnly, SameSite session cookie, so the token is never put in a URL. Binding a non-loopback--host(e.g.0.0.0.0) requires a token, so the server is never exposed unauthenticated (it exits otherwise). With no token the server is open, as before, which is safe only on loopback. Full multi-user auth (OAuth/RBAC) is out of scope here (BE-0015).- CSRF + security headers (BE-0051). When a token is set, a state-changing POST whose
Origindoesn't match theHostis rejected (defense-in-depth atop theSameSite=Strictsession cookie); non-browser clients (noOrigin) are unaffected. Every response carriesX-Content-Type-Options: nosniff,X-Frame-Options: DENY, andReferrer-Policy: no-referrer.
Opening a config¶
--configis optional. Omit it and open aconfig.ymlfrom the UI's file browser (an "Open config" button); the browser is confined to--root(default: the current directory).--scenarios <dir>is available as an override of the selected app's configured dir.- From a Git repository (BE-0063).
--configalso accepts a Git source (github:owner/repo@ref:path), and the "Open config" dialog has a From a Git repository field for the same spec: serve materializes the repo subtree at the ref into its cache, binds that config, and serves from the checkout root — so the config's relativepaths.scenarios/subject.path/subject.buildresolve against the fetched tree. This Git binding is the self-hosted payoff (BE-0016 Tier A): point serve at the team's test repository instead of hand-syncing files, and switch branches in the UI rather than redeploying. The file browser stays confined to--root; the checkout is a managed content-addressed cache, and a Git-sourced run confines the config's path fields to the checkout root (BE-0063). --baselinessets the visual-regression baselines dir (default: abaselines/folder under the app's scenarios dir); runs launched from the UI use it, and the report's Approve button promotes the captured screenshot into it viaPOST /api/approve.--themes <dir>is a drop-in theme directory (BE-0191): each*.cssfile adds a selectable UI theme. A theme is declarative — a block of the design tokens documented inbajutsu/templates/serve.themes.css, with a leading/* bajutsu-theme name: … kind: dark|light */manifest comment; the theme's id is the filename stem and its[data-theme="<id>"]selector must match it. No JavaScript, so a dropped-in theme sits at the same trust level as your scenarios and config. The built-inmidnight/daylightpair is always offered; discovered themes extend it. Scanned once at startup (no live reload). An optionalui.default_themein the config sets the initial selection (otherwise the OS scheme is followed until the user picks one).- Pick an app (its scenarios populate the dropdown), set backend / udid / erase /
disable alert-dismiss, hit Run; the output streams live and thereport.htmlembeds on completion. - The Crawl tab picks an app, a pool of simulators (multi-select, like Replay — two or more
crawl in parallel, sharing one screen map; BE-0064),
and a budget (max screens / steps), then
POST /api/crawlspawns the crawl; the returned run id lets the UI pollruns/<id>/screenmap.jsonand draw the screen map as it grows (screens laid out in breadth-first layers, transitions as arrows). The Stop button aborts it, like Replay. - The Stats tab renders the aggregate run-stats dashboard
(BE-0102). The dashboard shows the same
read-only trend as
bajutsu stats, live over the server's run history: pass-rate over time, slowest and flakiest scenarios, failure hotspots, and run volume. It reuses the same deterministic aggregator, reading each run'smanifest.jsonfrom the artifact store (the run-id list comes from the system of record when a database is wired, org-scoped, else the artifact store), with no device, no AI, and no verdict. The server serves the dashboard atGET /stats, and the tab's refresh button refreshes it. - The Flaky tab renders the ranked flaky-scenario surface
(BE-0220).
The tab shows the same read-only ranking as
bajutsu flakiness, live over the server's run history: scenarios sorted flaky-first by verdict flip rate, each row linking to the representative passing and failing runs' evidence. When a database is wired it groups straight from the provenance stamp on each run row (org-scoped); otherwise it builds the same records from each run'smanifest.json. Each row also names the device OS version its runs happened on, and rows are grouped per version (BE-0358). The tab needs no device and no AI, and renders no verdict. The server serves it atGET /flakiness, and the tab's refresh button refreshes it. - Upload a bundle (BE-0073).
The "Open config" dialog has a third source, Upload a bundle, that lets a browser user bring
their own suite to a hosted
servewith no file-system access to the host. Drop a.zipwhose layout is just a working local checkout — abajutsu.config.yaml, itspaths.scenariostree, and the built app binary the config'ssubject.pathnames (a.appdir, a zipped.app, or an.ipa).serveextracts it into a confined sandbox (a sibling ofruns/, never--root) and binds it as the active config, exactly like the file-browser and Git sources — so the Replay / Record / Crawl tabs then run from the extracted tree (the config's relativesubject.path/paths.scenarios/paths.baselinesresolve against the bundle). Runs land inserve's own store (--runs-dir), not the sandbox, so they survive in History; the upload's file name and the zip's sha256 are recorded into each run'smanifest.json(provenance), so "what did this run execute?" stays answerable. Only one bundle is bound at a time — opening another config (any source) removes the sandbox. The bundle carries no secrets:${secrets.*}resolve from theservehost's environment as on any run (BE-0032), and an uploaded config'ssubject.buildcommand is never executed on the host (the bundle ships a prebuilt binary; DESIGN §1). Extraction is hardened against zip-slip (absolute /../ symlink entries are rejected) and zip-bombs (entry-count, total-uncompressed, and per-entry compression-ratio caps); every target's path fields are confined to the bundle at bind; and binding a config is an admin-role action behind the same token auth as every other request — bringing an arbitrary binary is only exposed on an authenticated, single-Macserve.
Composing a config from artifacts¶
- Compose from artifacts (BE-0268).
A combined bundle couples three pieces of very different change cadence — a large binary that
changes every build, a small scenario tree that changes every edit, and a config that almost never
changes — so re-shipping the whole zip for a one-line scenario fix pays for the unchanged binary
again. The "Open config" dialog's fourth source, Compose from artifacts, splits the bundle
into three independently uploadable, content-addressed parts — Config, Scenarios (a
.zipof the scenario tree or a single.yaml, told apart by content and, for a single file, written into the config'spaths.scenariosdirectory with its extension normalized to.yaml), and Binary (a zipped.app, an.ipa, or an Android.apk, extracted or written raw by the config'ssubject.pathsuffix) — each stored by the sha256 of its bytes (POST /api/artifacts/{config,scenarios,binary}). Because a part is addressed by its content, an unchanged part is skipped entirely: the browser hashes the file and asksGET /api/artifacts/existsbefore uploading, so the same binary never travels the wire twice. Compose & load (POST /api/compose) then assembles a chosen(config, scenarios, binary)triple into exactly the same confined tree a combined bundle produces — the config is the sole layout authority, naming via its own relativepaths.scenarios/subject.pathwhere each part lands — and binds it as the active config, so the Replay / Record / Crawl tabs run from it just as with a bundle. A triple whose config needs a part you did not supply is rejected before any run (never a half-built tree). Swapping one leg and composing again runs a new combination (binary A vs. B against the same scenarios, or several scenario sets against one binary) with no re-upload of the shared parts. Each run'smanifest.jsonrecords the triple's provenance as acompositionIdplus one<kind>Shaper part, so "what did this run execute?" is answerable to the exact bytes of each piece. Every artifact upload and the compose step are admin-role actions behind the same token auth, with the same zip-slip / zip-bomb hardening as the combined bundle. (The three-drop-zone UI and this compose endpoint are BE-0268's UI slice; the split-vs-combined internal unification of the combinedPOST /api/uploadis a separate follow-up.) - The AI provider for authoring (Record and Crawl) is one global choice in Settings → AI
provider: Anthropic API (
ANTHROPIC_API_KEY), Amazon Bedrock (AWS credentials +BAJUTSU_BEDROCK_MODEL), the Anthropic CLI (ant, a browser-based OAuth/SSO sign-in on your Pro/Max/Console seat — BE-0163, with its own/api/ant/loginroute), or the Claude Code CLI (claude-code, billed on a Claude Code Pro/Max/Console seat — BE-0176; the panel holds its long-lived OAuth token write-once, for a headless host that can't runclaude setup-tokeninteractively — configuration, BE-0215).serveapplies it to spawned jobs viaBAJUTSU_AI_PROVIDER, so there is no per-tab picker; every AI path (authoring, the alert guard, triage) uses the one provider.
Editing and validating a scenario¶
- Inline scenario validation in the editor (BE-0138).
The Author tab's YAML editor validates as you type, not only on Save: a debounced
POST /api/lintruns the samebajutsu lintchecks and returns line-anchored diagnostics, shown as a gutter marker on each offending line and a clickable problems list (click a finding to jump to its line). YAML parse errors carry the exact line and column; schema/validation errors resolve to their line best-effort. With a target selected, the request carries that target. The server then adds the findings ofbajutsu lint --config --targetagainst the bound config, each on its line (BE-0467). With no target selected, the editor's lint reports nothing new. The scenario JSON Schema (GET /api/schema, the same outputbajutsu schemaprints) also drives lightweight key completion (Ctrl/⌘+Space) and hover descriptions. The validation is deterministic and AI-free, so no device or model runs; Save keeps its own validation guard, so the inline layer only surfaces failures earlier. - If the app's built binary (config
subject.path) is missing, the app'ssubject.buildcommand runs first (its output streams into the job log); a build failure aborts the run before it spawns. Settargets.<name>.subject.buildto the shell command that producessubject.path(e.g.make -C demos/showcase swiftui-build) to build on demand from the UI without a manual build first. - A History list under the controls shows past runs (newest first, with a pass/fail dot and
scenario summary); click one to reopen its report.
GET /api/runsbacks it. - The worker spawns the run subprocess. The subprocess inherits the worker's environment, with the
venv
binprepended toPATH. It also gets the stored secrets the lease relays. A local job runs from its config's directory, sobajutsu.config.yamlresolves. - Input validation on
/api/run. The scenario must be an existing*.yamlinside the selected app's scenarios dir (no arbitrary host paths or..traversal), andbackend/udidmust be known tokens, not free text — so a request can't run an arbitrary file or smuggle surprising argv.backendfollows the same check asrun --backend, and any other value is a 400. This hardening is the prerequisite for hostingservebeyond loopback (BE-0015 / BE-0016); it still binds127.0.0.1and has no auth, so don't expose it to an untrusted network yet. - Running on an Appium grid from
/api/run(BE-0452). The request takes an optionalenvironment: appium, which routes the job to a worker whose worker capability file names anappiumenvironment. Such a job requiresenvironment:appiumalone. Any other value is a 400. - Per-job artifact overrides on
/api/run(BE-0431). Hosted,POST /api/runtakes two more fields.binaryArtifactandscenariosArtifacteach hold the sha256 of an artifact stored for the org. One job installs that binary at its target'ssubject.path. The same job runs against that scenarios zip. The org's active config keeps its binding. The run'smanifest.jsonrecords each overridden sha256 underprovenance. A server with no job queue refuses both fields. See self-hosting.
Run limits, evidence, and hosting¶
--max-concurrent-runs(default 4) caps how many run/record jobs may run at once so one caller can't monopolize the scarce device (BE-0051); dispatch over the cap returns 429. Set0for unlimited.--evidence-store <uri>(or$BAJUTSU_EVIDENCE_STORE) uploads each run's evidence (BE-0110). Point it ats3://bucket/prefixorgs://bucket/prefixand every completed run's tree is uploaded there, keyed<prefix><evidence_prefix><runId>/…so the path selects the cloud lifecycle policy. Unlike the standalonerun --evidence-store(which uploads directly with the runner's own credentials),serveholds the credentials and never hands them to a worker: the control plane signs a presigned PUT URL per file, and the worker uploads over plain HTTP with no cloud SDK or credentials of its own. A caller picks the per-run path by passingevidence_prefixin thePOST /api/runbody (validated as a safe relative segment); the server prepends its own bucket + base prefix, so the run id is always in the key and runs never collide. Upload runs after the verdict, so a failure only warns. Needs thes3orgcsextra on the server (the worker needs neither). See self-hosting for the topology.- Every job runs on a worker (BE-0449).
servequeues every kind of job for abajutsu worker. Theserveprocess runs none of them. A queued job waits for a worker. With no live worker able to take it, the job carries a notice that no worker is available.make servestarts a worker beside the server. A barebajutsu serveneedsbajutsu workerbeside it. The lease hands the worker the stored secrets of the job's org. On one machine, with noBAJUTSU_SERVER_STORE, a server needs no environment variables. It keeps a SQLite queue at.bajutsu/queue.dbbeside the runs directory. It generates a key there with mode0600for signed URLs. It also generates one for stored secrets, unless you provideBAJUTSU_SECRETS_KEY. Its artifact store reads the default org's runs straight from the runs tree, over URLs the server signs. Every other object goes to.bajutsu/objects/. At startup it indexes the runs already underruns/into its history. An admin'sPOST /api/runs/rescanpicks up later ones.BAJUTSU_DATABASE_URL,BAJUTSU_SECRETS_KEY,BAJUTSU_SERVER_STORE, andBAJUTSU_HOSTEDoverride those defaults. A server with no bucket uses the FastAPI transport, as--asgidoes, and needs a fixed--port. A server with a bucket must also setBAJUTSU_DATABASE_URL. Without it, the server would have no queue and refuses to start. See self-hosting → The server backend on one machine. --backend(deprecated). The flag once chose between two backends. Thelocalbackend ran jobs in theserveprocess, and theserverbackend queued them. Both values now start the same server.serveaccepts either value, prints a warning, and ignores it. An unknown value still stops the start with a usage error. The next release removes the flag, so drop it from your command lines.- Hosting flags (advanced).
--emit-launchagentprints a launchd plist to runserveas a token-authenticated LaunchAgent on a single Mac;--upload-exec(or$BAJUTSU_UPLOAD_EXEC) governs whether an uploaded bundle'sservices.servercommand may run (deny/reuse/sandbox, defaultsandbox);--allow-remote-build(or$BAJUTSU_ALLOW_REMOTE_BUILD) opts a UI-bound Git config'ssubject.buildanddriver.runner.buildcommands into running on the host (off by default). All are covered in self-hosting.
mcp¶
Starts a Model Context Protocol (MCP) server so an agent (Claude Desktop or Claude Code) can run
scenarios and read run evidence. Needs the optional bajutsu[mcp] extra (fastmcp).
bajutsu mcp [--config bajutsu.config.yaml] [--runs runs] [--transport stdio] [--worker-config <worker.yaml>]
| Option | Default | Description |
|---|---|---|
--config |
bajutsu.config.yaml |
config the tools resolve targets against |
--runs |
runs |
the runs dir exposed as resources |
--transport |
stdio |
stdio (a local agent) or sse (HTTP) |
--worker-config |
the shipped default | the worker capability file the tools run under, as in run; an appium file sends bajutsu_run and bajutsu_doctor for a target of the grid's platform to the grid |
- Tools:
bajutsu_run(deterministic run) andbajutsu_doctor(convention score). Both mirror the CLI, and no AI enters the verdict. - Resources: a finished run's
manifest.json/report.html/junit.xmland any nested artifact (bajutsu://runs/<id>/…), plusruns/latest.
worker¶
Polls the control plane (bajutsu serve) for queued jobs and runs them. A worker is the half of
every deployment that runs jobs, on one machine as on the hosted server backend
(BE-0015;
self-hosting). It leases a job from /api/worker/lease over plain HTTP (no
Redis or RQ, BE-0106), runs it with the unchanged run_job, and posts the result back to
/api/worker/result. It holds no cloud credentials: every object-store access, whether downloading
baselines before a run or uploading the run tree after, goes through presigned URLs the control
plane signs (BE-0160), so the worker needs only an HTTP client.
A lease with nothing to run answers 204. A worker whose maxJobConcurrency is above 1 — a
device-cloud worker, a Device Farm one included — sends its free slots as maxJobs, from 1 to 100,
and the response is then a list; a one-slot worker sends no maxJobs and gets one body. The oldest
job the worker can serve may belong to a run-set group. The list then adds that group's next jobs,
within the server's concurrency caps. The slots can thus share the group's uploads.
A run-set job carries a run intent instead of a command
(BE-0484). The worker that
leases it turns the intent into its own run. A local worker runs a bajutsu run command. A Device
Farm worker submits its provider's batch.
A local bajutsu serve needs a worker as well, since it queues every run (BE-0449). make serve
starts one. The lease response hands a worker two more fields:
secret_envholds the stored secrets of the leased job's org. Those are the AI key and each${secrets.X}value a user set in the UI. The server reads them at lease time and never stores them in the queue. The worker lays them over the run's environment, never its argv. A lease taken with the shared token receives them, as does a lease on a loopback start with no token. A browser login gets none.- The spec's
cwdnames the bound config's directory, for a job that ships no files of its own. A run on a local start is such a job: it names its config, scenario, runs, and baselines by path. The worker builds and runs the job from that directory. The config's relativesubject.pathandsubject.buildthen resolve as in the server's process. A worker on another machine lacks the directory, so it fails the job with a message saying the job needs a worker on the server's machine.
A SIGINT or SIGTERM stops the worker the way a cancel from the server does. The worker cancels its
leased job; a run gets its grace window to write its report. The worker then posts the cancelled
result and exits with 128 plus the signal number. The queue then records the job as cancelled and does not run it again. A second SIGINT ends
the job's process group at once, as with --once.
A device-cloud worker whose maxJobConcurrency is above one handles a stop signal differently
(BE-0448). It stops leasing, cancels nothing, and posts nothing. Each lease in flight lapses, and a
restarted worker resumes the run. A second SIGINT exits without waiting for the slots.
The Device Farm page says what happens to each run (restart and lease loss).
bajutsu worker [--server-url <url>] [--token <t>] [--platform ios] [--capabilities <tokens>]
[--poll-interval 2.0] [--heartbeat-interval 30.0] [--worker-id <id>]
[--worker-config <worker.yaml>] [--workdir <dir>]
[--machine-id <id> | --server-state-dir <dir>]
bajutsu worker --once <spec.json> [--worker-config <worker.yaml>]
| Option | Default | Description |
|---|---|---|
--once |
off | execute the one job spec in this JSON file on this machine, with no server, and exit with the job's exit code; see below |
--server-url |
$BAJUTSU_SERVER_URL or http://localhost:8765 |
the control plane's URL |
--token |
"" | operator token for auth (the control plane's --token / $BAJUTSU_SERVE_TOKEN) |
--platform |
ios |
comma list of backends this worker can drive (ios / web / android) — the axis the control plane routes jobs on. A local worker exits at startup, naming the driver, when its worker capability file omits a listed platform's driver or its host cannot run that driver, so --platform ios stops a Linux worker (BE-0450). A device-cloud or appium worker takes none |
--capabilities |
"" (also $BAJUTSU_WORKER_CAPABILITIES) |
deprecated (BE-0450): extra capability tokens to advertise beyond the platform, the host, and the Simulator inventory (for example ios18,ipad). Still honored for this release, with a one-time notice, and removed in the next; the inventory and the worker capability file now supply these tokens. A token with the reserved environment:, host:, machine:, device:, cap:, setting:, job:, platform:, or kind: prefix stops the worker at startup |
--poll-interval |
2.0 |
seconds between lease attempts while idle |
--heartbeat-interval |
30.0 |
seconds between lease heartbeats during a run (kept under the control plane's lease timeout, so a long run is never mistaken for a dead worker and reclaimed) |
--worker-id |
"" (auto) | worker identifier |
--machine-id |
"" (also $BAJUTSU_WORKER_MACHINE_ID) |
the server machine's identity, from its .bajutsu/machine-id. The worker advertises it as machine:<id> and so takes the jobs that name files by their paths on the server's machine: a local start's run, triage, record, and crawl (self-hosting). Give it to a worker that sees those paths. A device-cloud worker takes none |
--server-state-dir |
none | the server's state directory (.bajutsu beside its runs directory) to read --machine-id from, for a worker on the server's own machine; make serve passes it. The worker exits at startup when the directory holds no identity yet. Give this flag or --machine-id, not both. This flag wins over $BAJUTSU_WORKER_MACHINE_ID, which stands in for --machine-id alone |
--workdir |
a new temporary directory | where leased jobs run: their materials, fetched bundles, and run trees before upload. The default directory is removed when the worker stops, so a worker started inside a project never writes into it |
--worker-config |
the shipped default | the worker capability file this worker runs under (BE-0450; see drivers). The worker validates it at startup and exits before leasing anything when the file does not load. A file whose environment names a device cloud makes the worker register that provider and advertise environment:<name> (Device Farm). An appium file makes the worker advertise environment:appium and take no --platform; it runs each job as a local bajutsu run that drives the grid (BE-0452). Every worker also advertises job:run-intent, which says it can run a run-set job (BE-0484). The worker passes the file's absolute path to every bajutsu run it spawns, so each leased run is checked against the same file; the server never reads it |
--once executes one job spec without a server
(BE-0449). The spec has
the shape the queue stores: a cmd list, plus the optional fields a queued job carries. The worker
validates --worker-config first and, as it does for a leased job, passes the file's absolute path
to the run it spawns. The job then runs through the same code as a leased job, with these
differences:
- The child process inherits the terminal's standard input, output, and error.
- The worker downloads and uploads nothing, so the run tree stays where the run wrote it.
- The worker reads the run id from the run tree, not from the
PASS|FAILline. - A SIGINT, a SIGTERM, or a SIGHUP cancels the job as a heartbeat cancel does. The exit code is then 128 plus the signal number.
--once refuses some specs before it runs anything, and exits 2:
- a spec that names an uploaded bundle or artifact overrides, or asks for visual baselines
(
materialize_baselines), since those arrive by signed URL and a one-shot job has no server to sign one; point--baselinesat a directory on disk instead - a spec the
--worker-configenvironment does not run: a Device Farm batch spec under alocalorappiumfile, or a plain run spec under a device-cloud file - a
udidsthat is not a list of strings, ormaterialsthat do not map paths to strings - a
cmdthat names a run in neither form a worker recognises (python -m bajutsu.run.internal …or the olderpython -m bajutsu run …) materialswhose path is absolute, contains.., or leaves the current directory through a symbolic linkmaterialsthat would overwrite a different file in the current directory- any spec on Windows, since the signal relay and the liveness pipe need macOS or Linux
The worker writes a spec's materials (the scenario and config a server ships) into the current
directory, as a leased worker writes them into its workspace. A file with the same content is
reused. A file with different content is the caller's own, so --once refuses rather than
overwrite it. Run such a spec from an empty directory.
Only a run spec notices a worker --once that dies without a signal it can catch: a kill -9 of
the worker leaves a record, crawl, or triage child running until it ends on its own.
Submitting a Device Farm job without a server arrives with
BE-0448.
The other options configure the polling worker, and --once ignores them.
lint¶
Validates a scenario file against the grammar without running it, with the same strict validation
that run applies at load. The command exits 0 if the file is valid and non-zero with the error otherwise.
bajutsu lint <scenario.yaml>
bajutsu lint <scenario.yaml> --config <bajutsu.config.yaml> --target <name>
With --config, the lint also checks the scenario against the targets it would run on, the checks
run makes before it leases a device (BE-0467). It resolves the targets the way run
does. The lint checks a scenario that declares its own targets against each of them. It checks a
scenario that declares none against --target. Each target gets three checks on the expanded
scenario:
- Step check. Each step routed to the target needs a capability the target's route has, as in the run's capability preflight.
- Settings check. Each setting the scenario puts in effect must be one the route can apply,
unless the target declares it under
runsOn.ignoredSettings(configuration). - Precondition check. Each
preconditionsfield must exist on a platform the scenario drives.
Each finding names the file, the line, the scenario, and the target. A step that a component
produced reports the line of the use: that pulled the component in. The checks need no device:
the actuator is the first implemented one the target's backend names.
| Exit code | Meaning |
|---|---|
| 0 | No finding |
| 1 | A schema error or a finding |
| 2 | The checks cannot run as asked: a missing config, an unknown target, a scenario with no targets and no --target, or --target without --config |
Without --config, the lint reads no config and behaves as it did before.
schema¶
Prints the scenario JSON Schema to stdout, for editor integration (autocomplete and inline validation). The command takes no options.
bajutsu schema > bajutsu.schema.json
config schema¶
Prints the JSON Schema of bajutsu.config.yaml to stdout, for editor integration. The command takes no options.
bajutsu config schema > bajutsu.config.schema.json
The command generates the schema from the config models and the platform registry. Each target
is a oneOf with one branch per registered platform, and platform is its discriminator. An
editor that reads the schema completes the keys of the platform written in each group. Under
defaults.platforms, the same platforms list every field as optional. Each target completes what a
default leaves out.
The schema checks a target as written, before defaults merges into it. A default may supply
any group, so every group is optional in the schema. A target that names its platform alone
passes the schema. The loader still refuses that target when no default supplies subject.id.
runner¶
Builds the generic XCUITest runner.
Its one subcommand today, build --device, signs the runner for a real iPhone or iPad. The signing
comes from the per-user signing file, and the build lands in a cache. A target with
runsOn.ios.kind: device and no driver.runner.testRunner then resolves to that cache. The build is an
authoring step on macOS with Xcode, and never runs inside bajutsu run.
iOS on a real device covers the signing file, the
cache key, and the errors a run reports.
bajutsu runner build --device [--signing PATH] [--out DIR] [--force]
| Option | Meaning |
|---|---|
--device |
Build the signed device runner. Required today; the flag leaves room for the Simulator build later. |
--signing PATH |
The signing file. Defaults to BAJUTSU_SIGNING_FILE, then ~/.config/bajutsu/signing.yaml (honoring XDG_CONFIG_HOME). |
--out DIR |
Also copy the whole Products directory to DIR, for packaging (the Device Farm route). DIR must be empty or absent. |
--force |
Rebuild even when a matching build is cached. |
A successful build prints the path of the cached runner, with exit status zero. Omitting --device exits with
status 2. Any other outcome exits with status 1 and names the cause. The cause is a missing prerequisite, an error in the
signing file, or a build error.
Environment variables (.env)¶
_bootstrap (@app.callback) loads .env before every command (implementation: bajutsu/dotenv.py).
- The format is
KEY=VALUE. The loader handles#comments, blank lines, anexportprefix, and quotes. - Never overrides an existing environment variable (a real value always wins).
.envis a fallback. - The path defaults to
.env, changeable viaBAJUTSU_DOTENV. It is.gitignored. - Main use:
ANTHROPIC_API_KEY(forrecord,crawl, and the alert guard, which runs by default). - AI provider (BE-0053): set
BAJUTSU_AI_PROVIDER=bedrockto reach Claude through Amazon Bedrock instead of the Anthropic API — authenticated by the standard AWS credential chain (env vars / shared profile / instance or role), so noANTHROPIC_API_KEYis needed on that path. SetBAJUTSU_BEDROCK_MODELto a provider-prefixed model id (e.g.global.anthropic.claude-opus-4-6-v1; the bare Anthropic id is not a valid Bedrock id) andAWS_REGIONfor the region. Anthropic is the default. The same selection drivesrecord,crawl,triage, and the alert guard. - Anthropic CLI (BE-0163): set
BAJUTSU_AI_PROVIDER=ant(orai.provider: ant) to reach Claude through the officialantCLI — runant auth login(a browser-based OAuth/SSO sign-in) so a Claude Pro/Max/Console seat is billed instead of an API key;ANTHROPIC_PROFILEselects a named CLI profile. NoANTHROPIC_API_KEYis needed, and every AI path keeps full vision. BAJUTSU_SIGNING_FILE(BE-0456): the per-user signing file for the device runner, read byrunner build --deviceand by a run on a real device.
# .env — Anthropic (default)
ANTHROPIC_API_KEY=sk-ant-...
# .env — Amazon Bedrock instead (authenticated by your AWS credentials)
BAJUTSU_AI_PROVIDER=bedrock
BAJUTSU_BEDROCK_MODEL=global.anthropic.claude-opus-4-6-v1
AWS_REGION=us-east-1
# .env — Anthropic CLI instead (no API key; run `ant auth login` first)
BAJUTSU_AI_PROVIDER=ant
# ANTHROPIC_PROFILE=work # optional: select a named ant CLI profile
Run diagnostics (BAJUTSU_LOG_LEVEL)¶
BAJUTSU_LOG_LEVEL sets how much a run says about itself. It defaults to WARNING, which is what a
run printed before the knob existed, so turning diagnostics on is always deliberate.
BAJUTSU_LOG_LEVEL=DEBUG uv run bajutsu run --scenario …
Reach for DEBUG when a step fails and the assertion message does not explain why: the timing
failures, where the question is not what the assertion said but which tree the actuator resolved
against. On the adb backend, debug records each read and its device event mark, when the read-lag
barrier opened and what closed it, how the tree settled, the frame each selector resolved to, and
whether a gesture went to the device or fell back to injecting a coordinate. Read together, those
lines say where a tap actually landed and whether the screen underneath it had caught up.
Two of them are warnings rather than debug, because each means an actuator is about to resolve against a screen the driver could not vouch for: the read-lag barrier spending its whole budget, and the settle poll falling through its deadline while the tree was still changing. Both name the evidence they acted on, so a lane failure can be diagnosed from its log rather than re-run.
serve is the exception: it reads the same variable to set its own operational log level and owns a
secret-redacting sink on the root logger (BE-0055), so the CLI leaves its logging alone rather than
adding a second, unredacted one beside it.
Implementation: bajutsu/common/diagnostics.py. The Android e2e lane leaves it
at WARNING — a line per read lengthens the lane — and exposes E2E_LOG_LEVEL to raise it for one
run: make -C demos/showcase/android e2e E2E_LOG_LEVEL=DEBUG.