Snapshots and test cases

A coding agent that drives a simulator through rystic mcp can keep what a run did in four ways. The baseline marks where the run started. A snapshot saves the state to roll back to. A scenario file writes the state out for a later run to boot from. A test case keeps the run as a regression test and grades later runs against it.

The baseline

The baseline is the state diff_state compares against: the state the instance had when start_twin started it, load_scenario loaded a scenario, reset reset it or restore_twin restored a snapshot. seed leaves the baseline in place, so seeded rows read as created.

For an instance started outside the server, with rystic run say, the first diff_state stores the current state as the baseline. It returns "baseline": "first_seen" with an empty diff. The baseline resource holds the baseline whole.

Snapshots

snapshot_twin with a name saves the instance’s state, clock, id counters and pending scheduled events and, when your plan includes network shaping, the network profile. A snapshot of the same name is replaced, and the result’s overwrote says so. Names take letters, digits, ., _ and -, at most 64, and may not be a Windows device name such as CON, NUL or COM1, in any case or with any extension. Without a name it lists the instance’s snapshots.

restore_twin resets the instance, then lands the snapshot, so ids minted after the restore continue where the snapshot left off. The restored state becomes the baseline. A snapshot taken on another version of the simulator is refused with snapshot_version_mismatch.

The CLI keeps snapshots on this machine, per simulator and instance name. They survive stop_twin and a restart of the same instance on the same version.

A snapshot does not hold, so a restore does not bring back:

• the idempotency cache, so a replayed request makes a new object;
• the request-id counter;
• the calls and sent logs, which start empty;
• the webhook delivery queue;
• credential bindings;
• upload sessions;
• the rate limiter;
• the clock’s wall offset;
• scheduled events in full: only the pending events of the scenario’s own schedule come back, less any that uses a value a fired event bound. A schedule loaded on its own replays from its start;
• built-in records deleted before the snapshot, which the reset brings back;
• whether a scenario installed the network profile: once restored, it stays through later resets.

snapshot_twin refuses an instance with a pending event that acts as a named caller (snapshot_unsupported) and a state too large to restore (snapshot_too_large). When the simulator refuses a snapshot after the reset, restore_twin fails with restore_failed and leaves the instance reset. On an older simulator release, every restore_twin result carries a warning: ids minted after the restore can collide with restored records, and per-account records can come back duplicated.

Scenario files

save_world with instance and file (an absolute path) writes the instance’s state as a scenario file. The file holds what a snapshot holds: the state, clock, id counters, pending scheduled events and network profile. It is the document rystic run <twin> --seed <file> boots and POST /_rystic/seed lands.

Check the file into your repo, and a teammate or a CI job starts the simulator in that state. seed with the file’s contents as body lands it on a running instance. rystic scenario save <instance> <file> writes the same file from the command line.

The file names the simulator (twin) but no instance, version or model hash, so it loads on a later release of the simulator. A record that release cannot take fails the load with the 400 the seed route returns, which lists the records. Ids minted after the load continue where the file left off, as they do after a restore.

An existing file is refused with cannot_write unless force is true, which replaces it, through a symlink. The result reports the file, its bytes, rows, events, sequences and clock, and whether a network profile was saved. The file carries a profile only when one is installed, so it loads on a plan without network shaping. On an older simulator release the file holds no id counters, and the result carries a warning: ids minted after a load can collide with loaded records.

The file leaves out what a snapshot leaves out, listed above. It holds no rng_seed and no credentials either, since the simulator does not expose them. Add those keys to the file by hand when a run needs them, in the shape the boot seed file takes. save_world refuses what snapshot_twin refuses: a pending event that acts as a named caller (snapshot_unsupported) and a state too large for the seed route (snapshot_too_large).

Test cases

Save a run as a test case

save_test_case turns the run an agent just made into a regression test. It writes a grid cell directory, the same shape rystic grid writes: cell.json and, per simulator, twins/<grid name>/before.json, after.json, calls.json and egress.json, numbers exact. The grid name is the simulator’s name without -twin (linear for linear-twin), as rystic grade and the eval grid name it. Then it drafts the eval’s deterministic checks with rystic grade --draft into the cell’s eval.yaml, so rystic grade <cell_dir> passes on this run and fails on a later run that differs.

The MCP tool reference lists its arguments. It returns {name, cell_dir, eval, draft, twins, notes}; each twins entry carries instance, twin, scenario, baseline, baseline_taken_at, seeded and counts (created, changed, deleted).

• Before is the instance’s baseline. Records seeded through seed after the baseline count as setup, not as the run’s. An instance with no baseline is refused with no_baseline.
• After is the state now, with the calls and sent log since the last reset.
• Rows a scenario’s own timeline seeds during the run count as the run’s changes.
• The draft expects each row the run created or changed, forbids any other creation, change, caller or method, and allows each deletion the run made. It is a starting point: trim the checks to what the task requires before keeping them.

Check a later run

check_test_case grades a new run against a saved case, as rystic grade does, with no model. After save_test_case, reset or load_scenario the instance and run the agent again. Then call check_test_case with case set to the cell_dir or eval path save_test_case returned, or any eval file, as an absolute path. It captures the run since the baseline into a new cell, under out_dir or a new temp directory, grades it and writes its verdict.json.

A case directory’s checks/ YAML files are graded too. A script check is skipped, not run. The case’s world must name exactly the instances’ simulators, by full name (linear-twin) or grid name (linear).