# Snapshots and test cases

The baseline a run is measured from, snapshots a coding agent rolls back to, scenario files a later run boots from and test cases that grade a later run, all through rystic mcp.

A coding agent that drives a simulator through [`rystic mcp`](https://www.rystic.ai/docs/mcp-server.md) can keep what a run did in four ways. The baseline marks where the run started. A snapshot saves the state to roll back to. A scenario file writes the state out for a later run to boot from. A test case keeps the run as a regression test and grades later runs against it.

## The baseline

The baseline is the state `diff_state` compares against: the state the instance had when `start_twin` started it, `load_scenario` loaded a scenario, `reset` reset it or `restore_twin` restored a snapshot. `seed` leaves the baseline in place, so seeded rows read as created.

For an instance started outside the server, with `rystic run` say, the first `diff_state` stores the current state as the baseline. It returns `"baseline": "first_seen"` with an empty diff. The [baseline resource](https://www.rystic.ai/docs/mcp-reference.md#resources) holds the baseline whole.

## Snapshots

`snapshot_twin` with a `name` saves the instance's state, clock, id counters and pending scheduled events and, when your plan includes network shaping, the network profile. A snapshot of the same name is replaced, and the result's `overwrote` says so. Names take letters, digits, `.`, `_` and `-`, at most 64, and may not be a Windows device name such as `CON`, `NUL` or `COM1`, in any case or with any extension. Without a `name` it lists the instance's snapshots.

`restore_twin` resets the instance, then lands the snapshot, so ids minted after the restore continue where the snapshot left off. The restored state becomes the baseline. A snapshot taken on another version of the simulator is refused with `snapshot_version_mismatch`.

The CLI keeps snapshots on this machine, per simulator and instance name. They survive `stop_twin` and a restart of the same instance on the same version.

A snapshot does not hold, so a restore does not bring back:

- the idempotency cache, so a replayed request makes a new object;
- the request-id counter;
- the calls and sent logs, which start empty;
- the webhook delivery queue;
- credential bindings;
- upload sessions;
- the rate limiter;
- the clock's wall offset;
- scheduled events in full: only the pending events of the scenario's own schedule come back, less any that uses a value a fired event bound. A schedule loaded on its own replays from its start;
- built-in records deleted before the snapshot, which the reset brings back;
- whether a scenario installed the network profile: once restored, it stays through later resets.

`snapshot_twin` refuses an instance with a pending event that acts as a named caller (`snapshot_unsupported`) and a state too large to restore (`snapshot_too_large`). When the simulator refuses a snapshot after the reset, `restore_twin` fails with `restore_failed` and leaves the instance reset. On an older simulator release, every `restore_twin` result carries a `warning`: ids minted after the restore can collide with restored records, and per-account records can come back duplicated.

## Scenario files

`save_world` with `instance` and `file` (an absolute path) writes the instance's state as a [scenario file](https://www.rystic.ai/docs/concepts/glossary.md#scenario-file). The file holds what a snapshot holds: the state, clock, id counters, pending scheduled events and network profile. It is the document `rystic run <twin> --seed <file>` boots and `POST /_rystic/seed` lands.

Check the file into your repo, and a teammate or a CI job starts the simulator in that state. `seed` with the file's contents as `body` lands it on a running instance. `rystic scenario save <instance> <file>` writes the same file from the command line.

The file names the simulator (`twin`) but no instance, version or model hash, so it loads on a later release of the simulator. A record that release cannot take fails the load with the `400` the seed route returns, which lists the records. Ids minted after the load continue where the file left off, as they do after a restore.

An existing file is refused with `cannot_write` unless `force` is true, which replaces it, through a symlink. The result reports the `file`, its `bytes`, `rows`, `events`, `sequences` and `clock`, and whether a `network` profile was saved. The file carries a profile only when one is installed, so it loads on a plan without network shaping. On an older simulator release the file holds no id counters, and the result carries a `warning`: ids minted after a load can collide with loaded records.

The file leaves out what a snapshot leaves out, listed above. It holds no `rng_seed` and no `credentials` either, since the simulator does not expose them. Add those keys to the file by hand when a run needs them, in the shape the [boot seed file](https://www.rystic.ai/docs/control-plane.md#boot-inputs) takes. `save_world` refuses what `snapshot_twin` refuses: a pending event that acts as a named caller (`snapshot_unsupported`) and a state too large for the seed route (`snapshot_too_large`).

## Test cases

### Save a run as a test case

`save_test_case` turns the run an agent just made into a regression test. It writes a grid cell directory, the same shape `rystic grid` writes: `cell.json` and, per simulator, `twins/<grid name>/before.json`, `after.json`, `calls.json` and `egress.json`, numbers exact. The grid name is the simulator's name without `-twin` (`linear` for `linear-twin`), as `rystic grade` and the eval grid name it. Then it drafts the eval's deterministic checks with `rystic grade --draft` into the cell's `eval.yaml`, so `rystic grade <cell_dir>` passes on this run and fails on a later run that differs.

The [MCP tool reference](https://www.rystic.ai/docs/mcp-reference.md#save_test_case) lists its arguments. It returns `{name, cell_dir, eval, draft, twins, notes}`; each `twins` entry carries `instance`, `twin`, `scenario`, `baseline`, `baseline_taken_at`, `seeded` and `counts` (`created`, `changed`, `deleted`).

- **Before** is the instance's [baseline](#the-baseline). Records seeded through `seed` after the baseline count as setup, not as the run's. An instance with no baseline is refused with `no_baseline`.
- **After** is the state now, with the calls and sent log since the last reset.
- Rows a scenario's own timeline seeds during the run count as the run's changes.
- The draft expects each row the run created or changed, forbids any other creation, change, caller or method, and allows each deletion the run made. It is a starting point: trim the checks to what the task requires before keeping them.

### Check a later run

`check_test_case` grades a new run against a saved case, as `rystic grade` does, with no model. After `save_test_case`, `reset` or `load_scenario` the instance and run the agent again. Then call `check_test_case` with `case` set to the `cell_dir` or `eval` path `save_test_case` returned, or any eval file, as an absolute path. It captures the run since the baseline into a new cell, under `out_dir` or a new temp directory, grades it and writes its `verdict.json`.

| Field | Meaning |
|---|---|
| `result` | `pass`, `fail`, `error`, or `incomplete` when every graded check passed but one was skipped |
| `reason` | why the case got that result |
| `cell_dir` | the new run's cell, with its `verdict.json` |
| `checks` | per check: its `id`, `result` (`pass`, `fail`, `error` or `skipped`), `detail` and, for a failure, the rows behind it |
| `notes` | anything to know about the run, such as a baseline scenario that differs from the case's |

A case directory's `checks/` YAML files are graded too. A script check is skipped, not run. The case's `world` must name exactly the instances' simulators, by full name (`linear-twin`) or grid name (`linear`).

---

Source: https://www.rystic.ai/docs/concepts/snapshots-and-test-cases · Markdown: https://www.rystic.ai/docs/concepts/snapshots-and-test-cases.md