# Batch backtests `rystic batch` replays every tape in a library through its own simulator with your bot against each, natively or one container per tape, and ends in a per-market P&L table. `rystic batch` replays every tape in a library through its own simulator, your bot against each, and ends in a per-market P&L table. Your bot needs no SDK: it trades over plain HTTP against the address the run hands it ([the bot contract](https://www.rystic.ai/docs/simulators/kalshi/features/bots-and-batches.md#the-bot-contract)). Batch needs `rystic pull kalshi-twin` on this machine, `--runner docker` included, and the [CLI reference](https://www.rystic.ai/docs/cli-reference.md) lists its flags. Start with one tape and a bundled bot, so any later problem is your bot's and not the setup's: ```sh rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -tape KXBTC15M-26JUL231000-00 -out ./fleet -- python3 ~/.rystic/twins/kalshi-twin/0.0.55/random-walker.py ``` `-tape` picks one market from the library, and `-out` is where results land. Everything before `--` is for `rystic`, and everything after it is the command that starts your bot. `rystic list` shows the installed version in the bot's path. These commands read files from the archive and the CLI install, so they're shown as listings rather than examples the docs check runs. The tape plays at real speed, so a 15-minute market takes 15 minutes, with a heartbeat line per slot. A finished run prints its table: ```text note: no -settlements file — markets stay open at tape end, so any P&L in the summary will EXCLUDE their outcomes batch: 1 tapes through kalshi-twin 0.0.55, 2 slots → ./fleet slot 1 ▶ KXBTC15M-26JUL231000-00 (7.0 MB) slot 1 … KXBTC15M-26JUL231000-00 330476/478352 events (69%) · 8m46s · ~3m55s left ✓ KXBTC15M-26JUL231000-00 — 478352 events in 13m59s, bot exited 0 [1/1] · mtm 16.5556 · fees 2.9367 WORLD SETTLED BOT MTM NET FEES MARK TRADED FILLS POSITION BALANCE KXBTC15M-26JUL231000-00 — exit 0 16.5556 10.2646 2.9367 8.9910 104.5620 17 -9.00 10010.2646 TOTAL 16.5556 10.2646 2.9367 8.9910 104.5620 17 1 worlds · 0 failed · 0 settled, 1 left open ⚠ 1 market was never settled — realized columns EXCLUDE their outcomes; mark-to-market columns value the open positions at the LAST RECORDED price, which settlement can still move by up to the position size. Provide -settlements (market → yes|no; outcomes come from the exchange after close, never from the tape) and re-run to realize them exactly. 1 done, 0 failed in 13m59s → fleet/ledger.jsonl peak RSS of one twin-or-bot process, this process's lifetime: 61 MB (measured, not a budget) ``` `MTM` values open positions at the last recorded price, and `NET` is realized. A tape records the book, never how the market resolved. So a market left open has no outcome in its realized columns until `-settlements ` says how each one settled. `rystic report ./fleet` prints the table again at any time, with `-csv`, `-sort ` or `--json`. ### Run your bot on every tape Replace everything after `--` with the command that starts your bot, drop `-tape` and add `-j`: ```sh rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 my-bot.py --market {{market}} --addr {{addr}} ``` `-j 4` runs four tapes at once, each in its own simulator on its own port, with one copy of your bot per tape. So the bot reads `RYSTIC_ADDR` or takes `{{addr}}` rather than hardcoding `localhost:8080`. Type the braces literally, and batch fills them in per run. Without them the arguments after `--` are the same in every slot, so `-j 4` runs four tapes against whichever single ticker you typed. `-dry-run` lists every tape in the plan before any plays, each with its command: `{{market}}` filled in, `{{addr}}` left for the run. **Deriving the event and series tickers.** Batch knows the tape's folder name, not Kalshi's ticker grammar, so a bot that subscribes by event or series derives them itself. For every family with shipped tapes, the market ticker is `--`: ```python market = os.environ["RYSTIC_MARKET"] # KXBTCD-26AUG2017-T73749.99 event = market.rsplit("-", 1)[0] # KXBTCD-26AUG2017 series = market.split("-", 1)[0] # KXBTCD ``` An id that's already an event ticker (`KXDEMO-26`) has no strike segment to drop, and `rsplit` would hand you the series. ### Tapes `-tapes` takes a folder holding an `events` directory with one subfolder per market, named for its market ticker, in the [tape format](https://www.rystic.ai/docs/simulators/kalshi/features/tape-format.md#tape-format). Tapes are too large to ship in the release. Ask Rystic for a sample archive, or write your own from the process that reads Kalshi's WebSocket feed. The archive's `capture-setup.sh` sorts capture files you already have into that layout. It records nothing: ```sh bash ~/.rystic/twins/kalshi-twin/0.0.55/capture-setup.sh ~/Downloads/captures ``` It copies every `.ndjson.gz` under the paths you pass to `events//` under `$KALSHI_CAPTURE_DIR` (default `~/data/kalshi-orderbook`). The ticker comes from each file's first record, so a file needn't be named for its market. A file whose name is already in that market's folder counts as in place and isn't copied, even if its contents differ. A file that isn't gzip is skipped and named. ### Reading the results Under `-out`: - **`ledger.jsonl`** holds one line per tape: the market, how much of the tape played, how it settled, your bot's exit code, whatever it wrote to `RYSTIC_RESULT`, and fingerprints of the tape, simulator and bot. Matching fingerprints mean the same tape, simulator and bot file. They don't cover the bot's environment or `-settlements`, so compare those too. - **`state/`** holds the simulator's complete end state per tape. - **`logs/`** holds the simulator's and your bot's output per tape. ### Each tape in its own container For kernel isolation and no shared ports, the shape that ports to any fleet, put your bot and its runtime on the Rystic image and hand `batch` the result: ```dockerfile FROM registry.rystic.ai/kalshi-twin: USER root RUN apt-get update && apt-get install -y --no-install-recommends python3 && rm -rf /var/lib/apt/lists/* USER rystic COPY my-bot.py /work/my-bot.py ``` ```sh docker build --pull -t kalshi-fleet:local . rystic batch kalshi-twin --runner docker --image kalshi-fleet:local -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 /work/my-bot.py ``` - The `USER` and `RUN` lines install Python, which the image doesn't include. A compiled bot needs only `FROM` and `COPY`, built for Linux and the container's CPU. - `--pull` takes the current `FROM` image rather than a cached one. [Containers](https://www.rystic.ai/docs/simulators/kalshi/features/containers.md) covers the credential Docker pulls it with. - Your bot runs inside the container, so name it by its in-image path (the workdir is `/work`). The image holds a copy of your bot, not a link: after changing it, run `docker build` again. - Each run's container runs `rystic batch -tape ` inside your image, so the bot sees the same contract as natively. Its row, state and logs are copied back under `-out` when it exits, and `logs/.container.log` is the container's own output. - The key or license reaches each container only through its environment. The workers are labeled `ai.rystic.runner=batch`, never appear in `rystic ps` and are removed when the run ends, Ctrl-C included. `docker ps -a --filter label=ai.rystic.runner=batch` shows any a killed run left, and the next run sweeps them. - `--image` must be built from a Rystic image of the same product, and each container runs that image's simulator. Tapes play at speed 1 by design, so throughput is `-j`. - Serving works under any uid, so `--user 1000:1000` in a hardened CI runner is fine. `rystic doctor` inside such a container reports the read-only install as a warning, not a failure. ### When batch refuses | You see | It means | Do this | |---|---|---| | `no tapes in match "events/*/*.ndjson.gz"` | `-tapes` points at the wrong folder | point it at the folder that contains `events/` | | ` names no registry, so it must already exist on this machine` (exit 5) | your `--image` was never built here | `docker build --pull -t .`, since no login fixes this | | `bot did not start: fork/exec … no such file or directory`, or `BOT` says `exit 2` and the bot's log says `can't open file` | the bot or its script is named by a path on your machine, not in the image | use the path from your `COPY` line | | `image is built for , not kalshi-twin` | the `FROM` line names another product | rebuild `FROM registry.rystic.ai/kalshi-twin:` | | a worker exits 2 with no ledger row | the image was built from a stale base | rebuild with `docker build --pull` | | a row says the image's `rystic` is older and would have passed the braces through | your bot's arguments use placeholders, and the image predates them | rebuild the image `FROM` a version that has them | | exit 8, runtime unavailable | Docker is not running | start Docker, or drop `--runner docker` | `rystic doctor` names a broken piece before a run has to discover it. ### Backtest a bot on a recorded day Replay one tape by hand when you want to look at a single run: ```sh curl -s "$RYSTIC_URL/_replay/markets" curl -s -X POST "$RYSTIC_URL/_replay/start" -H 'Content-Type: application/json' -d '{"market":"KXBTC15M-26AUG010000-00","speed":1,"rebase":true}' curl -s "$RYSTIC_URL/trade-api/v2/portfolio/fills" curl -s "$RYSTIC_URL/_rystic/egress?tail=100" ``` At `speed: 1` a bot on the wall clock sees the market it would have seen live. The fills show what it got done and the egress log the frames it saw. [Tape replay](https://www.rystic.ai/docs/simulators/kalshi/features/tape-replay.md) covers the rest of the controls. --- Source: https://www.rystic.ai/docs/simulators/kalshi/features/batch-backtests · Markdown: https://www.rystic.ai/docs/simulators/kalshi/features/batch-backtests.md