Batch backtests

rystic batch replays every tape in a library through its own simulator, your bot against each, and ends in a per-market P&L table. Start with one tape and a bundled bot, so any later problem is your bot’s and not the setup’s:

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -tape KXBTC15M-26JUL231000-00 -out ./fleet -- python3 ~/.rystic/twins/kalshi-twin/0.0.54/random-walker.py

-tape picks one market from the library, and -out is where results land. Everything before -- is for rystic, and everything after it is the command that starts your bot. rystic list shows the installed version in the bot’s path. These commands read files from the archive and the CLI install, so they’re shown as listings rather than examples the docs check runs.

The tape plays at real speed, so a 15-minute market takes 15 minutes, with a heartbeat line per slot. A finished run prints its table:

note: no -settlements file — markets stay open at tape end, so any P&L in the summary will EXCLUDE their outcomes
batch: 1 tapes through kalshi-twin 0.0.54, 2 slots → ./fleet

slot 1  ▶ KXBTC15M-26JUL231000-00 (7.0 MB)
slot 1  … KXBTC15M-26JUL231000-00  330476/478352 events (69%) · 8m46s · ~3m55s left
        ✓ KXBTC15M-26JUL231000-00 — 478352 events in 13m59s, bot exited 0 [1/1] · mtm 16.5556 · fees 2.9367

WORLD                    SETTLED  BOT     MTM      NET      FEES    MARK    TRADED    FILLS  POSITION  BALANCE
KXBTC15M-26JUL231000-00  —        exit 0  16.5556  10.2646  2.9367  8.9910  104.5620  17     -9.00     10010.2646
TOTAL                                     16.5556  10.2646  2.9367  8.9910  104.5620  17
1 worlds · 0 failed · 0 settled, 1 left open
⚠ 1 market was never settled — realized columns EXCLUDE their outcomes; mark-to-market columns value the open positions at the LAST RECORDED price, which settlement can still move by up to the position size.
  Provide -settlements <file> (market → yes|no; outcomes come from the exchange after close, never from the tape) and re-run to realize them exactly.

1 done, 0 failed in 13m59s → fleet/ledger.jsonl
  peak RSS of one twin-or-bot process, this process's lifetime: 61 MB (measured, not a budget)

MTM values open positions at the last recorded price, and NET is realized. A tape records the book, never how the market resolved. So a market left open has no outcome in its realized columns until -settlements <file> says how each one settled. rystic report ./fleet prints the table again at any time, with -csv, -sort <column> or --json.

Run your bot on every tape

Replace everything after -- with the command that starts your bot, drop -tape and add -j:

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 my-bot.py --market {{market}}

-j 4 runs four tapes at once, each in its own simulator on its own port, with one copy of your bot per tape. That’s why the bot reads RYSTIC_ADDR or takes {{addr}} rather than hardcoding localhost:8080. The bot contract lists every value a run hands your bot.

Tapes

-tapes takes a folder holding an events directory with one subfolder per market, named for its market ticker, in the tape format. Tapes are too large to ship in the release. Ask Rystic for a sample archive, or write your own from the process that reads Kalshi’s WebSocket feed.

The archive’s capture-setup.sh sorts capture files you already have into that layout. It records nothing:

bash ~/.rystic/twins/kalshi-twin/0.0.54/capture-setup.sh ~/Downloads/captures

It copies every .ndjson.gz under the paths you pass to events/<market_ticker>/ under $KALSHI_CAPTURE_DIR (default ~/data/kalshi-orderbook). The ticker comes from each file’s first record, so a file needn’t be named for its market. A file whose name is already in that market’s folder counts as in place and isn’t copied, even if its contents differ. A file that isn’t gzip is skipped and named.

Reading the results

Under -out:

• ledger.jsonl holds one line per tape: the market, how much of the tape played, how it settled, your bot’s exit code, whatever it wrote to RYSTIC_RESULT, and fingerprints of the tape, simulator and bot. Matching fingerprints mean the same tape, simulator and bot file. They don’t cover the bot’s environment or -settlements, so compare those too.
• state/ holds the simulator’s complete end state per tape.
• logs/ holds the simulator’s and your bot’s output per tape.

Each tape in its own container

For kernel isolation and no shared ports, the shape that ports to any fleet, put your bot and its runtime on the Rystic image and hand batch the result:

FROM registry.rystic.ai/kalshi-twin:<ver>
USER root
RUN apt-get update && apt-get install -y --no-install-recommends python3 && rm -rf /var/lib/apt/lists/*
USER rystic
COPY my-bot.py /work/my-bot.py

docker build --pull -t kalshi-fleet:local .
rystic batch kalshi-twin --runner docker --image kalshi-fleet:local -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 /work/my-bot.py

• The USER and RUN lines install Python, which the image doesn’t include. A compiled bot needs only FROM and COPY, built for Linux and the container’s CPU.
• --pull takes the current FROM image rather than a cached one. kalshi-fleet:local is yours, built on this machine and never pulled, so a fresh machine needs the docker build before its first fleet batch. Without it, Docker falls through to Docker Hub and returns repository does not exist or may require 'docker login', which no login fixes.
• Your bot runs inside the container, so name it by its in-image path (the workdir is /work). The image holds a copy of your bot, not a link: after changing it, run docker build again.
• Each run’s container runs rystic batch -tape <id> inside your image, so the bot sees the same contract as natively. Its row, state and logs are copied back under -out when it exits, and logs/<world>.container.log is the container’s own output.
• The key or license reaches each container only through its environment. The workers are labeled ai.rystic.runner=batch, never appear in rystic ps and are removed when the run ends, Ctrl-C included. docker ps -a --filter label=ai.rystic.runner=batch shows any a killed run left, and the next run sweeps them.
• --image must be built from a Rystic image of the same product. The outer CLI needs a local rystic pull for the simulator’s manifest, as native batch does. Tapes play at speed 1 by design, so throughput is -j.
• Serving works under any uid, so --user 1000:1000 in a hardened CI runner is fine. rystic doctor inside such a container reports the read-only install as a warning, not a failure.

When batch refuses

rystic doctor names a broken piece before a run has to discover it.

Backtest a bot on a recorded day

Replay one tape by hand when you want to look at a single run:

curl -s "$RYSTIC_URL/_replay/markets"
curl -s -X POST "$RYSTIC_URL/_replay/start" -H 'Content-Type: application/json' -d '{"market":"KXBTC15M-26AUG010000-00","speed":1,"rebase":true}'
curl -s "$RYSTIC_URL/trade-api/v2/portfolio/fills"
curl -s "$RYSTIC_URL/_rystic/egress?tail=100"

speed: 1 is the measurement setting: a bot on the wall clock sees the market it would have seen live. Read the fills for what it got done, and the egress log for the frames it saw. Tape replay covers the rest of the controls.