Backtesting with batch

rystic batch replays captured markets — tapes — one simulator per tape, your bot against each, -j at a time. Per tape: a ledger row, the simulator’s full end state, and both logs.

No SDK, no language requirement. Your bot reads RYSTIC_ADDR from the environment and trades over plain HTTP — the same calls as the live exchange.

Before you start

rystic pull kalshi-twin
rystic doctor

You also need tapes — a folder containing an events directory, one subfolder per market:

$ ls ~/data/kalshi-orderbook/events
KXBTC15M-26JUL231000-00    KXBTC15M-26JUL231015-00    ...

Those subfolder names are market ids. No tapes? Ask us for a sample archive, or record your own with the capture-setup.sh that ships with the simulator.

What your bot reads

base = "http://" + os.environ["RYSTIC_ADDR"]        # RYSTIC_ADDR is e.g. localhost:52114

That’s the whole integration: wherever your bot sets the Kalshi base URL, use http:// + RYSTIC_ADDR.

Exit 0 is a pass. Non-zero is recorded, not fatal — the run continues and the code is data in your results. A bot still running when the tape ends is stopped for you.

If your bot takes flags instead of environment variables

The same values reach your bot’s own command line. Anywhere after --, batch substitutes {{addr}}, {{world}}, {{world_kind}}, {{result}} and {{market}} per run:

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet \
  -- python3 my-bot.py --market {{market}} --url {{addr}}

Type the braces literally — batch fills them in, one value per run. Without them the arguments after -- are byte-identical in every slot, so -j 4 runs four tapes against whichever single market you typed: three of the four wrong, and nothing in the output says so.

-dry-run prints the resolved command for every tape, which is the cheapest place to check all four before spending an hour of tape on one:

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -dry-run \
  -- python3 my-bot.py --market {{market}}

Three things it refuses rather than guessing:

• A name that isn’t one of the five — a misspelled market, say. Refused before anything boots, because left alone it would reach your bot as those literal characters and the run would look like it worked.
• {{market}} where there is no market — rystic grid and batch -scenarios run the simulator’s own worlds, which aren’t a replayed market. The environment simply leaves RYSTIC_MARKET unset there; a command line has no way to say “unset”, and --market "" is an argument your bot can’t tell from a real one. Use {{world}}.
• An image that predates this — under --runner docker the substitution happens inside your image, so batch tells the container it needs a rystic that can do it. Too old an image fails its run naming the rebuild, rather than handing your bot the literal braces and finishing green.

{{market}} is the market ticker your tape folder is named for (KXBTCD-26AUG2017-T73749.99). A bot that subscribes by event or series derives those from it — market.rsplit("-", 1)[0] and market.split("-", 1)[0] — because batch knows the folder name, not Kalshi’s ticker grammar.

Your first backtest

Start with one tape and the bundled example bot, so every later problem is your bot’s, not the setup’s.

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook \
  -tape KXBTC15M-26JUL231000-00 -out ./fleet \
  -- python3 ~/.rystic/twins/kalshi-twin/0.0.25/random-walker.py

1. -tapes is the folder containing events/; -tape is one market id from it — drop it to run every tape; -out is where results land.
2. Everything before -- is for rystic, everything after is the command that starts your bot. The version in the path is yours to check — rystic list shows what’s installed.

The tape plays at real speed — a 15-minute market takes 15 minutes, with a heartbeat line per slot. Success:

batch: 1 tapes through kalshi-twin 0.0.25, 2 slots → ./fleet

slot 1  ▶ KXBTC15M-26JUL231000-00 (7.0 MB)
slot 1  … KXBTC15M-26JUL231000-00  330476/478352 events (69%) · 8m46s · ~3m55s left
        ✓ KXBTC15M-26JUL231000-00 — 478352 events in 13m59s, bot exited 0 [1/1] · mtm 16.5556 · fees 2.9367

WORLD                    SETTLED  BOT     MTM      NET      FEES    MARK    TRADED    FILLS  POSITION  BALANCE
KXBTC15M-26JUL231000-00  —        exit 0  16.5556  10.2646  2.9367  8.9910  104.5620  17     -9.00     10010.2646
TOTAL                                     16.5556  10.2646  2.9367  8.9910  104.5620  17

1 worlds · 0 failed · 0 settled, 1 left open
⚠ 1 market was never settled — realized columns EXCLUDE their outcomes; mark-to-market columns value the open positions at the LAST RECORDED price, which settlement can still move by up to the position size.
  Provide -settlements <file> (market → yes|no; outcomes come from the exchange after close, never from the tape) and re-run to realize them exactly.
1 done, 0 failed in 13m59s → fleet/ledger.jsonl

mtm marks open positions at the last recorded price; net is realized. A capture records the book, never how the market resolved — so a market left open has no outcome in its realized columns until -settlements <file> says how each one settled. rystic report ./fleet re-renders the table any time, with -csv, -sort <column>, --json.

Then swap in your own bot: replace everything after -- with however you start it.

To run every tape, drop -tape and add -j:

rystic batch kalshi-twin -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 my-bot.py

-j 4 runs four tapes at once, each in its own simulator on its own port. Batch starts one copy of your bot per tape with RYSTIC_ADDR already pointing at that tape’s simulator — which is why the bot must read it rather than hardcode localhost:8080. The same goes for the market: pass {{market}} rather than a fixed ticker, or all four copies trade the one you typed.

Reading the results

Under -out:

1. ledger.jsonl — one line per tape: the market, how much played, how it settled, your bot’s exit code, anything it wrote to RYSTIC_RESULT, plus fingerprints of tape, simulator and bot. Two rows with the same fingerprints are the same experiment.
2. state/ — the simulator’s complete end state per tape.
3. logs/ — the simulator’s and your bot’s output, per tape.

The same thing in containers

Optional — for kernel isolation, CI, or handing the fleet to a test platform. Result files are identical.

Build an image with your bot inside:

FROM registry.rystic.ai/kalshi-twin:0.0.25
USER root
RUN apt-get update && apt-get install -y --no-install-recommends python3 && rm -rf /var/lib/apt/lists/*
USER rystic
COPY my-bot.py /work/my-bot.py

docker build --pull -t kalshi-fleet:local .

Your license (from rystic login) is what lets docker pull the base image. The USER/RUN lines install Python — our image doesn’t include it. A compiled bot needs only FROM and COPY, built for Linux and the container’s CPU.

Then the same batch command with two additions — and note the bot path is now the in-image path:

rystic batch kalshi-twin --runner docker --image kalshi-fleet:local \
  -tapes ~/data/kalshi-orderbook -j 4 -out ./fleet -- python3 /work/my-bot.py

The license reaches each container through its environment, never the image. Workers are labelled, cleaned up when the run ends (Ctrl-C included), and never clutter rystic ps. Changed your bot? Re-run docker build — the image holds a copy, not a link.

When it refuses

Still stuck? rystic doctor names the broken piece before a run has to discover it.