# S3 simulator

A stateful Amazon S3 simulator — buckets, objects, listings, multipart uploads, copies, tagging, conditional requests and checksums for storage code and the agents that drive it.

`s3-twin` is a local, stateful simulator of Amazon S3's REST API — buckets, objects under any key, listings, multipart uploads, copies, tagging, batch deletes, conditional and Range requests and request checksums, 32 operations in all. It never calls AWS: no credential that works anywhere else is involved, and no bucket fills up in a real account.

## Quickstart

You need an API key from [Account → API keys](https://www.rystic.ai/account/api-keys) ([how](https://www.rystic.ai/docs/licensing.md#get-an-api-key)).

```bash
curl -fsSL https://www.rystic.ai/install.sh | sh
rystic login rys_…                                           # your API key
rystic run s3-twin -d --port 0 --scenario s3_static_site     # prints the URL it serves on
curl -s http://localhost:<port>/_rystic/version
```

Expect `"product":"s3-twin"` and `"version":"0.0.1"`. An older version means an older install; `rystic pull s3-twin` upgrades it. `--scenario s3_static_site` boots the same world on every plan. The examples below use `localhost:8080`; use the port `rystic run` printed. The archive also carries `check.sh`, `HOWTO.md` and `SCENARIOS.md`.

On Enterprise, `rystic login ./rystic-license.json` takes the place of the key ([license files](https://www.rystic.ai/docs/licensing.md#enterprise-license-files)).

List the site's keys, then put one of your own and read it back:

```bash
AUTH='Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20260201/us-east-1/s3/aws4_request, SignedHeaders=host, Signature=placeholder'
curl -s -H "$AUTH" 'localhost:8080/acme-demo-site?list-type=2&prefix=assets/'
# -> <ListBucketResult ...><Contents>...<Key>assets/app.css</Key>...</Contents>...<KeyCount>3</KeyCount>...

curl -s -X PUT -H "$AUTH" -H 'Content-Type: text/plain' --data 'hello' localhost:8080/acme-demo-site/reports/2026/hello.txt
curl -s -H "$AUTH" localhost:8080/acme-demo-site/reports/2026/hello.txt
# -> hello
```

Point the SDK at it — an endpoint override, path-style addressing and any access key pair, code unchanged:

```python
s3 = boto3.client("s3", endpoint_url="http://localhost:8080", region_name="us-east-1",
                  aws_access_key_id="AKIAIOSFODNN7EXAMPLE", aws_secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
                  config=Config(s3={"addressing_style": "path"}))
```

```bash
# AWS CLI v2: the endpoint and credentials from the environment, path-style from the config file
printf '[default]\nregion = us-east-1\ns3 =\n    addressing_style = path\n' > aws-config
AWS_CONFIG_FILE=$PWD/aws-config AWS_ENDPOINT_URL=http://localhost:8080 AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
  AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY aws s3 ls s3://acme-demo-site/ --recursive
```

## Plans

What `s3-twin` does on each plan ([plans](https://www.rystic.ai/docs/licensing.md#plans)):

| | Free | Pro | Enterprise |
|---|---|---|---|
| Credential | API key | API key | license file |
| Boots | `s3_static_site` | an empty account, or the `--scenario` you name | same as Pro |
| `POST /_rystic/reset` | re-applies `s3_static_site` | wipes to empty | wipes to empty |
| S3 REST API (all 32 operations) | yes | yes | yes |
| Control plane | `version`, `routes`, `reset` only | every route | every route |

**Free** runs 5 hours of simulator time a month. It serves the whole S3 surface against `s3_static_site`. Every control route beyond `GET /_rystic/version`, `GET /_rystic/routes` and `POST /_rystic/reset` answers 403, naming the feature it needs (the loopback-only `GET /_rystic/activity` aside):

```json
{"feature":"state_set","message":"/_rystic/seed needs the \"state_set\" feature, which this plan does not include; upgrade at https://www.rystic.ai/pricing","upgrade_url":"https://www.rystic.ai/pricing"}
```

Free also refuses these boot inputs, and the twin exits naming the feature: `--scenario` naming any other world (`scenarios`), `--seed` or `RYSTIC_SEED` (`state_set`), and `RYSTIC_NETWORK` (`faults`). See [boot inputs](https://www.rystic.ai/docs/control-plane.md#boot-inputs).

**Pro** opens every control route below and all five worlds. Without `--scenario` the twin boots an **empty** account, and `POST /_rystic/reset` wipes it back to empty. An empty account has no bucket, so every object call answers `NoSuchBucket` until you create one or load a world.

**Enterprise** is Pro on a license file instead of an API key.

## Auth

S3 takes AWS Signature Version 4 in the `Authorization` header. The twin accepts any header that starts `AWS4-HMAC-SHA256 ` and checks nothing else — no key, no signature, no expiry — so AWS's documented example key pair signs every request. A request with no header or another scheme answers 403 `AccessDenied`; that refusal is not measured against S3, and presigned URLs are not served.

## Coverage

32 operations, path-style, on `/`, `/{Bucket}` and `/{Bucket}/{Key+}`, each told apart by its query parameters and headers as S3 tells them apart. `GET /_rystic/routes` returns the list from a running twin.

| Family | Operations |
|---|---|
| Objects | `PutObject`, `GetObject`, `HeadObject`, `DeleteObject` |
| Copies | `CopyObject`, `UploadPartCopy` |
| Multipart | `CreateMultipartUpload`, `UploadPart`, `ListParts`, `CompleteMultipartUpload`, `AbortMultipartUpload`, `ListMultipartUploads` |
| Object tagging and reads | `GetObjectTagging`, `PutObjectTagging`, `DeleteObjectTagging`, `GetObjectAttributes`, `GetObjectAcl` |
| Batch delete | `DeleteObjects` |
| Listings | `ListObjectsV2`, `ListObjects`, `ListObjectVersions` |
| Bucket reads | `HeadBucket`, `GetBucketLocation`, `GetBucketVersioning`, `GetBucketAcl`, `GetBucketTagging` |
| Bucket lifecycle | `CreateBucket`, `ListBuckets`, `DeleteBucket` |
| Bucket writes | `PutBucketVersioning`, `PutBucketTagging`, `DeleteBucketTagging` |

A key may hold `/`; object bodies are bytes up to 64 MiB, and a larger one answers `400 EntityTooLarge`. Request checksums (`x-amz-checksum-crc32`, `-crc32c`, `-crc64nvme`, `-sha1`, `-sha256`), which the AWS SDKs send by default, are verified, stored and echoed under `x-amz-checksum-mode: ENABLED`; a mismatch answers `400 BadDigest`. `If-Match` and `If-None-Match` on `PutObject` answer `412 PreconditionFailed`; `If-None-Match` and `If-Modified-Since` on `GetObject` and `HeadObject` answer `304`; `Range` answers `206` with its `Content-Range`. `x-amz-expected-bucket-owner` passes for the owning account (`123456789012`) and answers 403 for any other.

Not modelled, on purpose: virtual-hosted addressing, presigned URLs, `versionId` addressing, ACL writes, and every bucket setting but versioning and tags. An unmodelled call on a modelled path — policy, CORS, lifecycle, encryption, website, replication; object legal hold, retention, restore, select — answers `501 NotImplemented` in S3's XML error body and changes nothing.

## Control plane

Every route under `/_rystic/`, and the plan that reaches it. [Routes by plan](https://www.rystic.ai/docs/control-plane.md#routes-by-plan) covers the whole control plane.

| Route | What it does | Plan (feature) |
|---|---|---|
| `GET /_rystic/version` | health check: product, version, model hash, edition, `auth_scheme`, `blob_cap_bytes`, `initial_clock` | every plan |
| `GET /_rystic/routes` | the S3 routes this twin serves; not the control plane | every plan |
| `POST /_rystic/reset` | Free: back to `s3_static_site`. Pro, Enterprise: wipe every bucket and object, rewind the id sequence and the clock | every plan |
| `GET /_rystic/scenarios` | the shipped worlds | Pro, Enterprise (`scenarios`) |
| `POST /_rystic/scenario` | `{"name":"s3_versioned_archive"}`: reset, then load the world | Pro, Enterprise (`scenarios`) |
| `GET /_rystic/inspect` | dump the entire state | Pro, Enterprise (`state_view`) |
| `GET /_rystic/registers` | every seedable register and its fields | Pro, Enterprise (`state_view`) |
| `GET /_rystic/egress` | the outbound effect log; the twin sends no notifications, so it is always empty | Pro, Enterprise (`state_view`) |
| `POST /_rystic/seed` | land a register map | Pro, Enterprise (`state_set`) |
| `POST /_rystic/clock` | `{"time":"<RFC3339>"}` pins the clock, `{"advance_seconds":N}` moves it | Pro, Enterprise (`state_set`) |
| `POST /_rystic/seed-rng` | `{"seed":N}` fixes the RNG | Pro, Enterprise (`state_set`) |
| `GET`, `POST /_rystic/network` | read or set a network profile: rejected requests and latency ([fault injection](https://www.rystic.ai/docs/control-plane.md#network-profiles-and-fault-injection)) | Pro, Enterprise (`faults`) |
| `GET /_rystic/ui` | the state UI | Pro, Enterprise (`ui`) |
| `GET /_rystic/activity` | the usage counters the CLI reads; any other caller gets 403 | every plan, loopback callers only |

## Scenarios

Five synthetic worlds ship, every one the `acme-demo` account's at `2026-02-01T00:00:00Z`. `s3_static_site` is the one Free boots.

| World | What is in it |
|---|---|
| `s3_static_site` | `acme-demo-site`, an unversioned website bucket: `index.html`, `about/index.html`, `assets/app.css`, `assets/app.js`, `assets/logo.svg`, `robots.txt` and a zero-byte `legacy.html`, each with its `Content-Type`, `Cache-Control` and SHA-256 checksum; beside it `acme-demo-site-logs` with two access logs under `access/` |
| `s3_versioned_archive` | `acme-demo-archive`, versioning enabled: six version rows across three keys — one overwritten once, one single-version, and `notes/draft.md` whose latest version is a delete marker |
| `s3_encrypted_vault` | `acme-demo-vault`, versioning and MFA delete enabled: four objects pinning `aws:kms` and `AES256`, CRC32, SHA-1, SHA-256 and no checksum, across `STANDARD_IA`, `INTELLIGENT_TIERING`, `GLACIER` and `DEEP_ARCHIVE` |
| `s3_bucket_farm` | six buckets over five regions and every versioning state — unset, Enabled, Enabled with MFA delete, Suspended — each with its own tags, three carrying a `meta/region.txt` marker |
| `s3_fresh_bucket` | `acme-demo-scratch`, one tagged bucket with no keys and no versions |

To list the worlds: `GET /_rystic/scenarios` (Pro); `rystic docs s3-twin scenarios` prints the archive's `SCENARIOS.md` (the container image does not carry it); `--list-scenarios` on the installed binary works on every plan without a key:

```bash
~/.rystic/twins/s3-twin/0.0.1/s3-twin --list-scenarios
```

## Seeding state

Pro and Enterprise. `POST /_rystic/seed` takes a register map and lands it directly: `state_bucket` (keyed by `Name`), `state_object` (by `Bucket` and `Key`, the body as `body_b64`), `state_object_version`, `state_multipart_upload`, `state_upload_part` and `state_account`. `GET /_rystic/registers` lists every field. A file booted with `--seed <file>` or `RYSTIC_SEED=<file>` wraps the same map in `seed`, beside optional `clock`, `rng_seed` and `network`; a bare register map boots too.

```bash
curl -X POST localhost:8080/_rystic/seed -d '{"state_bucket":{"my-bucket":{"Name":"my-bucket","LocationConstraint":"","TagSet":[{"Key":"env","Value":"test"}]}}}'
curl -s -H "$AUTH" localhost:8080/
# -> <ListAllMyBucketsResult ...><Buckets><Bucket>...<Name>my-bucket</Name></Bucket></Buckets>...
```

Success returns 204. Only an unknown register or a record that is not a JSON object is refused (400, and an unknown register's error names every valid one). A record that, once merged with what the register already holds, still lacks a field the model marks required (a misspelled required field counts as missing) is accepted and logged as `WARNING — incomplete seed`, naming the register and the keys; the twin serves it as though it were whole. Boot the twin with `RYSTIC_SEED_GATE=reject` to have such a seed refused instead. Missing optional fields and fields the model does not declare are not checked.

## Determinism & reset

Execution is serialized. ETags are the MD5 of the body as on S3, upload ids and request ids are counters the reset rewinds, and the clock moves one millisecond per write, never by wall time: in a world it starts at the world's pinned instant (`2026-02-01T00:00:00Z`), on an empty Pro boot at the moment the twin started. Two identical runs from the same reset and pinned clock give byte-identical objects and listings, down to `ETag`, `Last-Modified`, `CreationDate` and `UploadId`.

1. **Free:** `POST /_rystic/reset` re-applies `s3_static_site`, clock and id sequence included.
2. **Pro, Enterprise:** reset wipes to empty and rewinds the sequence; `POST /_rystic/clock` pins the clock and `POST /_rystic/seed-rng` the RNG.

State is in memory: a restart boots the plan's world again — `s3_static_site` on Free; on Pro and Enterprise whatever `--scenario` or `--seed` names, or an empty account.

## Egress

None. The twin sends no event notifications and makes no outbound call; `GET /_rystic/egress` is always empty.

## Limitations

1. Objects are held to 64 MiB, where S3 takes 5 GiB per PUT and per part; a larger body answers `400 EntityTooLarge`.
2. A versioning-enabled bucket keeps no versions of what you write: a `PutObject` answers no `x-amz-version-id` and adds no row to `ListObjectVersions`, and after a `DeleteObject` a `GetObject` still answers 200 where S3 answers 404. The shipped versioned worlds list their seeded versions as stored (RYS-1845).
3. `ListObjectVersions` lists no `null` version for an object you write in an unversioned bucket, so a caller that empties a bucket through the version listing leaves your objects behind; use `ListObjectsV2` and `DeleteObjects` (RYS-1839).
4. `DeleteObjects` in quiet mode still lists every deleted key; read `Errors` (RYS-1836).
5. `GetObjectAttributes` answers `ETag` and `ObjectSize` only and quotes the ETag; read a checksum through `HeadObject` with `x-amz-checksum-mode: ENABLED` (RYS-1837).
6. A multipart upload completes with `ChecksumCRC64NVME` / `FULL_OBJECT` whatever algorithm it was created with, and the completed object reads back with no `x-amz-checksum-*` and no `x-amz-server-side-encryption` (RYS-1838).
7. No signature is verified, so the bad-credential path cannot be tested; presigned URLs and virtual-hosted addressing are not served, and `?versionId=` is ignored — the call acts on the current object.
8. One account (`123456789012`) with one synthetic owner; ACLs are read-only and grant it `FULL_CONTROL`.

Fidelity is scored differentially against S3 itself, per response field and header: the last full-suite round matched **117 of 117 probes on 2026-10-06**, with 0 inconclusive, on a 117-probe suite. Each release states its own number. Items 2–6 were found by driving the twin as boto3 and the AWS CLI would, outside what the probes select: read the score as the fidelity of what the suite exercises, not of every path a client can take.

The full guide ships in the release as `HOWTO.md`; `rystic docs s3-twin` prints it.

## Release notes

Newest first.

### v0.0.1 — 2026-10-07

1. **A local Amazon S3 for your storage code and the agents that drive it.** The first release of the S3 twin: 32 operations of S3's REST API, path-style on `/`, `/{Bucket}` and `/{Bucket}/{Key+}` — buckets, objects under any key, listings, multipart uploads, copies, tagging, batch deletes, conditional and Range requests and request checksums — served from a model validated against S3 itself. Point boto3 or the AWS CLI at it with an endpoint override, path-style addressing and any access key pair (`HOWTO.md` shows both), and your code runs against it unchanged; nothing it stores reaches AWS. (RYS-1548)

2. **Objects under any key, read back as S3 answers them.** PutObject, GetObject, HeadObject and DeleteObject on `/{Bucket}/{Key+}`, where a key such as `logs/2026/09/report.csv` is one object and `%2F` in a path is the `/` it stands for; ListObjectsV2, ListObjects and ListObjectVersions list by `prefix` and `delimiter`; GetObjectTagging, PutObjectTagging, DeleteObjectTagging, GetObjectAttributes and GetObjectAcl read and write an object's settings. Bodies run to 64 MiB; a larger one answers `400 EntityTooLarge`. Every routed answer is S3's own XML or headers and carries `x-amz-request-id` and `x-amz-id-2`, every error a routed S3 request gets is S3's `<Error>` document (only a path no route matches answers the twin's own JSON 404), and an operation on a modelled path that the twin does not model (ACL writes, legal hold, retention, restore, select, and every bucket setting but versioning and tags) answers `501 NotImplemented` and changes nothing. (RYS-1368, RYS-1369, RYS-1370, RYS-1373)

3. **Multipart uploads, copies and batch deletes.** CreateMultipartUpload, UploadPart, ListParts, CompleteMultipartUpload, AbortMultipartUpload and ListMultipartUploads let an SDK's upload manager — boto3's `upload_file`, `aws s3 cp` — start an upload, send its parts and complete it with the multipart ETag S3 gives it, up to twelve parts at S3's 5 MiB minimum; a part whose ETag matches none uploaded answers `InvalidPart`, an unknown upload id `NoSuchUpload`, and the completed object's `Location` leads back to the twin. CopyObject and UploadPartCopy copy within a bucket or from another by `x-amz-copy-source`, and DeleteObjects deletes every key its `<Delete>` document names. (RYS-1369, RYS-1370, RYS-1910)

4. **Request checksums, conditional writes and Range reads.** A PutObject or UploadPart carrying `x-amz-checksum-crc32`, `-crc32c`, `-crc64nvme`, `-sha1` or `-sha256`, as the AWS SDKs send by default, is checked against the body — a mismatch answers `400 BadDigest` and stores nothing — and the stored checksum is echoed under `x-amz-checksum-mode: ENABLED`, so SDK integrity validation passes. `If-None-Match: *` or a stale `If-Match` on PutObject answers `412 PreconditionFailed` and leaves the object alone; `If-None-Match` and `If-Modified-Since` on GetObject and HeadObject answer `304`; `If-Match` naming another ETag answers `412`; and a `Range` answers `206 Partial Content` with its `Content-Range`. (RYS-1368, RYS-1369, RYS-1835, RYS-1840)

5. **Buckets: create, list, read, configure and delete.** CreateBucket makes a bucket by the name you choose, ListBuckets lists the account's buckets with each one's `CreationDate` (and `BucketRegion` when asked), HeadBucket, GetBucketLocation, GetBucketAcl, GetBucketVersioning and GetBucketTagging read one, PutBucketVersioning turns versioning on or suspends it, PutBucketTagging and DeleteBucketTagging write its tags, and DeleteBucket removes an empty one — a bucket still holding objects answers `BucketNotEmpty`. `x-amz-expected-bucket-owner` passes for the owning account's 12-digit id (`123456789012` unless a seed says otherwise) and answers `403 AccessDenied` for any other. (RYS-1370, RYS-1856, RYS-1865)

6. **Five shipped worlds, and the one your test needs.** `--scenario s3_static_site` boots `acme-demo-site`, an unversioned website bucket with seven keys under real prefixes beside its access-log bucket, and is the world Free boots and `POST /_rystic/reset` returns to. `s3_versioned_archive` holds six version rows with a delete marker among them, `s3_encrypted_vault` four objects pinning different encryption, checksum and storage-class combinations, `s3_bucket_farm` six buckets over five regions and every versioning state, and `s3_fresh_bucket` one empty tagged bucket; `SCENARIOS.md` says what each is for. On Pro and Enterprise, `POST /_rystic/seed` lands buckets, objects, versions and multipart uploads directly, and `POST /_rystic/reset` rewinds the id sequence and the clock, so two identical runs give byte-identical objects and listings down to `ETag`, `Last-Modified` and `UploadId`. (RYS-1368, RYS-1548)

7. **Measured against S3 itself.** The 2026-10-06 full-suite round matched 117 of 117 probes against `s3.us-east-1.amazonaws.com`, with 0 inconclusive. `HOWTO.md` states what the suite leaves out and the differences this release ships with, each with its ticket: a versioning-enabled bucket keeps no versions of what you write (RYS-1845), ListObjectVersions lists no `null` version for an object you write (RYS-1839), DeleteObjects in quiet mode lists every deleted key (RYS-1836), GetObjectAttributes answers `ETag` and `ObjectSize` only (RYS-1837), and a multipart upload completes with CRC64NVME whatever algorithm it was created with and reads back without its checksum or encryption headers (RYS-1838). (RYS-1548)

---

Source: https://www.rystic.ai/docs/simulators/s3 · Markdown: https://www.rystic.ai/docs/simulators/s3.md