Slack Harbor reference run

This recipe runs Rystic’s reference harness on the same Slack task and image contract as Slack MCP in Harbor. To connect your existing runner, follow that guide instead.

Prerequisites

Complete the image access and pull steps. You need Docker on Linux amd64, uv, Python 3.12 or newer, a license covering Slack and Stripe, and GitHub read access to Rystic-Inc/twin-envs. Ask for that repository access with your Harbor onboarding; a 404 while signed in means the account has not been granted access.

The lockfile pins the Rystic verifiers fork that honors distinct agent and verifier users. Keep the lockfile and use --frozen. The image supplies the simulators and MCP package; no rystic-sim checkout or Go compiler is required.

Stage the task

Run these commands on the trusted host. Keep MCP_IMAGE set to the image supplied for your license.

git clone https://github.com/Rystic-Inc/twin-envs.git
cd twin-envs
git checkout --detach 2d6d6712bd0d314d00aa7bc82a1e0a21e3a942d6
uv sync --frozen
uv run --frozen python scripts/harbor_stage.py slack-heldout --mcp --image "$MCP_IMAGE"

Expect the dataset name rystic/slack-heldout-mcp and its local Harbor cache path. The generated files are also under harbor/tasks/slack-heldout-mcp/slack_ops-001. The checkout pins the task, grader, adapter and dependency lock together.

Run a policy

Load RYSTIC_LICENSE_B64 and ANTHROPIC_API_KEY into the host environment from your secret manager. RYSTIC_LICENSE_B64 is the base64-encoded license JSON. The provider key stays with host-side inference; never put it in task.toml, the agent prompt or the container environment.

The supplied configuration selects claude-opus-5. A rollout makes billable calls to that provider. The dry run checks configuration without running the policy:

: "${RYSTIC_LICENSE_B64:?Load your base64-encoded license}"
: "${ANTHROPIC_API_KEY:?Load your host-side inference key}"
uv run --frozen eval @ configs/slack-heldout-harbor-mcp-hosted.toml --dry-run --no-rich
uv run --frozen eval @ configs/slack-heldout-harbor-mcp-hosted.toml \
  -o outputs/mcp-qualification --run.dir agent --no-rich

The configuration selects the Harbor Docker runtime, one rollout, the original slack_ops-001 task and the twinenv_mcp_harness adapter. That adapter validates the protected manifest and adds the container-local MCP connection to the existing policy loop. Stripe keeps its HTTP interface.

Read the result

Inspect the reward and trace under outputs/mcp-qualification/agent. A successful process exit alone does not establish a successful task. Check the exact staged task name, completed rollout, numeric reward and acknowledged tool calls.

The measured reference rollout earns 1.0 with 17 MCP replies, zero extra mutations and zero forbidden calls. Its Slack work uses the four selected MCP tools; Stripe calls use HTTP. This is a reference outcome, not a guarantee for every policy or sampling run.

For the full provider-free lifecycle and failure checks, run the existing checker against your image before a large batch:

uv run --frozen python scripts/check_slack_mcp_episodes.py \
  --image "$MCP_IMAGE" \
  --task-dir harbor/tasks/slack-heldout-mcp/slack_ops-001 \
  --release-manifest configs/slack-mcp-release-linux-amd64.json \
  --repeat 3 --out outputs/mcp-qualification/episodes

It must exit zero and produce outputs/mcp-qualification/episodes/summary.json. It exercises two episodes, deterministic reset and failure recovery without model inference. Keep receipts private until they have been checked for credentials and customer information.

Use full episode reset and cleanup when integrating the lifecycle into your own runner. Keep the measured support scope with any results you share.