Workflows and CI runs

A push, or a pull request that opens, moves or reopens, starts the repository’s own workflows, and each job becomes a check run that the simulator completes without running it. Create a workflow on a branch through the contents API and read what it starts:

H='Authorization: Bearer ghp_anything'
R="$RYSTIC_URL/repos/tb-owner/tb-repo"
sha=$(curl -s "$R/branches/tb-base-branch" -H "$H" | jq -r .commit.sha)
curl -s -X POST "$R/git/refs" -H "$H" -d "{\"ref\":\"refs/heads/ci-demo\",\"sha\":\"$sha\"}" | jq -c '{ref}'
# -> {"ref":"refs/heads/ci-demo"}
wf=$(printf 'name: ci\non: [push]\njobs:\n  test:\n    runs-on: ubuntu-latest\n    steps:\n      - run: go test ./...\n' | base64 | tr -d '\n')
curl -s -X PUT "$R/contents/.github/workflows/ci.yml" -H "$H" -d "{\"message\":\"add ci\",\"content\":\"$wf\",\"branch\":\"ci-demo\"}" | jq -c '{commit: .commit.sha[0:8]}'
# -> {"commit":"e24228f3"}
curl -s "$R/commits/ci-demo/check-runs" -H "$H" | jq -c '{total_count, runs: [.check_runs[] | {name, status, conclusion, app: .app.slug}]}'
# -> {"total_count":1,"runs":[{"name":"test","status":"completed","conclusion":"neutral","app":"github-actions"}]}
curl -s "$R/actions/runs" -H "$H" | jq -c '{total_count, runs: [.workflow_runs[] | {id, name, event, status, conclusion, run_attempt}]}'
# -> {"total_count":1,"runs":[{"id":6600001,"name":"ci","event":"push","status":"completed","conclusion":"neutral","run_attempt":1}]}
curl -s "$R/actions/runs/6600001/jobs" -H "$H" | jq -c '[.jobs[] | {id, name, status, conclusion}]'
# -> [{"id":6400001,"name":"test","status":"completed","conclusion":"neutral"}]
curl -s -X POST "$R/actions/runs/6600001/rerun" -H "$H" -o /dev/null -w '%{http_code}\n'
# -> 201
curl -s "$R/actions/runs/6600001" -H "$H" | jq -c '{status, conclusion, run_attempt}'
# -> {"status":"completed","conclusion":"neutral","run_attempt":2}
curl -s -X POST "$RYSTIC_URL/_rystic/seed" -d "{\"state_workflow_run\": {\"9001\": {\"id\": 9001, \"repository\": {\"full_name\": \"tb-owner/tb-repo\"}, \"path\": \".github/workflows/pipeline.yml\", \"name\": \"pipeline\", \"event\": \"workflow_dispatch\", \"head_branch\": \"tb-base-branch\", \"head_sha\": \"$sha\", \"status\": \"completed\", \"conclusion\": \"failure\"}}, \"state_workflow_job\": {\"9101\": {\"id\": 9101, \"run_id\": 9001, \"name\": \"test\", \"status\": \"completed\", \"conclusion\": \"failure\", \"log\": \"##[error]3 tests failed\"}}}" -o /dev/null -w '%{http_code}\n'
# -> 204
curl -s -L "$R/actions/jobs/9101/logs" -H "$H"
# -> ##[error]3 tests failed
curl -s -X POST "$R/actions/runs/9001/rerun" -H "$H" | jq -c '{message}'
# -> {"message":"This workflow run cannot be re-run"}

The first run reads neutral because no evaluator is registered. CI results and the evaluation engine says how a result is decided and how to register an evaluator.

Which workflows run

A push starts the repository’s own workflows, whether it comes from git or through the contents and git data APIs. So does a pull request that opens, moves or reopens. For each, the simulator:

• reads .github/workflows/*.yml at the pushed commit;
• applies each workflow’s push or pull_request triggers, with their branch, tag, path and activity-type filters;
• expands needs and strategy.matrix, with include and exclude;
• creates one check run per job as the github-actions app.

A check run is named the way GitHub names the job, as in test (1.24, ubuntu-latest). A job skipped before it expands keeps its raw name. gh pr checks and GET .../commits/{ref}/check-runs list them.

A pull request’s workflows evaluate its test merge: the head merged into the base branch’s current tip, which refs/pull/{n}/merge serves and merge_commit_sha names. github.sha is that merge commit, and the check runs report on the head. A change that passes alone but breaks once merged with a moved base therefore fails. github.event carries number and pull_request, the pull request as its webhook sends it. A job’s if: and its steps read its head and base from there. A pull request whose head conflicts with its base starts no workflow.

Job ids and matrices

Each workflow an event starts is one Actions workflow run, in a check suite of its own. Each check run is one of the run’s jobs, and the job’s id is the check run’s id. The evaluation engine names a job by its workflow path and job id instead: see Job ids.

A matrix built from a dependency’s output, such as ${{ fromJSON(needs.plan.outputs.legs) }}, waits to expand. It expands once its own dependencies have finished, and so has every other dependency of the jobs that need it. It then becomes one job per combination, named as GitHub names it. Until then, those combinations and the jobs that need them are not listed.

A job that needs such a matrix job reads the combinations’ result together. Two such matrices that one job needs expand one after the other. A matrix job whose condition is false is one skipped job under its own name. Re-running a job that such a matrix needs keeps what the matrix first expanded to, or its skip.

Differs from GitHub: GitHub expands each such matrix as soon as its own dependencies finish, and evaluates it again on a re-run.

Run status

A run is queued until its first job completes, then in_progress until its last does. It then reads completed, with the conclusion of the first row whose condition holds:

Re-runs, cancels and webhooks

A re-run is the run’s next attempt. Each kind also re-runs every job that needs a job it re-runs:

• POST .../runs/{id}/rerun re-runs every job.
• .../rerun-failed-jobs (gh run rerun --failed) re-runs the failed, canceled and timed-out jobs.
• .../jobs/{id}/rerun re-runs one job.

A job not re-run carries into the new attempt with its result, and the re-run jobs that need it read that result and its outputs. GET .../runs/{id}/attempts/{n} (with /jobs and /logs) serves an earlier attempt, and .../jobs?filter=all lists every attempt’s jobs.

A run still running refuses a re-run with 403. POST .../runs/{id}/cancel ends every unfinished job cancelled, and a completed run returns 409. POST /check-runs/{id}/rerequest on a job’s check run re-runs that job. On any other check run the simulator created, it evaluates the check again as a new run, keeping its original prerequisites and outputs.

Each step fires these webhooks to the hooks subscribed to them:

• workflow_run: requested, in_progress, completed
• workflow_job: queued, in_progress, completed
• check_run: created and completed, for each job’s check run
• check_suite: completed

gh and dispatch

These gh commands work against the simulator:

• gh workflow run and gh workflow list
• gh run list and gh run cancel
• gh run view, with --log, --log-failed, --job and --attempt
• gh run rerun, with --failed and --job

A workflow is a .github/workflows file on the default branch, or one a stored run names. A dispatch needs a ref the repository holds and a workflow_dispatch trigger in the file at that ref. It runs that workflow at the ref’s tip, with the inputs it was sent as inputs and github.event.inputs.

A merge queue requesting a group’s checks starts the workflows with a merge_group trigger on the group’s head. There github.event_name is merge_group, and github.ref is the gh-readonly-queue/<base>/pr-<number>-<sha> branch. No push workflow runs on that branch.

A job’s log is its evaluator’s output. A run’s or a job’s logs return 302 to an expiring download: a zip with one <n>_<job name>.txt per job, or the job’s text.

Seeding runs

A harness can also set runs, outcomes and logs through /_rystic/seed. state_workflow_run takes GitHub’s run fields plus repository.full_name. state_workflow_job takes a job’s fields, its run_id and its log text in log. A partial record merges. A seeded run shows what it was given and cannot be re-run.

What a job is not

A job is one evaluation, not its steps.

Differs from GitHub: a job lists no steps and its log is one text, so gh run view --log shows UNKNOWN STEP and a run lists no artifacts.

Differs from GitHub: reusable workflows, and a matrix or condition whose expression falls outside the simulator’s expression subset, are not expanded and show neutral.