# `shoalcheck gate` on CircleCI -- the job that reads the cluster and fails on what changed. Save as # .circleci/config.yml (or merge the job and the two workflows into yours). # # THE CONTRACT. Exit 0 the policy is satisfied, 2 it fired, 1 the run or the comparison could not be made, 4 the # licence does not cover the gate -- and a comparison the gates refuse is never a pass. The findings land in the # job's Tests tab (JUnit) and its artifacts (the report, the reading, a Markdown summary). # # WHAT IT NEEDS. A context named `shoalcheck` (Organization settings > Contexts) holding KUBECONFIG_B64, a kubeconfig # with the read-only access `shoalcheck audit --show-permissions` prints, base64-encoded on one line # (`base64 < kubeconfig | tr -d '\n'`), whose context is the one named below, and SHOALCHECK_LICENCE, a Team licence # key (the gate is a Team command). SHOALCHECK_VERSION, the release to pin without the leading v (e.g. 0.1.0-rc.12), # as a project environment variable -- a pipeline should not change under you, so this never asks for "latest". And # SHOALCHECK_BASE_BRANCH if pull requests merge into something other than main: CircleCI does not tell a job which # branch a pull request targets. # # TWO RUNS FROM ONE FILE. A scheduled pipeline (Project settings > Triggers, weekday mornings, on main) reads the # cluster against the committed baseline: drift, with nobody having opened anything. A pull request reads the cluster # as it is, against the baseline on the branch it merges into, and judges the request's rendered manifests laid over # that reading in memory (--with) -- before anything is applied. Nothing is applied by the gate. # # A RUN WITHOUT THE SECRETS DOES NOT GATE, AND SAYS SO. CircleCI gives a pull request from a fork no context and no # project secrets unless you allow it. Such a run reads nothing, compares nothing, prints "shoalcheck did not gate # this run" and ends without failing -- not exit 4, and not a pass. # # THE BASELINE IS A READING OF THIS SAME CLUSTER. shoalcheck refuses to compare readings of two different clusters -- # it knows a cluster by its kube-system namespace, not by a context name a rebuilt cluster can reuse -- and a refused # comparison fails the job with exit 1. Record the baseline once from the cluster this job reads, with # --update-baseline, and commit it. A cluster made new for every run records its baseline in the same run: apply the # manifests the change starts from, run the gate with --update-baseline, apply the change, run the gate again. version: 2.1 jobs: shoalcheck-gate: docker: # Pinned by digest: the tag is for the reader, the digest is what is pulled. - image: cimg/base:2026.09-24.04@sha256:668174490adc470cd34eb3aa178939d65f817bc13ef657a94ff0278338f7d8da environment: # The store lives here so the run leaves nothing in the image's home. SHOALCHECK_HOME: /tmp/shoalcheck steps: # A RUN WITHOUT THE SECRETS DOES NOT GATE: it says so first and halts the job, before installing anything. - run: name: Secrets present command: | if [ -z "${SHOALCHECK_LICENCE:-}" ] || [ -z "${KUBECONFIG_B64:-}" ]; then echo "shoalcheck did not gate this run: it has no licence or no cluster credentials (a pull request from a fork is given neither). Nothing was read and nothing was compared, so this is not a pass." circleci-agent step halt fi - checkout # CHECKED THE WAY IT IS SIGNED. cosign itself is pinned and checked against the sha256 its release publishes # (cosign_checksums.txt, v2.5.2). Then the signature over checksums.txt with the release key, then the archive # against its checksum: a binary that is not the one that was signed has no business reading a production # cluster. The key is written here, not downloaded: a key fetched from the release it vouches for proves # nothing. It is the cosign.pub the releases repository publishes. - run: name: Install shoalcheck command: | set -euo pipefail : "${SHOALCHECK_VERSION:?set the project variable SHOALCHECK_VERSION to the release to pin, without the leading v}" bin="$HOME/bin" mkdir -p "$bin" curl -fsSL -o "$bin/cosign" https://github.com/sigstore/cosign/releases/download/v2.5.2/cosign-linux-amd64 echo "bcfeae05557a9f313ee4392d2f335d0ff69ebbfd232019e3736fb04999fe1734 $bin/cosign" | sha256sum -c - chmod +x "$bin/cosign" export PATH="$bin:$PATH" echo "export PATH=\"$bin:\$PATH\"" >> "$BASH_ENV" base="https://github.com/Shoalcheck/shoalcheck/releases/download/v${SHOALCHECK_VERSION}" archive="shoalcheck_${SHOALCHECK_VERSION}_linux_amd64.tar.gz" cd /tmp for f in "$archive" checksums.txt checksums.txt.sigstore.json; do curl -fsSLO "$base/$f" done cat > cosign.pub \<<'KEY' -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE69u2ja9r2k/Vy8lSibowBdCnCD3P OVzmcbjHVIvMgoOfkEzEwOY/b8kyjTVJcwrJ89+qC5bjW4XzLqz1S/AryQ== -----END PUBLIC KEY----- KEY cosign verify-blob --key cosign.pub --bundle checksums.txt.sigstore.json checksums.txt grep " $archive\$" checksums.txt | sha256sum -c - tar xz -C "$bin" -f "$archive" shoalcheck shoalcheck version # THE BASELINE COMES FROM THE BRANCH BEING MERGED INTO, never from the pull request: a pull request that edits # .shoalcheck/baseline.json could otherwise record its own finding as already known and pass its own gate. - run: name: Baseline from the target branch command: | if [ -n "${CIRCLE_PULL_REQUEST:-}" ]; then set -euo pipefail git fetch --no-tags --depth=1 origin "${SHOALCHECK_BASE_BRANCH:-main}" mkdir -p .shoalcheck if git show "FETCH_HEAD:.shoalcheck/baseline.json" > .shoalcheck/baseline.target 2>/dev/null; then mv .shoalcheck/baseline.target .shoalcheck/baseline.json else rm -f .shoalcheck/baseline.target .shoalcheck/baseline.json echo "the target branch has no .shoalcheck/baseline.json yet, so there is nothing to compare with" fi fi # The verdict in the step log, and the exit code the job ends on. --json keeps the reading the gate judged; # every output after it is rendered from that file, so the test results, the summary and the red build cannot # disagree. - run: name: Gate command: | set -uo pipefail export KUBECONFIG=/tmp/kubeconfig printf '%s' "$KUBECONFIG_B64" | base64 -d > "$KUBECONFIG" mkdir -p .shoalcheck /tmp/shoalcheck-out /tmp/test-results/shoalcheck # THE PULL REQUEST, BEFORE IT LANDS: its rendered manifests go to the gate with --with, which lays them # over the reading in memory and judges them -- nothing is applied. not-evaluated is in its policy, so an # object the gate could not judge (a kind it does not read, a Secret, a ConfigMap) fails the run rather # than passing unexamined; drop it if your changes carry ConfigMaps you accept as unjudged. A binding at # an API version the gate does not judge is refused (exit 1). code=0 if [ -n "${CIRCLE_PULL_REQUEST:-}" ]; then # THE CHANGE, RENDERED: --with judges what would be applied, so it takes the rendered manifests, never the # sources. Swap in your renderer -- for a chart: helm template charts/ -f values.yaml. # The image's kubectl when it has one; otherwise a pinned release, checked against its published sha256. change="$(mktemp -d)" k="$(command -v kubectl || true)" if [ -z "$k" ]; then k="$(mktemp -d)/kubectl" curl -fsSLo "$k" https://dl.k8s.io/release/v1.31.4/bin/linux/amd64/kubectl echo "$(curl -fsSL https://dl.k8s.io/release/v1.31.4/bin/linux/amd64/kubectl.sha256) $k" | sha256sum -c - chmod +x "$k" fi "$k" kustomize deploy/ > "$change/rendered.yaml" shoalcheck gate \ --context staging \ --baseline .shoalcheck/baseline.json \ --with "$change" \ --fail-on new:high,regressed,unreadable,not-evaluated \ --output /tmp/shoalcheck-out/report.html \ --json /tmp/shoalcheck-out/reading.json || code=$? else shoalcheck gate \ --context staging \ --baseline .shoalcheck/baseline.json \ --fail-on new:high,regressed,unreadable \ --output /tmp/shoalcheck-out/report.html \ --json /tmp/shoalcheck-out/reading.json || code=$? fi if [ -s /tmp/shoalcheck-out/reading.json ]; then shoalcheck findings --input /tmp/shoalcheck-out/reading.json -o junit > /tmp/test-results/shoalcheck/junit.xml shoalcheck findings --input /tmp/shoalcheck-out/reading.json -o md-summary --baseline .shoalcheck/baseline.json > /tmp/shoalcheck-out/summary.md cat /tmp/shoalcheck-out/summary.md fi rm -f "$KUBECONFIG" exit $code # Every finding as a test: a failure at high or above, a skipped test below it (never a pass), and a skipped # test per kind the run could not read. The gate's exit code is the verdict; this is what the team reads beside # it. The report is the artifact a human opens when the gate fails: a single file with no external requests. - store_test_results: path: /tmp/test-results - store_artifacts: path: /tmp/shoalcheck-out destination: shoalcheck workflows: # The schedule lives in Project settings > Triggers; this workflow is what it runs. shoalcheck-drift: when: equal: [scheduled_pipeline, << pipeline.trigger_source >>] jobs: - shoalcheck-gate: context: shoalcheck shoalcheck-pull-request: when: not: equal: [scheduled_pipeline, << pipeline.trigger_source >>] jobs: - shoalcheck-gate: context: shoalcheck # Recording a baseline the first time (or after a deliberate change): # shoalcheck gate --context staging --baseline .shoalcheck/baseline.json --update-baseline # It compares first and writes afterwards, so the run that records still reports what moved.