Drift baseline in CI

The idea

Commit an architecture baseline next to your code, and get a markdown diff on every pull request when the recovered architecture changes shape — new cross-component edges, moved entities, merged or split components. The baseline is a plain JSON file versioned by git like anything else in your repo; there’s no server, no account, nothing to sign up for.

Two commands, both the arcade-arch-diff console script:

• arcade-arch-diff --source . — parse the current code, recover its architecture, diff it against the committed baseline, print a markdown report. Read-only.
• arcade-arch-diff --source . --update-baseline — same recovery, but overwrite .arcade/baseline.json with the current architecture instead of diffing it.

A CI job runs the first on every pull request and posts the report as a comment; a second job runs the second on every push to your default branch, so the baseline always reflects what actually merged. See Exit codes below for what “diff” means here — this is a report, not a merge gate, by default.

CLI flags

Flag	Default	Meaning
--source	.	Path to the repository to parse.
--language	auto-detect	Force a language instead of auto-detecting (java, python, typescript, c, go, kotlin).
--baseline	.arcade/baseline.json	Path to the baseline file to read/write.
--update-baseline	off (flag)	Overwrite the baseline with the current architecture instead of comparing against it.

Setup

name: Architecture Drift

on:
  pull_request:
  push:
    branches: [main]

jobs:
  # Every PR: parse the repo, diff against the committed baseline, post a comment.
  drift-report:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - run: pip install "arcade-agent[languages]"

      - name: Run arcade-arch-diff
        run: arcade-arch-diff --source . > drift-report.md 2>&1 || true
        # arcade-arch-diff never exits non-zero (see "Exit codes" below) — the
        # `|| true` guards against that changing later, it isn't load-bearing today.

      - name: Post report as a PR comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const marker = '<!-- arcade-agent-drift-report -->';
            const report = fs.readFileSync('drift-report.md', 'utf8');
            const body = report.includes(marker) ? report : `${report}\n${marker}`;
            const comments = await github.paginate(github.rest.issues.listComments, {
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
            });
            const existing = comments.find(c => c.body && c.body.includes(marker));
            if (existing) {
              await github.rest.issues.updateComment({
                owner: context.repo.owner, repo: context.repo.repo,
                comment_id: existing.id, body,
              });
            } else {
              await github.rest.issues.createComment({
                owner: context.repo.owner, repo: context.repo.repo,
                issue_number: context.issue.number, body,
              });
            }

  # Every push to main: refresh the baseline so the next PR diffs against
  # what actually merged.
  update-baseline:
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - run: pip install "arcade-agent[languages]"

      - run: arcade-arch-diff --source . --update-baseline > /dev/null

      - name: Commit baseline if it changed
        run: |
          if git diff --quiet -- .arcade/baseline.json; then
            echo "No architecture baseline changes to commit."
            exit 0
          fi
          git config user.name "github-actions[bot]"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add .arcade/baseline.json
          git commit -m "chore: update architecture baseline"
          git push

Adapted from the tool repo’s own .github/workflows/arch-drift.yml — same two-job split (PR comment / baseline commit) and the same comment marker. The notable differences — both intentional simplifications — are: branches: [main] assumes main as the default branch (swap it for yours), and the install step is a plain pip install "arcade-agent[languages]" instead of the tool repo’s own editable pip install -e ".[languages]", since a consumer repo installs the published package rather than checking out the tool’s own source. The real workflow also carries repo-specific plumbing a consumer copy doesn’t need — a fork-safety guard on the comment step, duplicate-comment cleanup, and a manual-dispatch language input.

The first run after adding this workflow won’t have a baseline yet — .arcade/baseline.json doesn’t exist, so that first PR’s report has no “Drift from Baseline” section (there’s nothing to diff against), and the push-to-main job creates the baseline file for the first time. Every report from the next PR onward compares against what’s actually committed.

Live example

Real output from the tool repo’s own CI, dogfooding this drift workflow on itself:

• PR #22, drift comment — a real report the drift-report job posted, showing an actual structural change (“1 entity movement(s) between components”) alongside the full metrics table.
• 922ccbc, “chore: update architecture baseline” — a real commit the update-baseline job made after a merge to main; this is exactly what lands in your own repo’s history once the workflow is running.

Exit codes

arcade-arch-diff never exits non-zero. Its main() has no sys.exit() call anywhere — a run that finds a fresh drift, new smells, or a dropped component still falls through to an implicit exit 0. That’s deliberate, not an oversight: the comparison is informational, the same posture its sibling arcade-compare-baseline script documents explicitly (“exits with code 0 always — comparison is informational, not a pass/fail gate”). The workflow above enforces this by piping the command’s output through || true and posting whatever comes out as a PR comment — never a failing check.

If you want a hard gate, you have to build it yourself by grepping the report. The “Changes” section of the markdown output prints the literal line No structural changes detected only when nothing changed (no components added or removed, no entities moved, no merges or splits) — every other case replaces that line with the actual list of changes. So:

arcade-arch-diff --source . > drift-report.md
if [ -f .arcade/baseline.json ] && ! grep -q "No structural changes detected" drift-report.md; then
  echo "::error::Architecture drift detected — see drift-report.md"
  exit 1
fi

The -f .arcade/baseline.json guard matters: on the very first run there’s no baseline yet, so there’s no “Drift from Baseline” section at all and the string is absent regardless of whether anything meaningfully “changed” — without that guard, every repo’s very first PR would fail.