Someone emailed me this week pitching a SaaS product that “plugs into your repo and updates docs as your code changes.” Here’s the thing: we already built this — over the past four months, across three source repos and one docs site. It’s been running in production since February. This post is everything we built, open and copy-pasteable.
The problem nobody talks about honestly
Documentation doesn’t go stale all at once. It goes stale one merged PR at a time. A new env var gets added. A config key gets renamed. A default changes. Nobody opens a ticket for “update the docs for the thing I just shipped.” The sprint is done. The PR is merged. The docs lie quietly.
The standard advice is “make docs part of the PR.” That works until it doesn’t — until the team is moving fast, until the PR is “just a small fix,” until the reviewer doesn’t check. The advice is right but the enforcement mechanism is vibes.
What we wanted: docs that update themselves. Not on a schedule, not from a separate team — on every merge, triggered by the exact commit that caused the drift, reading the actual changed files at that exact SHA.
What we built
AutoMem has three source repos:
- automem — the core Python memory service
- mcp-automem — the TypeScript MCP client and CLI
- automem-graph-viewer — the React/Vite 3D graph frontend
And one docs site: automem-website (Astro Starlight, deployed at automem.ai).
The pipeline:
code PR merges to main in any source repo
↓
docs-dispatch.yml: git diff --name-only, match against file-doc-map.json
↓
if matched: repository_dispatch → automem-website with full context
↓
docs-update.yml: check out source repo at exact SHA, run Claude Code Action
↓
Claude reads changed files + affected doc pages, opens a PR
↓
docs-auto-merge.yml: scan PR body for open questions
↓
clean PRs merge themselves; ambiguous ones wait for you
There’s also a second pipeline for benchmark data — benchmarks-update.yml — that does the same cross-repo dispatch but runs npm run sync-benchmarks instead of Claude. No AI needed when the update is deterministic.
The timeline
Four months of iteration, not a weekend project:
- Feb 21 —
feat: automated docs maintenance pipeline (#15)— initial pipeline concept landed - Mar 10 —
docs: full audit of all 56 doc pages against source code (#19)— first full Claude-powered audit of every doc page - Mar 25–26 — a week of CI fixes: OIDC auth, short SHA resolution, allowed tools, max-turns tuning
- Apr 23 — dispatch moved into the release-please workflow for tighter coupling to releases
- May 1 — auto-merge added; clean PRs now merge themselves
- May 8–13 — merge queue alignment, draft-PR normalization, hardening
- Jun 12 — latest hardening pass on the auto-merge guard logic
Step 1: The sender (docs-dispatch.yml)
This lives in each source repo — identical copy in automem, mcp-automem, and automem-graph-viewer. Because REPO_KEY="${GITHUB_REPOSITORY#*/}" auto-derives the map key from whichever repo is running it, it truly is a single copy-paste with zero customization needed between repos:
name: Docs Dispatch
on:
push:
branches: [main]
workflow_dispatch:
inputs:
base_sha:
description: "Base SHA or ref for the docs diff. Defaults to head^."
required: false
head_sha:
description: "Head SHA or ref to document. Defaults to main."
required: false
reason:
description: "Why docs should be audited."
required: false
permissions:
contents: read
jobs:
check-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Get changed files
id: changed
env:
EVENT_NAME: ${{ github.event_name }}
INPUT_BASE_REF: ${{ inputs.base_sha }}
INPUT_HEAD_REF: ${{ inputs.head_sha }}
INPUT_REASON: ${{ inputs.reason }}
PUSH_BASE_REF: ${{ github.event.before }}
PUSH_HEAD_REF: ${{ github.event.after }}
run: |
ZERO_SHA="0000000000000000000000000000000000000000"
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
HEAD_REF="${INPUT_HEAD_REF:-main}"
BASE_REF="$INPUT_BASE_REF"
REASON="${INPUT_REASON:-Manual docs dispatch for ${HEAD_REF}}"
else
HEAD_REF="$PUSH_HEAD_REF"
BASE_REF="$PUSH_BASE_REF"
REASON="Push to main"
fi
HEAD_SHA=$(git rev-parse "$HEAD_REF")
if [ -z "$BASE_REF" ]; then
if git rev-parse --verify "${HEAD_REF}^" >/dev/null 2>&1; then
BASE_REF="${HEAD_REF}^"
else
BASE_REF=$(git rev-list --max-parents=0 --max-count=1 "$HEAD_SHA")
fi
fi
if [ "$BASE_REF" = "$ZERO_SHA" ]; then
BASE_SHA=$(git rev-list --max-parents=0 --max-count=1 "$HEAD_SHA")
else
BASE_SHA=$(git rev-parse "$BASE_REF")
fi
COMMIT_URL="https://github.com/${GITHUB_REPOSITORY}/commit/${HEAD_SHA}"
COMPARE_URL="https://github.com/${GITHUB_REPOSITORY}/compare/${BASE_SHA}...${HEAD_SHA}"
FILES=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA" \
| jq -R -s -c 'split("\n") | map(select(. != ""))')
{
echo "base_sha=$BASE_SHA"
echo "head_sha=$HEAD_SHA"
echo "files=$FILES"
echo "commit_url=$COMMIT_URL"
echo "compare_url=$COMPARE_URL"
echo "reason<<EOF"
echo "$REASON"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Check file-doc mapping
id: check
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN }}
CHANGED_FILES: ${{ steps.changed.outputs.files }}
run: |
if ! MAP=$(curl -fsSL --connect-timeout 10 --max-time 30 --retry 2 \
-H "Authorization: Bearer $GH_TOKEN" \
-H "Accept: application/vnd.github.raw" \
"https://api.github.com/repos/verygoodplugins/automem-website/contents/scripts/file-doc-map.json"); then
echo "::warning::Failed to fetch file-doc-map.json, dispatching anyway"
echo "affected=[]" >> "$GITHUB_OUTPUT"
echo "mapping_status=unknown" >> "$GITHUB_OUTPUT"
exit 0
fi
REPO_KEY="${GITHUB_REPOSITORY#*/}"
AFFECTED=$(echo "$MAP" | jq -r \
--arg repo "$REPO_KEY" --argjson changed "$CHANGED_FILES" '
def matches_pattern($file; $pattern):
if ($pattern | endswith("/**")) then
($file | startswith($pattern[0:-2]))
else
$file == $pattern
end;
.[$repo] // {} | to_entries | map(
select(.key as $pattern |
$changed | any(. as $file | matches_pattern($file; $pattern)))
) | map(.value) | flatten | unique | .[]
')
if [ -z "$AFFECTED" ]; then
echo "affected=[]" >> "$GITHUB_OUTPUT"
echo "mapping_status=none" >> "$GITHUB_OUTPUT"
else
AFFECTED_JSON=$(echo "$AFFECTED" | jq -R -s -c 'split("\n") | map(select(. != ""))')
echo "affected=$AFFECTED_JSON" >> "$GITHUB_OUTPUT"
echo "mapping_status=matched" >> "$GITHUB_OUTPUT"
fi
- name: Dispatch to automem-website
if: steps.check.outputs.mapping_status != 'none'
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN }}
run: |
jq -n \
--arg event_type "docs-update" \
--arg source_repo "${{ github.repository }}" \
--arg source_sha "${{ steps.changed.outputs.head_sha }}" \
--arg base_sha "${{ steps.changed.outputs.base_sha }}" \
--argjson changed_files "${{ steps.changed.outputs.files }}" \
--argjson affected_docs "${{ steps.check.outputs.affected }}" \
--arg mapping_status "${{ steps.check.outputs.mapping_status }}" \
--arg reason "${{ steps.changed.outputs.reason }}" \
--arg commit_url "${{ steps.changed.outputs.commit_url }}" \
--arg compare_url "${{ steps.changed.outputs.compare_url }}" \
'{event_type: $event_type, client_payload: {
source_repo: $source_repo, source_sha: $source_sha,
base_sha: $base_sha, changed_files: $changed_files,
affected_docs: $affected_docs, mapping_status: $mapping_status,
reason: $reason, commit_url: $commit_url, compare_url: $compare_url
}}' \
| gh api repos/verygoodplugins/automem-website/dispatches \
--method POST --input -Three things worth calling out:
- No GitHub API for the diff. The repo is checked out with
fetch-depth: 0sogit diff --name-onlyruns locally — no extra API calls, no rate limits. mapping_statushas three states.noneskips the dispatch entirely.matchedis the happy path.unknownfires anyway — if the map fetch fails, you’d rather have Claude look at a PR that didn’t need updating than silently miss one that did.- The map lives in the docs repo. One PR to add a new mapping, not three PRs across three source repos.
Step 2: The map (file-doc-map.json)
The brain of the whole system. Centralized in automem-website/scripts/file-doc-map.json. Here’s a representative slice of the 80+ entries:
{
"_meta": {
"description": "Maps source file patterns to doc page slugs.",
"last_updated": "2026-06-12"
},
"automem": {
"automem/config.py": ["reference/configuration", "getting-started/environment-variables"],
"automem/runtime_environment.py": ["reference/configuration"],
"automem/api/memory.py": ["reference/api/memory-operations"],
"automem/api/recall.py": ["reference/api/recall-operations"],
"automem/embedding/**": ["architecture/embeddings", "architecture/background-processing"],
"automem/search/**": ["core-concepts/hybrid-search", "reference/api/recall-operations"],
".env.example": ["getting-started/environment-variables", "reference/configuration"],
"README.md": ["overview", "getting-started/introduction"]
},
"mcp-automem": {
"src/env.ts": ["getting-started/environment-variables", "mcp/configuration"],
"src/tools/**": ["mcp/tools", "reference/api/memory-operations"],
"src/cli/**": ["mcp/cli"],
"README.md": ["mcp/overview", "getting-started/introduction"]
},
"automem_graph_viewer": {
"src/components/**": ["graph-viewer/overview", "graph-viewer/customization"],
"src/App.tsx": ["graph-viewer/overview"],
"README.md": ["graph-viewer/overview"]
}
}Two pattern types: exact path ("automem/config.py") and subtree glob ("automem/embedding/**"). One file change can map to multiple doc pages. The jq matches_pattern function in the dispatch handles both — [0:-2] strips the trailing /** to get the directory prefix for startswith.
Step 3: The receiver (docs-update.yml)
name: Auto-Update Docs
on:
repository_dispatch:
types: [docs-update]
workflow_dispatch:
inputs:
source_repo:
description: "Source repo"
required: true
type: choice
options:
- verygoodplugins/automem
- verygoodplugins/mcp-automem
- verygoodplugins/automem-graph-viewer
source_sha:
description: "Commit SHA or tag (defaults to main)"
required: false
reason:
description: "What changed and why docs need updating"
required: true
jobs:
update-docs:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
id-token: write
steps:
- uses: actions/checkout@v6
- name: Validate and resolve source payload
id: resolve
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN }}
run: |
SOURCE_REPO="${{ github.event.client_payload.source_repo || inputs.source_repo }}"
SOURCE_REF="${{ github.event.client_payload.source_sha || inputs.source_sha || 'main' }}"
REASON="${{ github.event.client_payload.reason || inputs.reason || 'Automated dispatch' }}"
if [ "$SOURCE_REPO" != "verygoodplugins/automem" ] && \
[ "$SOURCE_REPO" != "verygoodplugins/mcp-automem" ] && \
[ "$SOURCE_REPO" != "verygoodplugins/automem-graph-viewer" ]; then
echo "::error::Unsupported source repo: $SOURCE_REPO"
exit 1
fi
FULL_SHA=$(gh api "repos/$SOURCE_REPO/commits/$SOURCE_REF" -q '.sha')
echo "source_repo=$SOURCE_REPO" >> $GITHUB_OUTPUT
echo "source_sha=$FULL_SHA" >> $GITHUB_OUTPUT
echo "reason=$REASON" >> $GITHUB_OUTPUT
- name: Checkout source repo at exact SHA
uses: actions/checkout@v6
with:
repository: ${{ steps.resolve.outputs.source_repo }}
ref: ${{ steps.resolve.outputs.source_sha }}
path: .source-repo
token: ${{ secrets.RELEASE_PLEASE_TOKEN }}
- name: Setup Node.js + install deps
uses: actions/setup-node@v6
with:
node-version: 24
- run: npm ci
- name: Run Claude Code Action
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
branch_prefix: docs/audit-
prompt: |
Read .github/prompts/update-docs.md for full instructions.
Source repo: ${{ steps.resolve.outputs.source_repo }}
Source SHA: ${{ steps.resolve.outputs.source_sha }}
Base SHA: ${{ github.event.client_payload.base_sha || 'N/A' }}
Changed files: ${{ toJSON(github.event.client_payload.changed_files) }}
Affected doc pages: ${{ toJSON(github.event.client_payload.affected_docs) }}
Reason: ${{ steps.resolve.outputs.reason }}
The source repo is checked out at .source-repo/ for you to read.
After making changes, run `npm run build` to verify the site compiles.
claude_args: >-
--model claude-sonnet-4-6
--max-turns 100
--allowedTools 'Read'
--allowedTools 'MultiEdit'
--allowedTools 'Edit'
--allowedTools 'Write'
--allowedTools 'Glob'
--allowedTools 'Grep'
--allowedTools 'Bash(git status --short:*)'
--allowedTools 'Bash(git --no-pager diff --no-ext-diff:*)'
--allowedTools 'Bash(git --no-pager log --no-ext-diff:*)'
--allowedTools 'Bash(npm run build)'The --allowedTools list is explicit and intentional. Claude gets read/write file access and specific git subcommands. No arbitrary shell, no curl, no npm install. You don’t want an AI with unrestricted shell access in a CI runner that has write access to your docs repo.
The SHA resolution step matters: the dispatch sends whatever SHA triggered it — could be a short SHA, a tag, or a branch name. Resolving to a full 40-char SHA before checkout means the PR body always references an immutable commit.
Step 4: The prompt (update-docs.md)
# Documentation Update Instructions
You are updating the AutoMem documentation site (Astro Starlight) to reflect
changes in the source code.
## Your task
1. Read the changed source files in `.source-repo/`
2. Read the affected doc pages listed in the workflow context
3. Update ONLY the sections actually affected by the code changes
4. Preserve all existing content that is still accurate
5. Maintain the same writing style, depth, and formatting
## Rules
- This is a SURGICAL UPDATE, not a rewrite.
- Keep every table, code example, and technical detail that's still correct.
- Do NOT summarize, shorten, or "improve" existing content.
- Do NOT change frontmatter unless the title/description genuinely needs updating.
- After making changes, run `npm run build` to verify the site compiles.
- If you find env vars / API fields / config options that exist in the source
but are missing from the docs entirely, LIST THEM in the PR body under
"Open question (not fixed — needs maintainer input)" rather than adding them
without context.“SURGICAL UPDATE, not a rewrite” is doing a lot of work in that prompt. Without it, the model will helpfully restructure your whole page in its own voice. The last rule — surface undocumented things as open questions rather than silently adding them — is what makes auto-merge safe.
Step 5: Auto-merge with a dead man’s switch (docs-auto-merge.yml)
name: Docs Auto-Merge
on:
pull_request_target:
types: [opened, reopened, synchronize, ready_for_review]
permissions:
contents: write
pull-requests: write
jobs:
enable-auto-merge:
if: >-
${{
github.event.pull_request.base.ref == 'main' &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'docs/audit-') &&
startsWith(github.event.pull_request.title, 'docs:') &&
!github.event.pull_request.draft
}}
runs-on: ubuntu-latest
concurrency:
group: docs-auto-merge-${{ github.event.pull_request.number }}
cancel-in-progress: false
steps:
- name: Check for unresolved audit notes
id: guard
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || github.token }}
PR_URL: ${{ github.event.pull_request.html_url }}
run: |
BODY="$(gh pr view "$PR_URL" --json body --jq '.body // ""')"
if printf '%s\n' "$BODY" | grep -Eiq \
'follow-up questions|open question|not addressed|maintainer input|needs maintainer|unresolved'; then
echo "PR contains unresolved audit language; leaving for maintainer."
echo "eligible=false" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "eligible=true" >> "$GITHUB_OUTPUT"
- name: Enable auto-merge
if: steps.guard.outputs.eligible == 'true'
env:
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || github.token }}
PR_URL: ${{ github.event.pull_request.html_url }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
PR_JSON="$(gh pr view "$PR_URL" --json state,isDraft,autoMergeRequest)"
[ "$(echo "$PR_JSON" | jq -r '.state')" != "OPEN" ] && exit 0
[ "$(echo "$PR_JSON" | jq -r '.isDraft')" = "true" ] && exit 0
[ "$(echo "$PR_JSON" | jq -r '.autoMergeRequest != null')" = "true" ] && \
echo "Auto-merge already enabled." && exit 0
gh pr merge "$PR_URL" --auto --squash --match-head-commit "$HEAD_SHA"The guard fires only on branches prefixed docs/audit- with titles prefixed docs: — exactly what the Claude Code Action produces. Any other PR is untouched.
The guard regex: follow-up questions|open question|not addressed|maintainer input|needs maintainer|unresolved. If Claude used any of that language anywhere in the PR body, auto-merge is skipped and the PR lands in your queue.
What it looks like in practice
Since the pipeline went live in February, these are just some of the PRs that have merged without human involvement:
- docs: fix drift in System Overview
- docs: fix drift in Direct API vs MCP Tools
- docs: fix drift in Local Setup
- docs: fix drift in CLI Setup
- docs: fix drift in MCP Bridge
- docs: fix drift in Relationship Operations
- docs: fix drift in Authentication
- docs: fix drift in Hybrid Search
- docs: fix drift in Consolidation & Decay
- docs: fix drift in Recall Operations
- docs: fix drift in Enrichment Pipeline
- docs: fix drift in Embedding Generation
- docs: fix drift in Environment Variables
- docs: fix drift in Configuration Reference
- docs: fix drift in Memory Operations API
Here’s what a PR body looks like when Claude finds something but isn’t sure whether to add it:
Fixes
| Claim | Current state | Fix |
|—|—|—|
|VOYAGE_MODELabsent from Embeddings table |provider_init.pyreadsos.getenv("VOYAGE_MODEL", "voyage-4")— real, user-settable env var | Added row (default:voyage-4) |
|OLLAMA_TIMEOUTabsent | readsos.getenv("OLLAMA_TIMEOUT", "30")| Added row (default:30sec) |
|OLLAMA_MAX_RETRIESabsent | readsos.getenv("OLLAMA_MAX_RETRIES", "2")| Added row (default:2) |Open question (not fixed — needs maintainer input)
The page says “Complete reference for all AutoMem environment variables.” The following exist in
config.pybut are absent from this page — intentionally omitted or should they be documented?| Variable | Default |
|—|—|
|SEARCH_WEIGHT_METADATA|0.35|
|RECALL_RELEVANCE_GATE|0.0|
|SEARCH_RECENCY_WINDOW_DAYS|180|
|IDENTITY_SYNTHESIS_ENABLED|"false"|
| … 8 more … |
The three env vars it was confident about got documented and queued to auto-merge. The 12 intentionally-undocumented internal vars are flagged for a decision. That’s the system working as designed — ship the obvious fixes, surface the judgment calls.
Copy this for your own project
Files:
- docs-dispatch.yml — add to each source repo; update the
automem-websiteURL and your repo names in the curl - file-doc-map.json — build your own; start with 5–10 high-churn files
- docs-update.yml — add to your docs repo; update the allowlisted source repos
- update-docs.md — customize for your stack and doc conventions
- docs-auto-merge.yml — works as-is; adjust the guard regex if needed
- benchmarks-update.yml — optional, only if you have deterministic data to sync without AI
Secrets:
RELEASE_PLEASE_TOKEN— a GitHub PAT withreposcope, for cross-reporepository_dispatchand private repo checkoutsANTHROPIC_API_KEY— for the Claude Code Action
Total: ~300 lines of YAML, one JSON map, one markdown prompt. The pipeline took about four months to get right — the CI was straightforward, the prompt engineering took iteration, and the auto-merge guard logic needed several hardening passes to handle edge cases. But it’s been running clean since May and the docs stay honest.