doc-sync
Case study·4 min read

doc-sync

A Claude Code skill that keeps a repository’s documentation honest — by making it impossible to claim a document is current unless something actually checked it.

The problem

Doc automation stores one “last synced” commit and bumps it at the end of a run, which marks every document in the repository current — including the ones it never opened. The claim is false, permanent, and invisible.

The decision

Certify per document rather than per run: each verified doc carries its own stamp, and the global baseline is demoted to a scheduling hint that asserts nothing. Anything never checked stays visibly stale forever.

Almost every "keep the docs in sync" tool works the same way. It stores one commit hash — the last time docs were synced — diffs the repository from there to HEAD, edits whatever it decided to look at, and then bumps that hash to the new HEAD. That last step is the problem. Bumping one global marker asserts that every document in the repository is now current, including the hundreds the run never opened. The next run diffs from the new baseline, so the range that was skipped is never revisited. The claim is false, it is permanent, and nothing anywhere records that it was made.

doc-sync is a Claude Code skill that inverts the bookkeeping. It is not a documentation generator; it does not write docs from code. It reads a diff, works out which existing claims have become false, and corrects those — and it is deliberately unable to tell you that anything is current unless it actually checked.

Certification belongs to the doc, not to the run

Every document that is actually verified gets a stamp written into it, naming the commit it was verified against. The global baseline still exists, but it is demoted to a scheduling hint: it bounds the next diff and asserts nothing about completeness. A document no run ever looked at keeps its old stamp and shows up as stale, forever, until something genuinely checks it.

TypeScript
<!-- doc-sync: verified-at a1b2c3d -->

That one move is what makes the rest of it honest. Coverage stops being something the tool claims and becomes something you can read off the files themselves.

Staleness you can measure without a model

Because the evidence lives in the documents, the audit does not need a language model at all. A shell script walks the docs, pulls out every repository path they cite — backticked tokens beginning with one of your top-level directories, plus the usual root-level manifests — and existence-checks them against the tree. Then it reads the stamps back as an aging report: how many commits behind HEAD each document was last verified, and which are not stamped by anything at all.

Run it with a strict flag in CI and a document citing a file somebody deleted fails the build. No API key, no tokens, no non-determinism — the two failure modes it catches, a dead reference and an unverified claim, are both decidable without judgement.

Two tiers, because the blast radius differs

Agent instructions — CLAUDE.md, AGENTS.md and their kin — are read by every future agent session in the repository, so a wrong line there mis-steers all later work rather than merely misleading a reader once. Those are never edited automatically at any confidence level; doc-sync proposes and a human decides. Ordinary human documentation is different: it lives in the repo and inherits code review, so verified corrections are applied directly. Even then the edits are left staged and never committed, so the change still passes through a person who writes the commit message.

Note

doc-sync never commits and never pushes. That is not a missing feature — it is the boundary the whole design is arranged around.

Confidence has to mean verifiability

The thing that makes automated doc editing dangerous is a model that is confident rather than correct. So the bar is written down explicitly, and it is not about how sure the model feels. Only an edit that corrects a claim proved false against the working tree — and whose replacement text is itself checkable there — is allowed to apply unattended. Everything else escalates.

There is a hard ceiling on top of that: anything depending on knowing why a change was made is capped below the auto-apply threshold however obvious it looks, because a diff cannot evidence a reason. That is exactly the content most likely to be wrong and least likely to be caught in review, which is why it is the one category the tool refuses to guess at.

What is not finished

Every script ships in both bash and PowerShell, but the PowerShell ports have never been executed — they are careful line-by-line translations written on a machine that does not run them. The shell versions are the canonical ones. Saying so in the README costs nothing and is considerably cheaper than someone discovering it on Windows.