Skip to content

[5/5] feat(llm): Detect reverts by content - #152

Open
thedrow wants to merge 10 commits into
iepathos:masterfrom
thedrow:feat/llm-content-reverts
Open

thedrow wants to merge 10 commits into
iepathos:masterfrom
thedrow:feat/llm-content-reverts

Conversation

@thedrow

@thedrow thedrow commented Oct 13, 2026 •

Copy link
Copy Markdown

Stacked on #151 (→ #150 → #149 → #148). Review only the last commit (feat(llm): Detect reverts by content).

Summary

Git has no "reverted" flag. git revert only writes This reverts commit <hash>. in the message, and that line can be edited away. GitHub's revert button references the PR instead of the commit. This PR adds opt-in detection of reverts from file content:

[llm_editability.churn]
content_reverts = true   # default false
  • The repo-wide commit walk already diffs every commit. It now also records the sorted (path, old blob, new blob) triples for each commit.
  • Commit C reverts an earlier commit X when C changes exactly the same files and restores each one to the blob it had before X. Matching is a single hash-map lookup per commit, with no extra diffing. If several earlier commits match, the latest wins. Re-applying a reverted change counts as reverting the revert.
  • Partial undos, or undos mixed with other edits, are not counted.
  • A detected revert is attached to the reverting commit's fact as if its message named the target, so it goes through the existing revert logic from [3/5] feat(llm): Score recent LLM churn #150: it adds rework when it undoes an LLM edit, and the reverted edit never counts as proven. Commits whose message already names a target keep it.

This is PR 5 of 5.

Test plan

  • Unit test: exact inverses pair up, partial undos don't, and re-applying reverts the revert
  • Integration test: Claude edits hot_parser, then a human restores the old version with the message "Restore previous tab handling". With the setting off, rework is 1. With it on, 1 reverted LLM edit is detected and rework is 2.

Not yet done

  • Documentation: the book and README do not yet cover the content_reverts setting under [llm_editability.churn].

How to stack these

These pull requests target master. From a clone of iepathos/debtmap, link them bottom to top, then merge the stack:

gh stack link 148 149 150 151 152
gh stack merge

gh stack link records the stack on GitHub. gh stack merge merges that stack in one step. Pass a pull request number to stop at that pull request. Order: CRAP scores (#148), editability (#149), churn (#150), topics (#151), content reverts (#152).

CRAP (Change Risk Anti-Patterns) combines cyclomatic complexity with test
coverage: cc^2 * (1 - cov)^3 + cc. It is computed at output time from the
existing complexity and coverage data, so no analysis types change.

- JSON: metrics.crap is a number with coverage data and null without it,
  so consumers can parse the key unconditionally.
- Terminal and LLM markdown output show it next to coverage.
- [crap].coverage_source selects direct (default) or transitive coverage.
Score each function's editability per model tier (small to frontier)
from complexity, nesting, length, coverage, context tokens and ripple,
with ranked improvement actions. Opt-in via [llm_editability].
Lower LLM editability when LLM co-authored edits keep getting reworked:
repeated recent LLM edits, fixes or reverts after them, and bursts of
conventional fixes per scope (or per file for scope-less fix: commits).
Older LLM edits never fixed or reverted add a small, capped bonus.
Agents are detected from Co-authored-by trailers and Generated-with
footers; configure via [llm_editability.churn].
@thedrow
thedrow force-pushed the feat/llm-content-reverts branch from 46423cf to f320ef4 Compare October 13, 2026 02:18
@thedrow
thedrow force-pushed the feat/llm-content-reverts branch from f320ef4 to f8124e2 Compare October 13, 2026 02:52
Opt-in [llm_editability.churn.topics]: compare recent fix commit
messages with each function's recent commits (stemmed, stopword-free
terms via bm25, IDF-weighted Jaccard). On-topic fixes form a third fix
group next to scopes and files. Fixes matching both scope and topic
add extra rework; on-topic fixes under other scopes add rework and are
reported as cross-scope fallout. fix_burst_scope becomes
fix_burst_group.
Opt-in [llm_editability.churn] content_reverts: a commit that changes
exactly the same files as an earlier commit and restores each to its
previous blob counts as reverting it, whatever its message says. The
existing commit walk records per-file blob ids, so matching is one
hash lookup per commit. Detected reverts feed the revert churn signal.
This branch contains the earlier LLM changes, so it needs the
same output, lookup, window, and topic-match corrections.
BSD-3-Clause is a permissive license in the same class as MIT.
The topic matcher pulls deunicode in through bm25.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant