Skip to content

[3/5] feat(llm): Score recent LLM churn - #150

Open
thedrow wants to merge 7 commits into
iepathos:masterfrom
thedrow:feat/llm-churn
Open

thedrow wants to merge 7 commits into
iepathos:masterfrom
thedrow:feat/llm-churn

Conversation

@thedrow

@thedrow thedrow commented Oct 12, 2026 •

Copy link
Copy Markdown

Stacked on #149 (which is stacked on #148). Review only the last commit (feat(llm): Score recent LLM churn).

Summary

Detect LLM-co-authored commits (Co-authored-by trailers and "Generated with" footers; built-in agent globs, extendable in config) and score how LLM edits to each function hold up. Only recent commits count (default 14-day window):

  • repeated LLM edits in the window,
  • fixes after an LLM edit,
  • reverts of LLM edits (matched by the This reverts commit <hash> line, or landing shortly after one),
  • bursts of conventional fixes grouped by fix(scope):, or by file for scope-less fix: commits.

These signals add up to a rework feature that lowers the score and adds an advice action. Older LLM edits that were never fixed or reverted add a small, capped bonus. Commit facts come from the existing repo-wide history walk, so there is no extra git pass. Configure with [llm_editability.churn].

Uses git2 trailer parsing, git-conventional, and globset.

This is PR 3 of 5. Next: topic matching (#151).

Test plan

  • Unit tests for commit facts, agent matching, reverts and fix bursts
  • llm_churn_integration_test: builds a real git repo fixture with agent trailers and backdated commits (repo-local core.hooksPath, no global config changes), checks per-function churn via the git provider and via debtmap analyze --format json
  • cargo test --bin debtmap, output_validation_test

Not yet done

  • Documentation: the book and README do not yet cover the [llm_editability.churn] section (window, agents, proven-edit bonus) and the churn fields in the JSON/markdown output.

How to stack these

These pull requests target master. From a clone of iepathos/debtmap, link them bottom to top, then merge the stack:

gh stack link 148 149 150 151 152
gh stack merge

gh stack link records the stack on GitHub. gh stack merge merges that stack in one step. Pass a pull request number to stop at that pull request. Order: CRAP scores (#148), editability (#149), churn (#150), topics (#151), content reverts (#152).

CRAP (Change Risk Anti-Patterns) combines cyclomatic complexity with test
coverage: cc^2 * (1 - cov)^3 + cc. It is computed at output time from the
existing complexity and coverage data, so no analysis types change.

- JSON: metrics.crap is a number with coverage data and null without it,
  so consumers can parse the key unconditionally.
- Terminal and LLM markdown output show it next to coverage.
- [crap].coverage_source selects direct (default) or transitive coverage.
@thedrow thedrow changed the title [3/3] feat(llm): Score recent LLM churn [3/5] feat(llm): Score recent LLM churn Oct 13, 2026
Score each function's editability per model tier (small to frontier)
from complexity, nesting, length, coverage, context tokens and ripple,
with ranked improvement actions. Opt-in via [llm_editability].
Lower LLM editability when LLM co-authored edits keep getting reworked:
repeated recent LLM edits, fixes or reverts after them, and bursts of
conventional fixes per scope (or per file for scope-less fix: commits).
Older LLM edits never fixed or reverted add a small, capped bonus.
Agents are detected from Co-authored-by trailers and Generated-with
footers; configure via [llm_editability.churn].
@thedrow thedrow mentioned this pull request Oct 17, 2026
2 tasks done
Reject windows above 100 years so date arithmetic cannot panic,
and carry the earlier editability review fixes onto this branch.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant