Skip to content

fix: show REPEAT_ERROR in replay verdicts #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ruslanlap merged 1 commit into from
Sep 27, 2026
Merged

fix: show REPEAT_ERROR in replay verdicts #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions loopguard.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ def build_parser() -> argparse.ArgumentParser:

replay = sub.add_parser("replay", help="Print an annotated event-by-event replay with detector verdicts.")
replay.add_argument("path", help="Path to a .jsonl file or directory.")
replay.add_argument(
"--repeat-n", type=int, default=REPEAT_N, metavar="N",
help=f"Consecutive failed bash threshold for REPEAT_ERROR (default: {REPEAT_N}).",
)
return p


Expand Down Expand Up @@ -501,12 +505,13 @@ def _process_new_lines(file_handle) -> int:

# ── Replay ─────────────────────────────────────────────────────────────────────

def replay_file(filepath: str, *, loop_n: int = LOOP_N, loop_m: int = LOOP_M, osc_n: int = 4) -> int:
def replay_file(filepath: str, *, loop_n: int = LOOP_N, loop_m: int = LOOP_M, osc_n: int = 4, repeat_n: int = REPEAT_N) -> int:
"""
Print one annotated line per tool event with the detector verdict at that
point. Doubles as a debug surface and an honest terminal demo.
"""
window: deque = deque(maxlen=loop_m)
streak = 0
exit_code = 0
try:
fh = open(filepath, "r", encoding="utf-8", errors="replace")
Expand All @@ -526,8 +531,11 @@ def replay_file(filepath: str, *, loop_n: int = LOOP_N, loop_m: int = LOOP_M, os
window.append(fp)
osc, _ = detect_oscillation(window, osc_n)
loop_hit, _ = detect_loop(window, loop_n)
verdict = "⚠ OSCILLATION" if osc else ("⚠ LOOP" if loop_hit else "ok")
if osc or loop_hit:
streak = update_repeat_streak(streak, ev)
repeat_hit = streak >= repeat_n
verdict = ("⚠ OSCILLATION" if osc else ("⚠ LOOP" if loop_hit
else ("⚠ REPEAT_ERROR" if repeat_hit else "ok")))
if osc or loop_hit or repeat_hit:
exit_code = 1
print(f"{i:>4} {ev['tool']:<24} {verdict}", flush=True)
return exit_code
Expand Down Expand Up @@ -567,7 +575,7 @@ def main(argv: list[str] | None = None) -> int:
return 2

if args.command == "replay":
return replay_file(target)
return replay_file(target, repeat_n=args.repeat_n)

@devin-ai-integration devin-ai-integration Bot Sep 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Replay command crashes on every transcript

For any existing transcript, main reads args.repeat_n, but the replay parser never defines it. Replay exits with AttributeError before printing verdicts.

Learn more

The replay subparser only registers path in build_parser. Argparse therefore creates no repeat_n attribute for replay. Reading it before invoking replay_file raises AttributeError, so even ordinary replay commands fail. The new test checks return code 1, which is also the exit code from this unhandled exception.

Example: loopguard replay fixtures/repeat_error.jsonl prints a traceback and exits 1 instead of printing three annotated events. loopguard replay fixtures/repeat_error.jsonl --repeat-n 5 rejects the option and exits 2.

Recommended fix: Register --repeat-n with the replay subparser in build_parser, using the same default and integer type as watch. Extend the CLI test to assert the expected verdict, successful quiet replay with a higher threshold, and no traceback.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@ruslanlap ruslanlap Sep 28, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already fixed — this finding is against a stale intermediate push. The PR branch was force-updated from bd65b86 (which had the AttributeError bug) to 287cba9 before merge; 287cba9 already registers --repeat-n on the replay parser (build_parser, replay subparser) and passes repeat_n=args.repeat_n to replay_file. Verified on current main (4a1a3ae): loopguard.py replay fixtures/repeat_error.jsonl → prints verdicts, no AttributeError; replay --repeat-n 5 → quiet, exit 0. Also note test_replay_detects_repeat_error asserts on stdout content (REPEAT_ERROR verdict), not just the exit code, so it can't pass via a crash.

@chatgpt-codex-connector chatgpt-codex-connector Bot Sep 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Define repeat_n for the replay subcommand

Every replay invocation now accesses args.repeat_n, but --repeat-n is registered only on the watch parser, so replay exits with an AttributeError before processing any events. Add the option to the replay parser or pass the existing default without reading a missing namespace attribute.

Useful? React with 👍 / 👎.

@ruslanlap ruslanlap Sep 28, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already fixed — this finding is against a stale intermediate push. The PR branch was force-updated from bd65b86 (which had the AttributeError bug) to 287cba9 before merge; 287cba9 already registers --repeat-n on the replay parser (build_parser, replay subparser) and passes repeat_n=args.repeat_n to replay_file. Verified on current main (4a1a3ae): loopguard.py replay fixtures/repeat_error.jsonl → prints verdicts, no AttributeError; replay --repeat-n 5 → quiet, exit 0. Also note test_replay_detects_repeat_error asserts on stdout content (REPEAT_ERROR verdict), not just the exit code, so it can't pass via a crash.


return watch_file(
target,
Expand Down
12 changes: 12 additions & 0 deletions tests/test_loopguard.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -518,3 +518,15 @@ def test_replay_detects_loop(self):
self.assertEqual(r.returncode, 1)
self.assertIn(b"LOOP", r.stdout)
self.assertIn(b"bash", r.stdout)

def test_replay_detects_repeat_error(self):
lines = "\n".join(
f'{{"tool":"bash","tool_input":{{"cmd":"make {i}"}},"exit_code":1}}' for i in range(3)
) + "\n"
with tempfile.TemporaryDirectory() as d:
p = Path(d) / "s.jsonl"
p.write_text(lines, encoding="utf-8")
r = subprocess.run([sys.executable, "loopguard.py", "replay", str(p)],
capture_output=True, timeout=20)
self.assertEqual(r.returncode, 1)
self.assertIn(b"REPEAT_ERROR", r.stdout)
Loading