Skip to content

fleet: viewers read a fixed sentence when the word check cannot answer - #1372

Merged
ejc3 merged 1 commit into
mainfrom
fix/staging-text-check
Oct 27, 2026
Merged

ejc3 merged 1 commit into
mainfrom
fix/staging-text-check

Conversation

@ejc3

@ejc3 ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner

A viewer word that the text check could not judge now reads as one of two fixed sentences instead of the checker's internals. On staging, typing "Hi!" showed "Hi!" was not added: the text check failed: no text check is configured.

Change

  • src/fleet/coord/text_check.rs: an error verdict's reason is words are turned off right now when the coordinator runs with --text-check none (NoCheck now answers with a Disabled error), and couldn't check that word, try again for any other error or the 5 s timeout. The error itself goes to the coordinator log at warn (text check failed error=...). The reason reaches the receipt, its check, the actions feed and the coordinator_degraded failure, all public.
  • fleet/core/src/apply.rs: an error verdict refuses with its reason as is, without the the text check failed: prefix. Hide AWS account ids in the errors the wall's view shows #1362's account-id redaction still runs over it.
  • fleet/core/tests/vectors/07_words.json, the web fixture regenerated from the vectors (one summary changes), fleet/PROTOCOL.md.

Staging also needs the checker turned on: ejc3/aws#707 sets --text-check bedrock with us.anthropic.claude-haiku-5-5 in us-west-2 and grants the coordinator's task role bedrock:InvokeModel. Until that is applied, staging viewers read "words are turned off right now".

Tests

Each fails on the base behavior and passes here:

  • viewers_read_a_fixed_reason_not_the_checker_error (unit): NoCheck gives WORDS_OFF, a failing checker gives CHECK_FAILED.
  • a_slow_check_times_out_as_an_error: a timeout gives CHECK_FAILED.
  • without_a_text_check_viewers_read_that_words_are_off (coordinator over its WebSocket): the receipt reason is WORDS_OFF.
  • a_failing_text_check_degrades_the_coordinator: the receipt, its check and the failures entry carry CHECK_FAILED, not "fake text check is down". It previously asserted the checker's error reached the failures list.
  • replayed_vectors_hold: the words vector's error summary.
make test-fleet-coordinator            # test result: ok. 71 passed
make test-unit FILTER="-E 'test(/text_check/) | package(fleet-core)'"
cd fleet/web && pnpm test              # 10 files, 258 tests passed

Bedrock check, run locally

The coordinator from this branch with --text-check bedrock --text-check-model us.anthropic.claude-haiku-5-5 --text-check-region us-west-2, words sent over /live from a dev box (round trip from send to receipt; one shared box, not a performance number):

word sky outcome time
Hi! committed 0.79 s
Hello from 8 GPUs Hi!, you, world, good morning committed 0.74 s
world Hi!, you committed 0.71 s
an insult Hi! refused 1.53 s
a phone number Hi! refused 0.74 s
kill Hi! refused 0.98 s

With --text-check none the receipt reads words are turned off right now. With a model id that does not exist it reads couldn't check that word, try again, and the log has text check failed error=Bedrock answered 400 Bad Request: The provided model identifier is invalid. All answers were well inside CHECK_TIMEOUT (5 s).

A viewer who wrote a word on staging read "the text check failed: no text
check is configured": apply.rs prefixed the checker's own error, and that
error also went into the receipt's check and the public failures list, where
a Bedrock error would name the model, region or AWS message.

- text_check.rs: an error verdict's reason is "words are turned off right
  now" for --text-check none (NoCheck answers with a Disabled error) and
  "couldn't check that word, try again" for any other error or the 5 s
  timeout. The error itself is logged at warn.
- apply.rs: an error verdict refuses with its reason as is.
- 07_words.json vector and the web fixture regenerated from it, PROTOCOL.md.
- Tests: viewers_read_a_fixed_reason_not_the_checker_error,
  without_a_text_check_viewers_read_that_words_are_off, and
  a_failing_text_check_degrades_the_coordinator now requires the fixed reason
  in the receipt, its check and the failures list. Each fails on the old
  behavior.
@ejc3

ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 5 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7438580e-42ef-4447-884b-4ff64476a40e
Commits

Reviewing files that changed from the base of the PR and between 83242ba and 551b341.

Files selected for processing: 6
  • fleet/PROTOCOL.md
  • fleet/core/src/apply.rs
  • fleet/core/tests/vectors/07_words.json
  • fleet/web/core/test/fixtures/coordinator.json
  • src/fleet/coord/text_check.rs
  • tests/test_fleet_coord.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T07:18:43.034746Z 551b341 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 551b3418d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ejc3
ejc3 merged commit 7bf7ddd into main Oct 27, 2026
6 of 15 checks passed
@ejc3
ejc3 deleted the fix/staging-text-check branch October 27, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant