Skip to content

test(failure): pin that provider detail scrubs a key the provider echoes - #139

Merged
M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:test/provider-detail-scrubs-echoed-keys
Oct 23, 2026
Merged

M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
M3gA-Mind:test/provider-detail-scrubs-echoed-keys

Conversation

@M3gA-Mind

Copy link
Copy Markdown
Collaborator

Summary

  • Adds extract_provider_error_detail_scrubs_a_key_the_provider_echoes (failure_tests.rs). When a provider echoes the rejected key in its JSON error message, the extracted detail keeps the message and drops the key. Covers OpenAI-shaped (sk-proj-…) and Anthropic-shaped (sk-ant-api03-…) keys.

Why

extract_provider_error_detail (and with_provider_detail) quote the provider's message to the user after sanitize_api_error. Nothing tested the key-echo case. OpenHuman's host test for it was flagged in review (tinyhumansai/openhuman#8301) as belonging here, because the scrubbing is this crate's behaviour. The host keeps one smoke check.

Testing

  • cargo test -p tinyinference-llm --lib -- extract_provider_error_detail: 5 passed.
  • With sanitize_api_error(trimmed) replaced by trimmed.to_string() in failure/text.rs, the new test fails on the key-leak assertion. The file was restored afterwards.
  • The keys are synthetic and built at runtime, so no key-shaped literal is committed.
  • rustfmt --edition 2024 --check is clean.

Test-only. No behaviour change.

🤖 Generated with Claude Code

extract_provider_error_detail runs the quoted provider message through
sanitize_api_error, and that detail is shown to users. No test covered a
provider echoing the rejected key back in its error message. Pin it for
OpenAI- and Anthropic-shaped keys.
@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e3ddeeb9-9ae7-48b2-b4e2-38f11d9611c9
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@tinysweeper

tinysweeper Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: 32ec67f8bfacc3a44b2cfe08ca492903dcc73e82

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 1 of 1 files.
Pull request purpose (title): test(failure): pin t…

Diagnostics and supporting details

Full summary

Pull request purpose (title): test(failure): pin that provider detail scrubs a key the provider echoes. This test-only pull request adds `extract_provider_error_detail_scrubs_a_key_the_provider_echoes` to `crates/tinyinference-llm/src/failure_tests.rs` to pin that the extracted provider error detail keeps the human-readable message while dropping a provider-echoed key, covering both OpenAI-shaped (`sk-proj-…`) and Anthropic-shaped (`sk-ant-api03-…`) synthetic keys built at runtime. No implementation changed.

Review snapshot

Change surface Files Review signal Count
Production 0 Active findings 0
Tests 1 Noted findings 0
Documentation 0 Resolved findings 0
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

What changed

Adds a single test to `crates/tinyinference-llm/src/failure_tests.rs` that builds synthetic key-shaped strings at runtime, feeds a provider JSON error echoing the key into `extract_provider_error_detail`, and asserts the returned detail contains the message text but not the key body, for both OpenAI- and Anthropic-shaped keys. No production code was modified.

Features

  • Added — Key-echo scrubbing regression test: Pins that the extracted provider error detail retains the human-readable message while excluding an echoed `sk-`-shaped key, exercised for both OpenAI- and Anthropic-shaped keys; it can fail if scrubbing stops working, and its fixtures are synthetic and runtime-built so no key-shaped literal is committed. (crates/tinyinference-llm/src/failure_tests.rs, crates/tinyinference-llm/src/failure_tests.rs fn extract_provider_error_detail_pulls_openai_message() {)

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The critique lane (answered) reviewed the single changed file against `extract_provider_error_detail` and `sanitize_api_error`/`scrub_secret_patterns`, found the flow consistent with the assertions and the fixture JSON valid, and reported no defects.
  • Lane summary: The new test builds two `sk-…`-shaped keys at runtime and asserts the extracted provider detail both keeps the human-readable text and does not contain the key body. I checked `extract_provider_error_detail` and `sanitize_api_error`/`scrub_secret_patterns` at the reviewed commit; the flow (`sanitize_api_error` runs `OPENAI_KEY_REGEX` over the extracted message and replaces the whole `sk-…` token) is consistent with the assertions, and the fixture's JSON is valid escaped JSON so parsing reaches the `"message"` value. The assertions are behaviour this change introduces, so there is nothing regressed and no rule from the repo's instruction files is violated. No defects found. _The code index is behind this pull request (indexed at `be71b6eb8107`), so retrieved context may be out of date._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The security lane (answered) reported no findings in this file and noted the assertion targets the 32-character key body rather than the `sk-proj-`/`sk-ant-api03-` prefix; it could not read `sanitize_api_error` from its position, so it did not confirm whether the redaction covers that specific body, and it also noted the fixture is synthetic with no committed key-shaped literal.
  • Lane summary: The new test asserts that `extract_provider_error_detail` scrubs a provider-echoed key. The implementation delegates to `tinyinference_core::sanitize::sanitize_api_error`, and the test's assertion `!detail.contains(&body)` targets the 32-character key body, not the `sk-proj-`/`sk-ant-api03-` prefix. I could not read `sanitize_api_error` from here, so I cannot confirm whether the redaction it performs covers this specific 32-char alphanumeric body; the test itself is a synthetic, runtime-built fixture (no committed key-shaped literal) and is not a security defect. No findings in this file. _The code index is behind this pull request (indexed at `be71b6eb8107`), so retrieved context may be out of date._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Lane summary: The diff adds a single test pinning the existing scrubbing behaviour of `extract_provider_error_detail`: it asserts the payload is retained and that the synthetic key body appears nowhere in the returned detail, for both `sk-proj-` and `sk-ant-api03-` shapes, exercising the `sanitize_api_error` path in the function under test. No implementation changed, so there is no behavioural regression to catch and no new branch lacking coverage. The assertions can fail if scrubbing stops working, the keys are built at runtime rather than committed as literals, and the test uses no network or environment. Nothing reportable. _The code index is behind this pull request (indexed at `be71b6eb8107`), so retrieved context may be out of date._

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The commits lane (answered) reported no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files. _The code index is behind this pull request (indexed at `be71b6eb8107`), so retrieved context may be out of date._

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: The description lane (answered) found the added test matches the description — pinning that the extracted detail scrubs an echoed `sk-`-shaped key while preserving the message for both shapes — and found no defect in the change.
  • Lane summary: The added test matches the description: it pins that `extract_provider_error_detail` scrubs an echoed `sk-`-shaped key while preserving the message, for both OpenAI- and Anthropic-shaped synthetic keys built at runtime. I found no defect in the change. _The code index is behind this pull request (indexed at `be71b6eb8107`), so retrieved context may be out of date._
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4.1-flash
  • Spend: $0.003111
  • Tokens: 87982 input · 2362 output · 38160 cached · 103 embedding
  • Updated: 2026-10-23T03:23:56Z
Head State Pass summary
32ec67f8bfac ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-10-23T03:23:56Z)

tinysweeper 0.1.2

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0031

@M3gA-Mind
M3gA-Mind merged commit b21593f into tinyhumansai:main Oct 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant