Repository navigation
fix(model): keep the finish reason and accept empty tool arguments in response conversion - #136
Conversation
… response conversion
Tiny Sweeper reviewReviewed head: Verdict: Ready for maintainer review · Priority: none Diagnostics and supporting detailsFull summaryPull request purpose (title): fix(model): keep the finish reason and accept empty tool arguments in response conversion. Tiny Sweeper completed its review; deterministic results follow. Review snapshot
Completeness: Complete What changed`ChatResponse` gains a public `finish_reason: Option<String>` field with rustdoc, defaulting to `None`, plus a new `#[path = "response_conversion_tests.rs"]` test module in the conversion file. Conversion now normalizes empty/blank `arguments` to `{}` before parsing, and replaces the fixed `stop`/`tool_calls` assignment with a match that prefers the reported reason via `canonical_finish_reason` and falls back to the existing inference. A changelog entry under Unreleased documents the behavior and the breaking-field requirement. The existing `ChatResponse` literal in `response_tests.rs` is updated with `finish_reason: None`. Features
Tests
FindingsNo active actionable findings. Before mergeNone. Agent review detailscritique
security
tests
commits
description
Evidence and run details
|
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Comment |
Summary
What was wrong.
response_to_model_responsehad two gaps.serde_json::from_str. Several providers send a zero-argument call asarguments: "", which is not valid JSON, so the call came back marked invalid.tool_callswhen there were calls, otherwisestop.ChatResponsehad no way to carry the provider's reason, so a reply cut off by the output limit was reported as a cleanstop, and a caller could not tell to continue or retry.Fix.
argumentsare treated as{}before parsing. Other malformed arguments are still kept as invalid calls, unchanged.ChatResponsegainsfinish_reason: Option<String>. When it is set, the conversion keeps it (passed throughcanonical_finish_reason, somax_tokens/max_output_tokensbecomelength). When it is absent, the oldstop/tool_callsinference is used.Related issue
None.
API or behavior changes
ChatResponse::finish_reason(Defaultsets it toNone). This is breaking for downstream code that buildsChatResponsewith an exhaustive struct literal; such code must addfinish_reasonor use..Default::default(). Noted in the changelog. No version bump in this PR.{}.stop/tool_calls.Validation
cargo fmt --all -- --check. I ran rustfmt (edition 2024) on the touched files only; CI runs the full check.cargo clippy --all-targets --all-features -- -D warnings. Not run locally; CI runs it.cargo build --all-targets --all-features. Not run locally; CI runs it.cargo test --all-features. I ran the targeted tests below; CI runs the full suite.Tests
New
response_conversion_tests.rswith two tests.empty_or_blank_arguments_mean_an_empty_object"" marked invalidreported_finish_reason_is_kept. It also checks that a response with no reported reason still inferstool_calls.left: Some("stop"),right: Some("length")Also run after the fix:
cargo test -p tinyinference-llm --lib response(61 passed) and--lib usage::meta(15 passed), which cover the existing conversion tests and the updatedChatResponseliteral.Documentation
Changelog entry under Unreleased. The new field has rustdoc.
Checklist
#[allow(...)],#[ignore], or relaxed lints.envcontents in the diff or the description