Skip to content

fix(openai): folded system messages keep their JSON blocks - #138

Merged
M3gA-Mind merged 2 commits into
tinyhumansai:mainfrom
M3gA-Mind:fix/fold-system-json-blocks
Oct 23, 2026
Merged

M3gA-Mind merged 2 commits into
tinyhumansai:mainfrom
M3gA-Mind:fix/fold-system-json-blocks

Conversation

@M3gA-Mind

Copy link
Copy Markdown
Collaborator

Summary

single_system_message (strict local templates that need one leading system turn) and merge_system_into_user (endpoints that reject the system role) gathered system content with a text-only helper. It dropped ContentBlock::Json blocks and joined a message's blocks with "\n". The pass-through path (translate_text_content) serializes JSON blocks and joins blocks with nothing between them.

So a system message with a JSON block lost that block on a hoisting or merging endpoint, and a multi-block system message was sent with different bytes than on every other route. The fix renders folded system content the way the pass-through path does: text verbatim, JSON serialized, reasoning dropped, no separator inside a message. Separate system messages are still joined with "\n\n", and the merged-text shape is unchanged: one text block in front of the user's content, so text-only user turns stay a plain string and image turns keep one text part.

Related issue

None. Found in an edge-case audit of the OpenHuman host (consumed via tinyagents). Not tracked as an issue.

API or behavior changes

No public API change. Behaviour: on hoisting/merging endpoints, a system JSON block is now sent (it was dropped), and multi-block system messages join their blocks without a newline, matching the pass-through path. Single-text-block system messages, the common case, are byte-identical.

Validation

  • cargo fmt on the touched files (rustfmt --edition 2021)
  • cargo clippy --all-targets --all-features -- -D warnings: left to CI.
  • cargo build --all-targets --all-features: left to CI.
  • cargo test -p tinyinference-llm --lib -- folded_system_messages single_system merge_system: with the fix, 10 passed. With the original content_text restored, the new test failed and the 9 existing folding tests passed.

Tests

folded_system_messages_keep_their_json_blocks_as_sent_unfolded (providers/openai/mod_tests.rs): a system message [Text("Context: "), Json({"tenant":"acme"})] hoists to Context: {"tenant":"acme"} and merges into the user turn as Context: {"tenant":"acme"}\n\nhi. The existing folding tests (ordering, empty system messages, image preservation, promotion) are unchanged and pass.

Documentation

The helper's doc comment states the rule. No user docs affected.

Checklist

  • The change is focused on one logical change
  • No new #[allow(...)], #[ignore], or relaxed lints
  • No secrets, tokens, or .env contents in the diff or the description

`single_system_message` and `merge_system_into_user` gathered system
content with a text-only helper that dropped JSON blocks and joined blocks
with a newline. The pass-through path (`translate_text_content`) serializes
JSON blocks and joins blocks with nothing between them, so a prompt sent to a
hoisting or merging endpoint differed from the same prompt sent as-is, and
lost any structured block. Render folded system content the same way.
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fd02d338-3a23-4278-9408-e64f2c9ea226
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@tinysweeper

tinysweeper Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

Reviewed head: be71b6eb8107d01779e17b6af7a35c0cb0bdb9e8

Verdict: Ready for maintainer review · Priority: none
Coverage: Reviewed 2 of 2 files.
Pull request purpose (title): fix(openai): folded…

Diagnostics and supporting details

Full summary

Pull request purpose (title): fix(openai): folded system messages keep their JSON blocks. Tiny Sweeper completed its review; deterministic results follow.

Review snapshot

Change surface Files Review signal Count
Production 1 Active findings 0
Tests 1 Noted findings 0
Documentation 0 Resolved findings 0
Configuration 0 Pending checks/questions 0

Completeness: Complete
Test assessment: No supported feature-to-test mapping was available; this does not mean tests are absent or passed.

What changed

`content_text` in the OpenAI transport previously filtered to text-only blocks and joined with `"\n"`, so a system JSON block was lost and multi-block system content was sent with different bytes than on pass-through routes. It now clones text, serializes JSON blocks via `to_string()`, still drops reasoning, and collects blocks with no separator, aligning with `translate_text_content`. The doc comment was rewritten to state the rule. Separate system messages are still joined with `"\n\n"` by the callers, and the merged-text shape is unchanged. No public API change.

Features

  • Modified — Fold system content like the pass-through path: On hoisting (`single_system_message`) and merging (`merge_system_into_user`) endpoints, a system JSON block is now sent rather than dropped, and multi-block system messages join blocks without a newline, matching bytes sent on other routes. Single-text-block system messages remain byte-identical. (crates/tinyinference-llm/src/providers/openai/transport.rs pub(super) fn auth_headers(auth: &AuthStyle, api_key: &str) -> Vec<(String, Stri)
  • Added — Test for folded system JSON blocks: Pins the hoisted (`Context: {"tenant":"acme"}`) and merged (`Context: {"tenant":"acme"}\n\nhi`) shapes against the literal wire string, so regression on JSON dropping or the join separator would fail. (crates/tinyinference-llm/src/providers/openai/mod_tests.rs fn merge_system_preserves_user_image_blocks() {)

Tests

No supported feature-to-test mapping was produced. Test execution is not inferred.

Findings

No active actionable findings.

Before merge

None.

Agent review details

critique

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Critique lane answered with no findings: the change aligns `content_text` with `translate_text_content`, and both callers were checked against `translate_text_content` with no behavioral regression found; empty-system handling and joined-with-nothing semantics match.
  • Lane summary: The change aligns `content_text` with `translate_text_content`'s rendering (text verbatim, JSON serialized, reasoning dropped, no separator) and adds a test covering both the hoist and merge paths. I checked the two callers, `merge_system_into_user` and `single_system_message`, against `translate_text_content` and found no behavioural regression: the joined-with-nothing semantics match, empty-system handling is unchanged, and the new test only asserts behaviour the code now implements. _The code index is behind this pull request (indexed at `d1e131a36424`), so retrieved context may be out of date._

security

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Security lane answered with no findings: small self-consistent refactor, no untrusted input reaching a new sink, no auth/network/subprocess changes, and the test uses synthetic data.
  • Lane summary: This change is a small, self-consistent refactor: `content_text` now serializes JSON blocks instead of dropping them and joins with no separator, matching the doc comment and the new test's pinned expectation. No untrusted input reaches a new sink, no auth/network/subprocess changes, and the test uses synthetic data. I was unable to verify that merging multiple system messages with no separator can't silently concatenate two JSON blocks ambiguously, but nothing in the diff shows a concrete failure for reachable input. _The code index is behind this pull request (indexed at `d1e131a36424`), so retrieved context may be out of date._

tests

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Tests lane answered with no findings: the new test pins both hoisted and merged shapes against the literal wire string, covering the behavioral change.
  • Lane summary: The change makes `content_text` fold system content the way the pass-through path does (text verbatim, JSON serialized, no separator, reasoning dropped), and adds a test pinning both the hoisted and merged shapes against the literal wire string. The new test fails if JSON blocks are dropped again or the join separator changes, so the behavioural change is covered; the remaining arms (`Thinking`/others ignored) are the pre-existing fall-through. No defects found. _The code index is behind this pull request (indexed at `d1e131a36424`), so retrieved context may be out of date._

commits

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Commits lane answered with no findings: no sensitive content in committed files.
  • Lane summary: No sensitive content in committed files. _The code index is behind this pull request (indexed at `d1e131a36424`), so retrieved context may be out of date._

description

  • Conclusion: Success
  • Scope reviewed: all assigned evidence
  • Positive: Description lane answered with no findings: the description matches the diff, including the clone/serialize/no-separator behavior and the test paths described.
  • Lane summary: The description matches the diff: `content_text` now clones text, serializes `ContentBlock::Json`, and joins blocks without a separator, mirroring `translate_text_content` in `convert.rs`; the new test exercises exactly the hoisting and merging paths described. No findings. _The code index is behind this pull request (indexed at `d1e131a36424`), so retrieved context may be out of date._
Evidence and run details
  • Models: ladder/vectors-oai3, deepseek-v4.1-flash
  • Spend: $0.000592
  • Tokens: 201594 input · 3055 output · 20992 cached · 181 embedding
  • Updated: 2026-10-23T00:36:47Z
Head State Pass summary
be71b6eb8107 ready for maintainer review 0 active finding(s), 0 resolved finding(s) (at 2026-10-23T00:36:47Z)

tinysweeper 0.1.2

@tinysweeper tinysweeper Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tinysweeper found nothing blocking. Approving.

Diagnostics

Total spend: $0.0006

@M3gA-Mind
M3gA-Mind merged commit 0e7d43d into tinyhumansai:main Oct 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant