Repository navigation
feat(harness): tell text-only models when read omits an image - #102
Merged
Merged
Conversation
Tool calls now carry the input modalities of the model they answer (`ToolCallContext.modelInput`). On a model without image input, `read` adds a note that the image will be omitted, next to the placeholder the transform puts in its place. A session test reads an image on a text-only model, switches to a vision model and snapshots both requests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ToolCallContextgets an optionalmodelInputfield: the input modalities (["text"],["text", "image"]) of the model the call answers. The loop fills it from the resolved model. It is optional, so existing plugins are unaffected, and tool hooks receive it too.readadds[Current model does not support images. The image will be omitted from this request.]to its note when the model has no image input. This covers both a processed image and one that failed to process. With no model known, nothing is added.(tool image omitted: model does not support images)on text-only models. This adds the tool-side note, so the model knows why the image is missing.Test
packages/harness/test/tools.read.vision.test.tsruns the full harness with a scripted model and two custom models declared in settings. It readsred-circle.pngon a text-only model, then switches to a vision model and asks again.__artifacts__/tools.read.vision.jsonrecords what each request carries:image/png.Checks
vp run typecheckandvp lintpass.vp checkreports formatting only on the vendored skill markdown files and.claude/launch.json. Those files are unchanged from the base and untouched by this PR.