Skip to content

[bugfix] agent: fix Chinese text corruption in Windows CLI input - #323

Open
curealb wants to merge 1 commit into
ThinkFlowLab:mainfrom
curealb:fix/windows-cli-unicode
Open

curealb wants to merge 1 commit into
ThinkFlowLab:mainfrom
curealb:fix/windows-cli-unicode

Conversation

@curealb

@curealb curealb commented Nov 13, 2026 •

Copy link
Copy Markdown

Summary

On Windows PowerShell 5.1, piping JSON to opendirector ... --file - encoded non-ASCII text as ? before the CLI received it, so commands could succeed while saving corrupted Chinese titles. Reading Unicode back via opendirector project show also failed under default Windows Python stdout encodings.

  • Keep .opendirector-bin/opendirector.cmd as the Windows Agent CLI entry point (compatible with Restricted execution policy) and invoke Python with -X utf8 so CLI output preserves Unicode on readback.
  • When bundled or external OpenCode selects system Windows PowerShell 5.1, wrap it with a private launcher that sets $OutputEncoding and [Console]::OutputEncoding to UTF-8 without a BOM inside that process—no prompt instructions, profile edits, or execution-policy changes needed.
  • Decode JSON from both stdin (--file -) and --file <path> as strict UTF-8 with an optional BOM (utf-8-sig), rejecting invalid bytes before dispatching commands.

Linked issue

  • No separate issue applies.

Labels

Suggested labels: type:bug, area:agent, area:harness, area:docs.

  • Labels applied

Test plan

11 tests passed locally on Windows at commit 3b64793c. Two end-to-end tests covered bundled OpenCode 1.19.5 and standalone OpenCode 1.19.6. Each ran a fresh turn and a resumed turn in the same Agent session, with PYTHONUTF8=0 and PYTHONIOENCODING unset. The model responses were simulated. OpenCode, Windows PowerShell 5.1, the CLI, and project storage were real.

  1. Run bare opendirector --help under Restricted. In the Agent shell, PowerShell 5.1 blocked a temporary .ps1 script with PSSecurityException. In that same process, Get-Command opendirector resolved to .opendirector-bin\opendirector.cmd, and opendirector --help printed usage and exited 0. These checks passed on both turns in both runtimes.

  2. Preserve Chinese titles and prompts through a plain JSON pipeline. The model piped literal Unicode JSON to opendirector canvas create-generation --file - and opendirector asset write-text --file - without setting any encoding variables. The Agent then ran opendirector project show. The test located the node by ID and compared its title and two-line prompt with the original input. An independent read from reopened project storage confirmed the same values, including 窗边的小猫 🐈 1, 𠮷, quotes, a literal ?, and the newline. The saved text asset matched the expected UTF-8 bytes exactly. The project recorded zero generation jobs.

  3. Verify CLI input and launcher behavior. Nine additional tests passed. They covered JSON from stdin and files with and without a UTF-8 BOM, including a file written by PowerShell 5.1 with Set-Content -Encoding UTF8. They also verified that invalid UTF-8 was rejected before command dispatch, that the launcher preserved command behavior and exit codes, and that cancellation terminated the child PowerShell process.

The two end-to-end cases are in tests/test_native_runtime.py::test_real_opencode_windows_unicode_pipeline. The nine additional cases are in tests/test_windows_powershell.py.

Evidence that it works

Original reproduction and earlier revision

Before

After the earlier fix

Web verification on this revision

In a manual web run, the Default Agent created 窗边的小猫 🐈 with the two-line prompt below (zero generation jobs). Both opendirector project show and persisted storage matched verbatim:

镜头一:午后的阳光照进窗户。
镜头二:纸条上写着「𠮷,今天会下雨吗?」。

(Note: in this manual trace the model also set encoding variables before the pipe; the automated tests above verify the launcher when the model omits them and under enforced Restricted.)

image

After restarting the local web service and reopening the project, 窗边的小猫 🐈 and both prompt lines (including the ?) remained intact in the canvas and node editor:

image

Provider verification

  • Automated tests simulated model responses while running real OpenCode (1.19.5 and 1.19.6), Windows PowerShell 5.1, CLI, and project storage.
  • The manual web check on this revision used gpt-6.1-sol and saved the node without calling an image provider (the original before/after screenshots used gpt-6.1-sol and gpt-image-2.5-flare).

Bilingual UI

  • No UI strings changed. Chinese titles and prompts are preserved as supplied.

maker.variants = {''}
maker.make('opendirector = opendirector.cli:main')
(cli_bin / 'opendirector.cmd').unlink(missing_ok=True)
(cli_bin / 'opendirector.ps1').write_text(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PowerShell can reject this .ps1 launcher under Restricted, even for opendirector --help. Please keep the CLI usable under the existing execution policy and verify from the Windows Agent sandbox that --help works and a JSON pipeline preserves Chinese titles and prompts in project storage.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching the .ps1 launcher issue. In 3b64793c, I stopped generating the .ps1 launcher and kept .opendirector-bin\opendirector.cmd as the Windows Agent CLI entry point. It starts Python with -X utf8 for Unicode readback. When OpenCode selects system Windows PowerShell 5.1, a private launcher sets UTF-8 in the process that sends the JSON.

I verified both cases you asked about from the Windows Agent shell:

  1. Bare opendirector --help under Restricted. With Restricted in effect, PowerShell 5.1 rejected a temporary .ps1 with PSSecurityException. In that same process, Get-Command opendirector resolved to .opendirector-bin\opendirector.cmd, and opendirector --help printed usage and exited 0.

  2. Chinese titles and prompts through a plain JSON pipeline. The model piped literal Unicode JSON to opendirector canvas create-generation --file - without setting any encoding variables. The Agent then ran opendirector project show. Both the CLI readback and an independent read from reopened project storage matched the original title (窗边的小猫 🐈 1) and two-line prompt, including 𠮷, quotes, a literal ?, and the newline. The saved text asset also matched the expected UTF-8 bytes exactly. No generation jobs were submitted.

Both end-to-end tests passed, one with bundled OpenCode 1.19.5 and one with standalone OpenCode 1.19.6. Each covered a fresh turn and a resumed turn with PYTHONUTF8=0 and PYTHONIOENCODING unset. The model responses were simulated. OpenCode, PowerShell 5.1, the CLI, and project storage were real. Nine additional CLI input and launcher tests passed, for 11 total.

I’ve updated the PR description with the test details and screenshots of the node before and after restarting the service and reopening the project. Ready for another look.
这个版本呢

@curealb
curealb force-pushed the fix/windows-cli-unicode branch from 14dc698 to 3b64793 Compare November 20, 2026 22:13
@github-actions github-actions Bot added area:harness Harness and skill integration (Default, OpenCode, Codex, Claude Code, Jiuwen, DSH) type:bug Something is broken labels Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent runtime, assistant chat and tools area:docs Documentation / 文档 area:harness Harness and skill integration (Default, OpenCode, Codex, Claude Code, Jiuwen, DSH) type:bug Something is broken

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants