Skip to content

Commit e694e23

Browse files
feat: cut a streaming answer short when new guidance arrives
Guidance injected before the next LLM call still had to wait for the answer that was already streaming, so a long text answer could not be redirected until it finished on its own. The session now tracks the streaming request separately from the turn. When the user sends guidance while an answer is streaming, that request is aborted, whatever the model already wrote is kept as an assistant message marked meta.interrupted, and the loop immediately issues a new request that carries the guidance. Tools are never interrupted: steering only targets the model stream, so a running command finishes first and the guidance is injected at the next request boundary. - session: steerActiveSession(), a per-request steer signal released once the request settles, and LlmSteeredError carrying the partial answer - session: a steered answer is not an interruption - the turn continues, the status stays completed, and the request is not retried - settings: steerMode ("interrupt" by default, "queue" keeps the old behaviour) - cli: submitting while busy calls steerActiveSession() when steerMode is "interrupt"; the cut-short answer is labelled in the transcript - tests: core covers the aborted stream, the follow-up request payload, the kept partial answer, and the no-op case; settings cover steerMode resolution - docs: configuration table and section, plus the README notes Refs #113, #117
1 parent f9ff3ae commit e694e23

13 files changed

Lines changed: 363 additions & 9 deletions

File tree

‎README-en.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -98,9 +98,12 @@ Skills are discovered from these locations, in priority order:
9898
While a turn is running, `Enter` no longer blocks and does not have to wait: the prompt becomes
9999
supplemental guidance, appended to the conversation right before the model's next step of that
100100
turn. The model therefore reads it together with the work it already did and can revise or
101-
supersede the earlier instructions. Up to 10 messages can wait; press `Backspace` on an empty
102-
prompt to drop the last one. `Esc` still interrupts the turn immediately, and slash commands
103-
still wait for the turn to finish.
101+
supersede the earlier instructions. When the model is in the middle of writing an answer, that
102+
answer is cut short and whatever it already wrote stays in the conversation; set
103+
`steerMode: "queue"` to wait for the next request boundary instead. Up to 10 messages can wait;
104+
press `Backspace` on an empty prompt to drop the last one. Running commands are never
105+
interrupted, `Esc` still interrupts the turn immediately, and slash commands still wait for the
106+
turn to finish.
104107

105108
## Supported Models
106109

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,9 @@ Skills 会按以下优先级扫描:
9696

9797
AI 正在回复时,`Enter` 不再被拒绝,也不必等到本轮结束:消息会作为补充指引,在本轮下一次
9898
LLM 调用前作为 user 消息追加到对话中。模型因此能读到它,并结合已完成的工作修改甚至推翻之前的
99-
指令。最多可排队 10 条;输入框为空时按 `Backspace` 可移除最后一条。按 `Esc` 仍会立即中断本轮,
100-
斜杠命令仍需等待本轮结束。
99+
指令。如果模型正在输出回答,该回答会被截断,已生成的内容仍保留在对话里;可用 `steerMode: "queue"`
100+
改为等到下次请求边界再注入。最多可排队 10 条;输入框为空时按 `Backspace` 可移除最后一条。
101+
正在执行的命令不会被中断,按 `Esc` 仍会立即中断本轮,斜杠命令仍需等待本轮结束。
101102

102103
## 支持的模型
103104

‎docs/configuration.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ Deep Code 使用 `settings.json` 设置文件进行持久化配置,支持两
3333
| `thinkingEnabled` | boolean | 是否启用思考模式(DeepSeek V4 系列默认启用) |
3434
| `reasoningEffort` | string | 推理强度,可选 `"low"`、`"high"` 或 `"max"`(默认 `"max"`) |
3535
| `multimodal` | string | 多模态(图片)能力开关,可选 `"default"`、`"on"` 或 `"off"`(默认 `"default"`) |
36+
| `steerMode` | string | 模型正在输出时发送新指令的处理方式:`"interrupt"`(截断当前回复、立即读取新指令,默认)或 `"queue"`(等到下次请求边界再注入) |
3637
| `filesApiEnabled` | boolean | 是否通过 DeepSeek Files API 发送图片(默认 `false`) |
3738
| `filesApiTimeoutMs` | number | 单张图片 Files API 处理超时,默认 `60000`,最大 `600000` 毫秒 |
3839
| `fileExpiresAfterSeconds` | number | 远端文件有效期,默认 `604800` 秒 |
@@ -106,6 +107,19 @@ Deep Code 使用 `settings.json` 设置文件进行持久化配置,支持两
106107

107108
当使用的模型未内置在已知模型列表中、或其实际能力与默认判定不符时,可通过该配置覆盖。
108109

110+
#### `steerMode` — 模型输出中追加指令
111+
112+
控制模型还在输出回答时你发送新指令的处理方式:
113+
114+
| 值 | 说明 |
115+
| ----------- | ------------------------------------------------------------ |
116+
| `interrupt` | 截断正在流式输出的回答,已生成的内容保留在对话中,新指令立即被读取(默认值) |
117+
| `queue` | 新指令进入队列,在本轮下一次 LLM 调用前注入 |
118+
119+
两种方式都会把新指令作为本轮对话中的 user 消息,模型因此可以修改甚至推翻它原本正在执行的指令。
120+
正在执行的命令不会被中断:steering 只影响模型输出流,因此工具会先执行完,新指令在下一个请求边界注入。
121+
按 `Esc` 仍会中断整个轮次。
122+
109123
#### DeepSeek Files API
110124

111125
当 `BASE_URL` 为 `https://api.deepseek.com` 时,设置 `filesApiEnabled: true` 后,Deep Code 会将图片上传到固定的 `https://api.deepseek.com/files`,并在聊天请求中使用 `file_id`。其他 API 地址不会启用该功能。上传或缓存刷新失败时,本次请求直接失败;关闭开关时图片处理逻辑保持不变。

‎docs/configuration_en.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ The following are all the top-level fields supported in `settings.json`, along w
3333
| `thinkingEnabled` | boolean | Whether to enable thinking mode (enabled by default for DeepSeek V4 series)|
3434
| `reasoningEffort` | string | Reasoning intensity: `"low"`, `"high"`, or `"max"` (default `"max"`) |
3535
| `multimodal` | string | Multimodal (image) capability override: `"default"`, `"on"`, or `"off"` (default `"default"`) |
36+
| `steerMode` | string | How a prompt sent while the model is writing is handled: `"interrupt"` (cut the answer short so the prompt is read immediately, default) or `"queue"` (wait for the next request boundary) |
3637
| `filesApiEnabled` | boolean | Send images through the DeepSeek Files API (default `false`) |
3738
| `filesApiTimeoutMs` | number | Per-image Files API timeout; defaults to `60000`, maximum `600000` ms |
3839
| `fileExpiresAfterSeconds` | number | Remote file lifetime, default `604800` seconds |
@@ -106,6 +107,20 @@ Controls whether the current model is treated as a multimodal model that accepts
106107

107108
Use this to override the default detection when your model is not in the known-model list, or when its actual capability differs from the default.
108109

110+
#### `steerMode` — Steering While the Model Is Writing
111+
112+
Controls what happens when you send a prompt while the model is still producing an answer:
113+
114+
| Value | Description |
115+
| ----------- | --------------------------------------------------------------------------- |
116+
| `interrupt` | The answer that is streaming is cut short; its text is kept in the conversation and the prompt is read immediately (default) |
117+
| `queue` | The prompt waits and is injected before the next LLM call of the running turn |
118+
119+
Either way the prompt becomes a user message of the running turn, so the model can revise or
120+
supersede the instructions it was already following. Running commands are never interrupted:
121+
steering only affects the model stream, so a tool that is executing finishes first and the
122+
prompt is injected at the next request boundary. Pressing `Esc` still interrupts the whole turn.
123+
109124
#### DeepSeek Files API
110125

111126
When `BASE_URL` is `https://api.deepseek.com`, enabling `filesApiEnabled` uploads images to the fixed `https://api.deepseek.com/files` endpoint and sends `file_id` references in chat requests. Other API endpoints do not enable this feature. An upload or cache-refresh failure fails the request; disabling the setting preserves the existing image path.

‎packages/cli/src/tests/exec-runner.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ function createSettings(
3333
debugLogEnabled: false,
3434
telemetryEnabled: false,
3535
multimodal: "default",
36+
steerMode: "interrupt",
3637
filesApiEnabled: false,
3738
filesApiTimeoutMs: 60_000,
3839
fileExpiresAfterSeconds: 604_800,

‎packages/cli/src/ui/components/MessageView/index.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ export function MessageView({ message, collapsed, width = 80 }: MessageViewProps
8888
return <Text key={i}>{seg.body}</Text>;
8989
})
9090
: null}
91+
{message.meta?.interrupted ? <Text dimColor>— superseded by your guidance</Text> : null}
9192
</Box>
9293
</Box>
9394
);

‎packages/cli/src/ui/views/App.tsx‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -566,11 +566,17 @@ function App({ projectRoot, initialPrompt, resumeSessionId, forkSessionId, onRes
566566
imageUrls: submission.imageUrls,
567567
skills: submission.selectedSkills,
568568
});
569+
// With `steerMode: "interrupt"` (the default) the answer that is streaming is
570+
// cut short so the guidance is read straight away; `"queue"` only waits for the
571+
// next request boundary. Running tools are never interrupted.
572+
if (resolveCurrentSettings(projectRoot).steerMode !== "queue") {
573+
sessionManager.steerActiveSession();
574+
}
569575
return;
570576
}
571577
void handlePrompt(submission);
572578
},
573-
[handlePrompt, sessionManager]
579+
[handlePrompt, projectRoot, sessionManager]
574580
);
575581

576582
const handlePlanImplementationChoice = useCallback(

‎packages/core/src/common/llm-retry.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,26 @@ export class LlmStreamDisconnectedError extends Error {
3131
}
3232
}
3333

34+
/**
35+
* Raised when a streaming answer is cut short because the user sent new guidance
36+
* while the model was still writing. The partial answer is carried along so the
37+
* caller can keep it in the conversation before continuing with the guidance.
38+
*/
39+
export class LlmSteeredError extends Error {
40+
constructor(
41+
readonly partialContent: string,
42+
readonly partialThinking: string | null
43+
) {
44+
super("Model answer was superseded by new user guidance.");
45+
this.name = "LlmSteeredError";
46+
}
47+
48+
/** True when the model had already produced something worth keeping. */
49+
hasPartialAnswer(): boolean {
50+
return this.partialContent.trim().length > 0 || (this.partialThinking ?? "").trim().length > 0;
51+
}
52+
}
53+
3454
export function getLlmRetryDelayMs(attempt: number, random: () => number = Math.random): number {
3555
const exponentialDelay = BASE_RETRY_DELAY_MS * 2 ** Math.max(0, attempt - 1);
3656
const jitter = 0.9 + Math.min(1, Math.max(0, random())) * 0.2;

‎packages/core/src/index.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ export type {
3535
PermissionDefaultMode,
3636
McpServerConfig,
3737
ReasoningEffort,
38+
SteerMode,
3839
StatusLineSettings,
3940
ResolvedStatusLineSettings,
4041
StatusLineProviderConfig,

‎packages/core/src/session.ts‎

Lines changed: 103 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ import {
8181
LLM_STREAM_IDLE_TIMEOUT_MS,
8282
LlmStreamDisconnectedError,
8383
LlmStreamIdleTimeoutError,
84+
LlmSteeredError,
8485
MAX_LLM_RETRIES,
8586
waitForLlmRetry,
8687
} from "./common/llm-retry";
@@ -316,6 +317,8 @@ export type MessageMeta = {
316317
userPrompt?: UserPromptContent;
317318
/** User guidance that arrived while the turn was already running. */
318319
isSupplementary?: boolean;
320+
/** Assistant output that was cut short because new guidance superseded it. */
321+
interrupted?: boolean;
319322
};
320323

321324
export type SessionMessage = {
@@ -461,6 +464,10 @@ export class SessionManager {
461464
/** Supplemental guidance that arrived while a turn was already running. */
462465
private readonly supplementaryPrompts = new Map<string, SupplementaryPrompt[]>();
463466
private supplementaryPromptNextId = 1;
467+
/** Abort handle of the LLM request each session is currently streaming. */
468+
private readonly steerControllers = new Map<string, AbortController>();
469+
/** Sessions whose in-flight answer was superseded by new user guidance. */
470+
private readonly steeredSessions = new Set<string>();
464471
private readonly processTimeoutControls = new Map<string, ProcessTimeoutControl>();
465472
private readonly liveProcessKeys = new Set<string>();
466473
private readonly toolExecutor: ToolExecutor;
@@ -723,7 +730,12 @@ export class SessionManager {
723730
try {
724731
return await this.createChatCompletionStreamAttempt(client, request, options, sessionId, debug, requestId);
725732
} catch (error) {
726-
if (signal?.aborted || retryCount >= MAX_LLM_RETRIES || !isRetryableLlmError(error)) {
733+
if (
734+
error instanceof LlmSteeredError ||
735+
signal?.aborted ||
736+
retryCount >= MAX_LLM_RETRIES ||
737+
!isRetryableLlmError(error)
738+
) {
727739
throw error;
728740
}
729741
const attempt = retryCount + 1;
@@ -970,6 +982,11 @@ export class SessionManager {
970982
throw new LlmStreamDisconnectedError();
971983
}
972984
} catch (error) {
985+
if (sessionId && this.steeredSessions.has(sessionId)) {
986+
// The user sent new guidance while this answer was streaming. Hand the
987+
// partial answer to the caller instead of reporting a failure.
988+
throw new LlmSteeredError(content, reasoningContent.length > 0 ? reasoningContent : null);
989+
}
973990
const streamError = idleTimedOut ? new LlmStreamIdleTimeoutError() : error;
974991
this.logChatCompletionDebug(debug, {
975992
timestamp: new Date().toISOString(),
@@ -1579,6 +1596,79 @@ ${agentInstructions}
15791596
return pending.length;
15801597
}
15811598

1599+
/**
1600+
* Cut short the answer the model is currently writing so the guidance the user
1601+
* just sent is read now instead of after that answer finishes.
1602+
*
1603+
* Nothing is interrupted when no answer is streaming (for example while a tool
1604+
* is executing): the guidance then waits for the next request boundary, so a
1605+
* running command is never killed by normal typing. Returns whether an answer
1606+
* was cut short.
1607+
*/
1608+
steerActiveSession(): boolean {
1609+
const sessionId = this.activeSessionId;
1610+
if (!sessionId) {
1611+
return false;
1612+
}
1613+
1614+
const controller = this.steerControllers.get(sessionId);
1615+
if (!controller) {
1616+
return false;
1617+
}
1618+
1619+
this.steeredSessions.add(sessionId);
1620+
if (!controller.signal.aborted) {
1621+
controller.abort(new Error("Superseded by new user guidance."));
1622+
}
1623+
return true;
1624+
}
1625+
1626+
/**
1627+
* Keep the part of the answer the model produced before it was steered, so the
1628+
* conversation still shows what it had said when the user changed direction.
1629+
*/
1630+
private appendSteeredAnswer(sessionId: string, error: LlmSteeredError): void {
1631+
if (!error.hasPartialAnswer()) {
1632+
return;
1633+
}
1634+
1635+
const message = this.buildAssistantMessage(sessionId, error.partialContent, null, error.partialThinking ?? "");
1636+
message.meta = { ...(message.meta ?? {}), interrupted: true };
1637+
this.appendSessionMessage(sessionId, message);
1638+
this.onAssistantMessage(message, false);
1639+
}
1640+
1641+
/**
1642+
* Signal for one LLM request: it aborts when the turn is interrupted or when the
1643+
* answer is superseded by new guidance, and it is released once the request is
1644+
* settled so `steerActiveSession` only ever targets a live stream.
1645+
*/
1646+
private beginRequestSteer(sessionId: string, turnSignal: AbortSignal): { signal: AbortSignal; release: () => void } {
1647+
const steerController = new AbortController();
1648+
const forwardTurnAbort = () => {
1649+
if (!steerController.signal.aborted) {
1650+
steerController.abort(turnSignal.reason);
1651+
}
1652+
};
1653+
if (turnSignal.aborted) {
1654+
forwardTurnAbort();
1655+
} else {
1656+
turnSignal.addEventListener("abort", forwardTurnAbort, { once: true });
1657+
}
1658+
this.steerControllers.set(sessionId, steerController);
1659+
1660+
return {
1661+
signal: steerController.signal,
1662+
release: () => {
1663+
turnSignal.removeEventListener("abort", forwardTurnAbort);
1664+
if (this.steerControllers.get(sessionId) === steerController) {
1665+
this.steerControllers.delete(sessionId);
1666+
}
1667+
this.steeredSessions.delete(sessionId);
1668+
},
1669+
};
1670+
}
1671+
15821672
async handleUserPrompt(userPrompt: UserPromptContent): Promise<void> {
15831673
const controller = new AbortController();
15841674
this.activePromptController = controller;
@@ -1933,6 +2023,7 @@ ${agentInstructions}
19332023
references: [] as DeepSeekFileReference[],
19342024
};
19352025
const thinkingOptions = buildThinkingRequestOptions(thinkingEnabled, baseURL, reasoningEffort);
2026+
const steer = this.beginRequestSteer(sessionId, sessionController.signal);
19362027
const request = () =>
19372028
this.createChatCompletionStream(
19382029
client,
@@ -1943,7 +2034,7 @@ ${agentInstructions}
19432034
tools: getTools(this.getPromptToolOptions(), this.mcpToolDefinitions),
19442035
...thinkingOptions,
19452036
},
1946-
{ signal: sessionController.signal },
2037+
{ signal: steer.signal },
19472038
sessionId,
19482039
{
19492040
enabled: debugLogEnabled,
@@ -1956,6 +2047,12 @@ ${agentInstructions}
19562047
try {
19572048
response = await request();
19582049
} catch (error) {
2050+
if (error instanceof LlmSteeredError && !sessionController.signal.aborted) {
2051+
// The user changed direction while this answer was streaming: keep what
2052+
// the model already wrote, then let the loop continue with the guidance.
2053+
this.appendSteeredAnswer(sessionId, error);
2054+
continue;
2055+
}
19592056
if (!filesSettings.enabled || prepared.references.length === 0 || !this.isRejectedDeepSeekFile(error)) {
19602057
throw error;
19612058
}
@@ -1970,6 +2067,8 @@ ${agentInstructions}
19702067
sessionController.signal
19712068
);
19722069
response = await request();
2070+
} finally {
2071+
steer.release();
19732072
}
19742073

19752074
const message = response.choices?.[0]?.message;
@@ -2801,6 +2900,8 @@ ${agentInstructions}
28012900
}
28022901
this.sessionControllers.delete(sessionId);
28032902
this.setSupplementaryPrompts(sessionId, []);
2903+
this.steerControllers.delete(sessionId);
2904+
this.steeredSessions.delete(sessionId);
28042905
if (options.removeMessages) {
28052906
this.removeSessionMessages([sessionId]);
28062907
try {

0 commit comments

Comments
 (0)