Repository navigation
fix(core): redact request targets and preserve tool schema order - #966
LIghtJUNction merged 4 commits into
Conversation
…property order Port QuantumNous/new-api 30748c6558bc97de49ff74871cfec717e50f3322 and d0b840e75ff36d731b04ff981ee361819acd593f into services/core. Access logs mask credential query keys, and tool schemas sent upstream keep the client's property order. Fixes #944 Co-authored-by: LIghtJUNction <lightjunction.me@gmail.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
小时巡检（Asia/Taipei 2026-10-26 ~16:35）：
|
|
小时巡检：默认分支 tip 已前进到 |
|
小时巡检续：Update branch 后重跑结果——identity 再次 cancelled（run |
|
小时巡检（Asia/Taipei 2026-10-26 ~18:40）：
|
…lly [skip ci] Preserve selected-route authority and strict honeypot boundaries. Replace global schema substring matching and the custom lexer with bounded structural serde processing. Add independently authored protocol and real middleware regressions.
LIghtJUNction
changed the title
Requests with credential-bearing query strings must not disclose those fields in access logs, and tool schema property order must survive conversion without changing an earlier same-value tool argument. This revision removes the old global JSON substring replacement and custom lexer, while preserving current selected-route/deferred-request policy and the strict honeypot boundary. Fixes #944.
The logger keeps only bounded model/alt/api-version/stream values; all other query keys and values are omitted, URL userinfo/fragment are removed, and malformed targets return a fixed marker. Schema order is restored only at the actual protocol/tool/index path, using bounded serde RawValue parsing plus an indexed ordered object. Full Value equality and the existing encoded-request limits remain enforced. Global JSON ordering, billing hashes and money authority are unchanged.
Current independent review and local verification
ChatGPT Pro code review and separate implementation/test owners accepted the fixed v2 business patch. Its SHA256 is
6d5a2f5859fdec2676bacd03c3e5314c7cd98df747dcdc0f453fc37e2c269308; the additional real-middleware test changes test code only. Frozen source base:a2e98793aa2ca1c27825665e32805c9b1584f222.The combined scoped snapshot passed 309 tests, with one preexisting opt-in benchmark ignored and not counted passed. It covers protocol conversion, selected-route/strict transport, actual loopback HTTP, the actual Go-extension stop/stream case, isolated PostgreSQL authorization/unknown-money/lineage cases, and the independently authored schema/log regressions. Actual production middleware was exercised through Axum over four loopback HTTP requests in a child process; captured stderr had the expected 401/404/200 safe targets and no code/state/unknown-secret material, while a query-less health request was silent.
Strict
cargo clippy --locked --all-targets -- -D warningsand maintained-doc validation passed on that snapshot. The old routing source-bridge wiring failure was separately corrected without deleting its real assertions; its library routing and upstream-cost groups are both retained. That separate current-WIP test-wiring commit isef05110cdee1f6d8b689df3fd651c6801d768a9c. Original failed runs and later corrected fixture/infrastructure runs are retained as distinct evidence. No CI reruns or workflow dispatches.This is scoped local source acceptance, not production deployment, real OAuth/provider acceptance, paid model/image calls, release performance or whole-project acceptance. The source and behavior below describe the current revision; the original author record is preserved separately.
Original frozen head 4c2ab13 author description and historical validation
变更说明
问题与关联 Issue/PR：
Fixes #944。把 QuantumNous/new-api 的两处后端修复移植到 Rust
services/core：30748c6558bc97de49ff74871cfec717e50f3322（fix(logging): 隐藏访问日志中的查询凭据）properties保持客户端键序：d0b840e75ff36d731b04ff981ee361819acd593f（fix(relay): keep the client's JSON Schema property order）为什么采用此方案：
访问日志原先不记录 URL。一旦请求带 query，凭证仍可能进入 stderr。新增最外层访问日志，只在 request-target 含
?时写一行，并按上游规则脱敏。serde_json的Map是BTreeMap。全局打开preserve_order会改变计费、滥用策略、路由和工具市场所用 JSON 的 SHA-256。因此只在工具 schema 上保留客户端键序：解析原始 schema 字节，确认与已校验的Value语义相同后，再拼回上游 body。对不上则仍发送排序后的规范字节。复用了什么，替代或删除了什么：
行为按上述两个上游提交重新实现，没有拷贝 Go 源码，也没有改
apps/api-go、apps/web或其他仓库。上游发送从serde_json::to_vec改为Request::upstream_body。Request::upstream仅测试使用。可能破坏的行为、外部契约或数据路径：
core_access method=… status=… target=…。不含对端地址，query 中的凭证值被替换。无 query 的请求（含/health/live）不写这行。上游来源与许可证影响（不适用时说明）：
上游仓库 QuantumNous/new-api，提交
30748c6558bc97de49ff74871cfec717e50f3322与d0b840e75ff36d731b04ff981ee361819acd593f。本仓库保持 LMM 品牌、NOTICE 与 AGPL。本次是行为移植，不是整段复制；来源写在log_redact与schema_order的模块注释里。未改 NOTICE。Query 键：拒绝与保留
拒绝（大小写不敏感，先 TrimSpace）的精确键：
key、api_key、api-key、apikey、x-api-key、access_token、refresh_token、id_token、token、authorization、auth、client_secret、secret、password、passwd、signature、sig、awsaccesskeyid、x-amz-credential、x-amz-security-token、x-amz-signature。另外，规范化后的键名只要包含子串
token、secret或signature也拒绝（例如session_token、nottoken、client_secret）。author与authentication不因此被拒绝。auth只按精确键拒绝。掩码值为
***masked***。写入 query 时*编码为%2A。键名保留，只替换值。访问日志会先按 Go
Query().Encode()的方式重编码：无效百分号转义被丢弃，含;的片段被丢弃，键按解码后的字节序排列。然后做脱敏。因此alt、api-version、model、stream、client_id、code、state等运维键仍然可见。整段 query 被去掉的路径只有解码后的
/api/oauth/与/oauth/前缀。/api/oauth2/不去掉；其上的code与state会原样留下（它们不在拒绝表中）。sanitize_url_for_log在解析后没有任何敏感键时返回原字符串（含无效转义的单独key=bad%ZZ）。绝对 URL 的 userinfo（user:pass@）按上游url.URL.String()保留，不在本次拒绝范围内。访问日志记的是 request-target，不含 userinfo。JSON Schema 键序范围
覆盖：Chat / Responses 的 function
parameters，Anthropicinput_schema，Gemini OpenAPIparameters（类型改为小写，属性序保留）和parametersJsonSchema。本 relay 没有
response_format/json_schema结构化输出路径，本次不新增。重复键沿用现有语义：值取后者，键位取首次出现。改动文件
services/core/src/log_redact.rs（新）services/core/src/relay/schema_order.rs（新）services/core/src/lib.rsservices/core/src/main.rsservices/core/src/relay/mod.rsservices/core/src/relay/request.rsservices/core/src/relay/honeypot.rsservices/core/src/relay/native_anthropic.rsservices/core/src/relay/native_gemini.rs验证证据
审查基点 SHA：
3ecb3ede394f418b7d2b5e761d979879ad99a2d9（wip/rust-core-go-extensions）实际测试的 HEAD 与工作区摘要：
9d2fe268ae68c1d5cffbaf96e01d59d5aa09484c，工作区干净。工具链services/core/rust-toolchain.toml的 Rust 1.99.0。修复前失败、修复后通过的命令与日志：
在
services/core执行，均退出码 0：cargo fmt --allcargo clippy --all-targets --message-format=short（无 warning）cargo test --lib log_redact:: -- --test-threads=8：3 passedcargo test --lib relay:: -- --test-threads=8：152 passed（含 schema 键序 4 项与既有 relay 回归）黄金用例覆盖上游访问日志样例：
key/token/api_key/access_token/secret等不以明文出现；model、alt、api-version保留。Schema 往返后properties仍为reasoning然后answer，而serde_json排序结果相反。重构的行为保持证据，或明确的业务语义差异：
既有 relay 单测仍比较
upstream()的Value，不比较字节序。键序只影响实际上游 body。语义不等的拼接会退回排序字节。quick / related / full 的报告、失败、受阻及未执行项目：
本次是 related 的
services/core库测试与 clippy，不是 fulljust test-core，未跑 PostgreSQL 集成，未做独立验收。用户可见变化的浏览器证据；核心变化的同环境资源对比（适用时）：
无 UI。不适用浏览器。
必须独立审查的内容
门禁、测试断言、类型配置、阈值、锁文件、基线或例外的变更与理由：
无锁文件、阈值或 CI 门禁改动。新增断言锁定脱敏黄金字符串和 schema 键序。
高风险业务、临时方案的责任人、到期时间与退出条件：
日志脱敏影响安全审计可见性：拒绝表过宽会挡住排障，过窄会漏凭证。当前表与上游一致，并在上文列出保留键。拼接 schema 失败时退回排序编码，请求不会因此失败。
独立审查者与证据（尚未执行请写 pending，作者不能自行批准）：
pending。作者自测不能作为独立签收。
提交前核对
拒绝/保留键写在
services/core/src/log_redact.rs模块文档和本说明中，未另开文档。未包含 Claude cache writes billing（94cc3ff0）或 channel custom body（7fb6e57a）。提交不代表允许合并、发布或部署。