Skip to content

feat(ollama): shared stdlib-only Ollama HTTP client - #226

Closed
raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/ollama-client
Closed

raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/ollama-client

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

设计说明

问题:多个模块各自重复实现 Ollama HTTP 调用(cerebellum_core、router_cascade、token_saver_mcp_server、mcp-servers/ollama-mcp),且底层混用 urllib/httpx,行为不一致。

解法:一个共享的、零第三方依赖的 Ollama 接入层——只做 HTTP 封装 + 数据提取,异常向上抛,由调用方保持各自的降级策略。

关键设计决策:

  • 零第三方依赖:仅标准库 urllib(统一原 httpx 路径)
  • 低层接入:不动配置、不动默认模型,只收"怎么调"
  • 兼容双接口:/api/generate(补全式)与 /api/chat(聊天式)都支持;流式/非流式都支持
  • 超时默认 120s,可调

边界情况:

  • 网络/解析异常向上抛,调用方自行决定降级
  • host 带/不带尾斜杠均可(_url() 统一处理)

测试建议:对 /api/tags、/api/generate、/api/chat 各构造一个 mock 响应验证解析

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor Author

关键设计决策补充

  • 零第三方依赖的动机: 这个模块收敛 4 个已有的重复实现,其中 3 个用 urllib、1 个用 httpx。用 urllib 统一后,新模块无需 pip install,CMake/打包时少一个依赖。
  • 为什么不做配置收敛? 各调用方(cerebellum、router_cascade、token_saver)有不同的 env 前缀和默认模型——统一配置会破坏现有行为。本层只处理"怎么调",不碰"调谁"。
  • 流式支持: 返回 (chunk_iter, final_dict) 二元组,调用方可以逐块处理或等完整响应。

@Zongwei9888

Copy link
Copy Markdown
Collaborator

Thank you for the submission, @raymondginger2018-sudo. Closing because the problem it solves is not in this repository: cerebellum_core, router_cascade, token_saver_mcp_server and mcp-servers/ollama-mcp do not exist here (no matches in the tree), and Ollama is served through the OpenAI-compatible local provider entry in core/providers/registry.py. A synchronous urllib client would also not fit the fully async provider layer. If an Ollama-specific feature is missing from the OpenAI-compatible path (for example a native endpoint the compatible API does not expose), an issue describing it would help us scope the right change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants