Skip to content

feat(providers): add AI Stupid Level as an OpenAI-compatible gateway - #276

Open
StudioPlatforms wants to merge 1 commit into
HKUDS:mainfrom
StudioPlatforms:feat/aistupidlevel-provider
Open

StudioPlatforms wants to merge 1 commit into
HKUDS:mainfrom
StudioPlatforms:feat/aistupidlevel-provider

Conversation

@StudioPlatforms

@StudioPlatforms StudioPlatforms commented Oct 23, 2026 •

Copy link
Copy Markdown

Disclosure: this PR comes from the AI Stupid Level team, so we are the vendor of the service it adds. It was prepared with Claude Code (Claude Opus 5.5) for the AI Stupid Level team. Happy to adjust anything to match project conventions, or to close it if a named provider is not wanted.

Description

Adds AI Stupid Level's Smart Router as a provider template: deepcode provider set my-asl --template aistupidlevel --api-key. The router is an OpenAI-compatible endpoint at https://aistupidlevel.info/v1 (API docs). It mirrors the Opper template from #242: a ProviderSpec on the generic openai_compat gateway path, the matching ProviderConfig block, and tests pinning the wiring. This time the ("aistupidlevel", "gateway") row in tests/test_provider_egress.py is in the PR itself; for Opper it was added at merge.

The router forwards each request with the user's own provider key (OpenAI, Anthropic, Google, DeepSeek, Kimi or GLM, saved on their AI Stupid Level account), so the provider bills the user directly. The terms say: "The provider bills you directly for that usage, under your agreement with it. We do not resell model access and do not mark up provider prices."

A request names one of two kinds of model id:

  • A routing mode (list) picks the model from AI Stupid Level's benchmark results: auto-coding for coding, auto-tooling for tool use.
  • A real model id such as claude-opus-4-7 pins that model.

Each routed response carries two headers: X-AISM-Model (the model that answered) and X-AISM-Reasoning (why it was chosen).

The trust boundary is the same as for any gateway. The router terminates TLS and reads prompts and tool results, and the user's provider keys are stored on their AI Stupid Level account. endpoint_class="gateway" says so, and providers.egress can block aistupidlevel.info. ASL's terms say: "You give us permission to process it only to provide the service to you. We do not use your requests to train models or to build our benchmarks."

Related Issues

None.

Changes Made

  • core/providers/registry.py: an aistupidlevel ProviderSpec after opper, with these values: AISTUPIDLEVEL_API_KEY, default base https://aistupidlevel.info/v1, endpoint_class="gateway", base keyword aistupidlevel.info, and key prefix aism_ (router keys start with it).
  • core/config.py: the matching ProviderConfig block.
  • tests/test_provider_egress.py: the ("aistupidlevel", "gateway") endpoint-class row.
  • tests/test_aistupidlevel_provider.py (new): pins the wiring and the two decisions below. Both decision tests build a real request through OpenAICompatProvider._build_kwargs.

Two provider-specific decisions:

  • strip_model_prefix is on, as for Forge. The router serves bare ids only. In a sandbox request to the live endpoint (see below), aistupidlevel/auto-coding and anthropic/claude-opus-4-7 both get 400 model_not_found, while auto-coding and claude-opus-4-7 route.
  • Prompt caching stays off. The router itself asks Claude to cache every conversation that continues (docs, "Prompt caching is on"), so DeepCode's cache-control markers are not needed.

Checklist

  • Changes tested locally
  • Code reviewed
  • Documentation updated (if necessary)
  • Unit tests added (if applicable)

Additional Notes

Code reviewed is unticked because no one outside the AI Stupid Level team has reviewed the change. Documentation is unticked because no docs change is needed: docs/guide/models.md uses OpenRouter as its example and gives a section only to Bedrock, whose setup differs. Requesty, Opper and Forge are not listed there either.

Commands run (Python 3.12.15, deps from scripts/ci/requirements.lock, Linux):

  • pytest tests/test_aistupidlevel_provider.py tests/test_provider_egress.py tests/test_opper_provider.py tests/test_forge_provider.py: 62 passed.
  • All 23 provider, catalog, config and connection test files (tests/*provider*, test_catalog, test_config_*, test_model_compat, test_model_catalog_service, test_llm_runtime_connections, test_reasoning_*, test_init_config): 264 passed. Clean main gives 257; the difference is the 6 new tests plus the new egress row.
  • I flipped strip_model_prefix and supports_prompt_caching in the spec, and the two decision tests failed. With the values restored, they pass.
  • Full suite (pytest tests --ignore=tests/e2e): 2750 passed, 37 skipped, 11 failed.
    • The same 9 tests fail on clean main (49805ef): test_ci_scope ×4, test_verification, test_unified_impl_workflow, test_p4_code_workbench ×2 and test_shell_search_tools. They depend on this machine: there is no python executable on PATH, and the system python3 has an older pytest than the lock.
    • The other 2 (test_service_tui dropped connection, test_mcp_runtime_lazy timeout) pass when re-run alone, on both main and this branch.
  • pre-commit run --files <the 4 files>: all hooks passed. Actionlint was skipped because no workflow changed.

Live check without spending anything: ASL has a sandbox mode. A request with X-ASL-Sandbox: true goes through real authentication and routing but gets a mock answer. No provider is called and nothing is billed. I used a temporary router key and a local proxy that adds the header:

  • deepcode provider set asl-sandbox --template aistupidlevel --api-base http://127.0.0.1:<port>/v1 --api-key-env AISTUPIDLEVEL_API_KEY --model auto-coding
  • deepcode provider models asl-sandbox --refresh: 39 models (15 routing modes and 24 pinnable models).
  • deepcode provider test asl-sandbox --model auto-coding: the credential, catalog and model stages passed.
  • deepcode provider test ... --agent: the tool stage passed. The continuation stage failed, as expected: the sandbox's mock answer cannot reproduce the nonce. I did not run Agent verification against a real model.
  • OpenAICompatProvider with the registry spec, run both through the proxy and against the default base URL with extra_headers={"X-ASL-Sandbox": "true"}. Each request returned 200 with X-AISM-Model set:
    • auto-coding was routed to gpt-5.6-luna.
    • aistupidlevel/auto-coding was sent as auto-coding.
    • claude-opus-4-7 with reasoning_effort="high" was accepted.
    • auto-tooling with one tool returned a parsed read_file tool call.
    • Streaming auto-coding gave 3 deltas, then stop.

The catalog lists the routing modes with DeepCode's defaults for unknown models (128,000 context, 8,192 output), which is conservative. For any routed model the limits are 262,144 context and 64,000 output. Users can declare those with --model-declarations if they want. The router is text only, so image, audio and file parts get 400 unsupported_content.

🤖 Generated with Claude Code

AI Stupid Level's Smart Router is an OpenAI-compatible endpoint at
https://aistupidlevel.info/v1. Its routing modes (auto-coding,
auto-tooling, ...) send each request to a model chosen from ASL's
benchmark results, and the router calls that model with the provider
keys the user saved on their own ASL account.

Mirrors the Opper provider added in HKUDS#242: a ProviderSpec on the generic
openai_compat gateway path, the matching ProviderConfig block, the
endpoint-class row in tests/test_provider_egress.py, and tests pinning
the wiring.

Two provider specific decisions, both covered by tests that build a
request through OpenAICompatProvider._build_kwargs. strip_model_prefix
is on, as for Forge: the router serves bare ids only and answers
aistupidlevel/auto-coding with model_not_found. Prompt caching stays
off: the router already asks Claude to cache every conversation that
continues, so DeepCode's cache-control markers are not needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant