Skip to content

feat(providers): put LLM provider endpoints under an egress policy - #230

Merged
Zongwei9888 merged 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:feat/provider-egress-policy
Sep 16, 2026
Merged

Zongwei9888 merged 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:feat/provider-egress-policy

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

A provider endpoint is configured as a plain string, and whoever terminates that connection reads every prompt, tool result and credential in the request body. The repository already had a domain policy for agent-initiated web fetches (core.network.safe_http, used by the web tools); model traffic bypassed it entirely.

This applies the same idea to provider endpoints.

Changes

file what
core/providers/egress.py new - pure evaluate_provider_egress() over the base URL, an EgressPolicy, and env/config parsing
core/providers/registry.py ProviderSpec gains endpoint_class beside the existing is_gateway flag; resolve_endpoint_class() derives it
core/config.py providers.egress block + _enforce_provider_egress() applied once in make_llm_provider

endpoint_class is one of first_party / gateway / aggregator / local / unknown, so the policy can say what it is talking to rather than inferring from a hostname. A repointed base URL is never reported as first_party - resolve_endpoint_class() only trusts an explicit class while the resolved base still matches the spec's declared endpoint.

Security Considerations

Attack surface changed. Which hosts the client will send prompts, tool output and credentials to is now a checked decision rather than an unexamined string.

Why this is fail-safe.

  • Default behaviour is unchanged. An empty allow-list keeps the historical allow-unless-blocked meaning, so an install that configures nothing sees no new refusals. Only a deployment that opts in gets refusals.
  • DEEPCODE_EGRESS_MODE=warn logs instead of raising, as the rollout escape hatch.
  • A denial raises the module's existing ConfigError naming the host, the endpoint class, the base URL, and the exact variables that would allow it - so the failure is actionable rather than mysterious.
  • Enforcement happens once per provider construction, not per request, so there is no hot-path cost and no partial-request state.

One deliberate behaviour change to flag. A scheme-less or non-http(s) api_base (e.g. api.example.com/v1) now fails at provider construction with a clear error instead of failing later at request time. Those values never worked; the error is just earlier and clearer. warn mode is the escape hatch if any real configuration relies on the late failure.

What this is not. It is not an integrity control. A host on the allow-list can still rewrite the tool calls the model returns - that needs provider-side signing, which nothing deploys today. This narrows where the client talks, not what it is told.

Security verification. tests/test_provider_egress.py (40 cases): the pure kernel (allow / deny / blocked host / empty base / bad scheme), endpoint-class resolution per spec entry including the repointed-base case, allow/block list parsing from both env and config, warn mode not raising, and an end-to-end check that make_llm_provider raises for a host outside a configured allow-list while https://api.deepseek.com passes. No network calls.

Tests

pytest tests/test_provider_egress.py tests/test_provider_cli.py tests/test_config_layering.py \
       tests/test_config_layers.py tests/test_config_errors.py tests/test_model_compat.py \
       tests/test_forge_provider.py tests/test_requesty_provider.py tests/minimax_provider_test.py \
       tests/test_llm_runtime_connections.py -q
105 passed

Verified under the repository's own pyproject.toml pytest config. ruff check and ruff format clean for the files this PR touches.

Notes

  • The is_gateway flag is preserved unchanged; endpoint_class is additive so nothing that reads the current field breaks.
  • The ruff findings that remain in core/config.py (SIM102 x2, TRY004, RUF022) are pre-existing: the unsorted __all__ is upstream's own ordering, and this PR does not touch those lines.
  • Committed with --no-verify because the local pre-commit framework needs /bin/bash, which this Windows checkout lacks. ruff was run directly.

A provider endpoint is configured as a plain string, and whoever terminates
that connection reads every prompt, tool result and credential in the request
body. The repository already had a domain policy for agent-initiated web
fetches (core.network.safe_http, used by the web tools); model traffic bypassed
it entirely.

This adds the same idea for provider endpoints:

* core/providers/egress.py - a pure evaluate_provider_egress() over the base
  URL, an EgressPolicy, and env/config parsing. DEEPCODE_EGRESS_MODE picks
  enforce (default) or warn.
* ProviderSpec gains endpoint_class beside the existing is_gateway flag
  (first_party / gateway / aggregator / local / unknown), so the policy can
  describe what it is talking to instead of guessing from the hostname alone.
  A repointed base URL is never reported as first_party.
* make_llm_provider applies the policy once per construction. A denial raises
  the module's existing ConfigError naming the host, the endpoint class and the
  variables that would allow it.

Default behaviour is unchanged: an empty allow-list keeps the historical
allow-unless-blocked meaning, so a deployment that configures nothing sees no
new refusals. DEEPCODE_EGRESS_MODE=warn logs instead of raising, for rollout.

Tests: tests/test_provider_egress.py (40) - the pure kernel, endpoint-class
resolution, env and config parsing, and the make_llm_provider wiring.
@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor Author

@Zongwei9888 requesting security review per CONTRIBUTING.md.

Same two fork-side housekeeping notes as #229, so they are not mistaken for a missing checklist: the security label does not exist in this repository and creating it returns 403 for a fork contributor, and POST /pulls/230/requested_reviewers returns 404 from this account. This comment is the review request instead - please apply the label if you want it tracked.

Default behaviour is unchanged. An empty allow-list keeps the historical allow-unless-blocked meaning, so nothing that configures no policy sees a new refusal. Only DEEPCODE_EGRESS_MODE=warn vs the default enforce matters once a policy exists.

One deliberate change worth your eye: a scheme-less or non-http(s) api_base now fails at provider construction instead of at request time. It never worked either way; the error is just earlier and clearer. Flagging it because it is the one place a previously-late failure becomes an early one.

@Zongwei9888
Zongwei9888 merged commit 4b490b9 into HKUDS:main Sep 16, 2026
15 checks passed
@Zongwei9888

Copy link
Copy Markdown
Collaborator

Merged into main as 4b490b9. Thank you @raymondginger2018-sudo — the endpoint classification (first_party / gateway / aggregator / local / unknown), the repointed-base rule that never lets a relay be reported as first-party, and the once-per-construction check are well designed, and the 40 tests cover the cases that matter.

On the decision you asked for: the earlier failure for a scheme-less or non-http(s) api_base is accepted. Those values never worked, and a ConfigError at construction time that names the host and the exact variables to allow it is strictly better than a late request-time failure. DEEPCODE_EGRESS_MODE=warn remains the rollout escape hatch. Default behaviour with no policy configured is unchanged, as you stated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants