Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,9 @@ Every method above is mirrored on `AsyncThreatZone`.
is not retried; the error is raised at once. `max_retries` (default `2`) bounds every retry,
also with `http_client=`; `max_retries=0` turns retries off.
- A download error keeps the error body, so the exception has `code` and the server message.
- README and recipes use `APIError.code` and the current error codes (`WORKSPACE_INACTIVE`,
`PLAN_API_ACCESS_DENIED`, `BEHAVIOUR_INDEX_BUILDING`) and describe the retries and the list
envelope.

- `retry_after` on `RateLimitError` and the `503` errors falls back to
`details.retryAfterSeconds` when the header and the `retryAfter` body field are absent.
Expand Down Expand Up @@ -428,6 +431,14 @@ Every method above is mirrored on `AsyncThreatZone`.
`download_syscalls()`, `get_mitre_techniques()`, `unpin_evidence()` and
`get_url_analysis_indicator_evidence()` name the errors and fields that the API now sends.

### Deprecated

- The singular filters `type`, `pid`, `operation` and `process_name` of `get_behaviours()`
warn with `DeprecationWarning`, as the API marks them deprecated. Use `types`, `pids`,
`operations` and `process_name_contains`. They keep working.
- `Indicator.score_maximum` (always `100`) and `NetworkEndpoint.max_alert_severity` (read
`max_alert_severity_level`).

### Removed

- `ApiErrorCode` no longer lists `INVALID_UUID`, `SUBMISSION_PRIVATE` and
Expand Down
51 changes: 41 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,30 @@ client = ThreatZone(
The `base_url` **must** include the `/public-api` suffix. For self-signed certs, leave
`verify_ssl=False` (the default).

### Retries

`max_retries` (default `2`) sets how many times the SDK sends a failed request again:

- A connection failure, a connect timeout and a pool timeout: every method. The server did not
get the request.
- A read or write timeout: `GET`, `PUT` and `DELETE` only. The SDK never sends a `POST` again
after such a timeout, because the server can have created the submission. Check
`list_submissions()` before you upload the file again.
- A `429` or `503` response: `GET`, `PUT` and `DELETE` only, also downloads. The SDK waits
for the `Retry-After` header (without the header: 1, 2, 4... seconds). When the wait is more
than 60 seconds, the SDK raises the error at once; read `retry_after` on it.

`max_retries=0` turns retries off.

### List envelope

`ThreatZone(list_envelope=True)` sends `envelope=1` to every list route. The routes then
return `{items, total, page, pageSize, next}`. The methods return the same types:
`total_count` of the flat lists comes from `total`, and the list models (for example
`PaginatedSubmissions`) add `page_size` and `next`. The flat array form of the API is
deprecated. Servers before the 2026-10-19 Public API reject `envelope` with `400`, so the
default is `False`.

## Quick usage

```python
Expand Down Expand Up @@ -361,8 +385,13 @@ with client.download_syscalls(uuid) as download:
```

The index parameters (`pids` to `sort`) and the summary, facet and match routes read the
behaviour index. The public API never builds the index; the index is built when the Behaviour view of
the report is opened. Without a current index these calls raise `BehaviourIndexingError`.
behaviour index. When no index exists, the server starts the index build and answers from a
direct read of the behaviour file. When that read cannot answer, these calls raise
`BehaviourIndexBuildingError` (`503 BEHAVIOUR_INDEX_BUILDING`, a subclass of
`BehaviourIndexingError`) after the SDK retries; `retry_after` gives the seconds to wait. Older
servers raise `BehaviourIndexingError` with `409 BEHAVIOUR_INDEXING`. The singular filters
`type`, `pid`, `operation` and `process_name` are deprecated and warn with
`DeprecationWarning`; use `types`, `pids`, `operations` and `process_name_contains`.
`get_behaviours()` without the index parameters (`event_ids` is allowed) reads the behaviour
file and does not need the index. `get_behaviours()` with an index parameter,
`get_behaviour_summary()`, `get_behaviour_facets()`, `get_behaviour_matches()`,
Expand Down Expand Up @@ -561,24 +590,26 @@ Use `asyncio.gather()` for concurrent fan-out. See [recipe 10](./docs/RECIPES.md
## Errors

Every error inherits from `ThreatZoneError`. HTTP errors additionally inherit from
`APIError` and carry the parsed error envelope on `.body`, with the structured code at
`.body["code"]`. Branch on `.code`, never on `.message`.
`APIError`. They carry the error code on `.code` and the parsed error envelope on `.body`.
Branch on `.code`, never on `.message`. Download errors carry the code too.

| Status | Exception | When |
|---|---|---|
| — | `ThreatZoneError` | Base class. Catch for a single safety net. |
| — | `APIError` | HTTP error base class. `.status_code`, `.body`, `.response`. |
| 400 | `BadRequestError` | `INVALID_QUERY_PARAM` (also for a malformed UUID or ID). |
| 401 | `AuthenticationError` | Missing/invalid API key. |
| 402 | `PaymentRequiredError` | Workspace subscription inactive. |
| 403 | `PermissionDeniedError` | `FORBIDDEN` (plan permission or module missing), `LICENSE_INVALID`. |
| 400 | `BadRequestError` | `INVALID_QUERY_PARAM` (also for a malformed UUID or ID). `.field` names the first field that failed, for example `limit`. |
| 401 | `AuthenticationError` | `UNAUTHORIZED`: missing or invalid API key. |
| 402 | `PaymentRequiredError` | `WORKSPACE_INACTIVE`: the workspace is not active. Renew the plan or contact the workspace owner. |
| 403 | `PermissionDeniedError` | `FORBIDDEN` (plan permission or module missing), `PLAN_API_ACCESS_DENIED` (the plan has no API access; a retry does not help), `LICENSE_INVALID`. |
| 404 | `NotFoundError` | `NOT_FOUND` for a missing submission and for a private submission of another workspace; `ARTIFACT_NOT_FOUND`, `MEDIA_NOT_FOUND` and other route codes. |
| 409 | `ReportUnavailableError` | Required report not yet available. Carries `.code`, `.submission_uuid`, `.required_report`, `.current_status`, `.available_reports`. |
| 409 | `BehaviourIndexingError` | `BEHAVIOUR_INDEXING`: the behaviour index is not ready. Subclass of `ReportUnavailableError`. Has `.retry_after`. |
| 409 | `BehaviourIndexingError` | `BEHAVIOUR_INDEXING` from an older server: the behaviour index is not ready. Subclass of `ReportUnavailableError`. Has `.retry_after`. |
| 413 | `BehaviourTooLargeError` | `BEHAVIOUR_TOO_LARGE`: the behaviour file or the index scan is above the read limit. `.size_bytes`, `.max_bytes`, `.download_route`. |
| 429 | `RateLimitError` | `.code`: `RATE_LIMIT_EXCEEDED` (heavy-route limit, monthly API request limit), `SUBMISSION_LIMIT_EXCEEDED` (daily or concurrent submission limit), `AI_REPORT_QUOTA_EXCEEDED`. Has `.retry_after`. |
| 502 | `BehaviourUnreadableError` | `BEHAVIOUR_UNREADABLE`: the stored behaviour file is not valid. Subclass of `InternalServerError`. |
| 503 | `QuotaUnavailableError` | `QUOTA_UNAVAILABLE`: the server cannot check the submission quota. No submission was created. Subclass of `InternalServerError`. Has `.retry_after`. |
| 503 | `ServiceUnavailableError` | Any other `503`, for example `STORAGE_UNAVAILABLE`. Subclass of `InternalServerError`. Has `.retry_after`. |
| 503 | `BehaviourIndexBuildingError` | `BEHAVIOUR_INDEX_BUILDING`: the server builds the behaviour index. Subclass of `BehaviourIndexingError` and `ServiceUnavailableError`. Has `.retry_after`. |
| 503 | `QuotaUnavailableError` | `QUOTA_UNAVAILABLE`: the server cannot check the submission quota. No submission was created. Subclass of `ServiceUnavailableError`. Has `.retry_after`. |
| 5xx | `InternalServerError` | Server-side failure. |
| — | `TimeoutError` | HTTP request timed out. |
| — | `ConnectionError` | Network-level failure. |
Expand Down
41 changes: 27 additions & 14 deletions docs/RECIPES.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,9 +465,11 @@ from threatzone import (
AuthenticationError,
BadRequestError,
NotFoundError,
PaymentRequiredError,
PermissionDeniedError,
RateLimitError,
ReportUnavailableError,
ServiceUnavailableError,
ThreatZone,
)

Expand All @@ -476,19 +478,22 @@ def fetch_indicators_safely(client: ThreatZone, submission_uuid: str) -> None:
response = client.get_indicators(submission_uuid)
except AuthenticationError:
print("Invalid API key.")
except PaymentRequiredError:
# WORKSPACE_INACTIVE
print("The workspace is not active. Renew the plan or contact the workspace owner.")
except PermissionDeniedError as exc:
code = (exc.body or {}).get("code")
if code == "LICENSE_INVALID":
if exc.code == "PLAN_API_ACCESS_DENIED":
print("The plan has no API access. Upgrade the plan; a retry does not help.")
elif exc.code == "LICENSE_INVALID":
print("The self-hosted deployment has no valid license.")
else:
print(f"Access denied (plan permission or module): {exc.message}")
except NotFoundError as exc:
# A private submission of another workspace gives the same NOT_FOUND as a missing one.
print(f"Not found ({(exc.body or {}).get('code')}).")
print(f"Not found ({exc.code}).")
except BadRequestError as exc:
code = (exc.body or {}).get("code")
if code == "INVALID_QUERY_PARAM":
print(f"A path or query parameter is not valid: {exc.message}")
if exc.code == "INVALID_QUERY_PARAM":
print(f"Parameter {exc.field or '?'} is not valid: {exc.message}")
else:
print(f"Bad request: {exc.message}")
except ReportUnavailableError as exc:
Expand All @@ -497,6 +502,8 @@ def fetch_indicators_safely(client: ThreatZone, submission_uuid: str) -> None:
except RateLimitError as exc:
wait = exc.retry_after or 60
print(f"Rate limited ({exc.code}); retry after {wait}s.")
except ServiceUnavailableError as exc:
print(f"Service unavailable ({exc.code}); retry after {exc.retry_after}s.")
except APIError as exc:
print(f"Unhandled API error {exc.status_code}: {exc.message}")
else:
Expand All @@ -508,19 +515,25 @@ with ThreatZone() as client:

**Notes.**

- `APIError.body["code"]` is the machine-readable discriminator. It's one of the values
- `APIError.code` is the machine-readable discriminator. It's one of the values
in the `ApiErrorCode` literal union in `threatzone.types.errors`, or a code that a newer
server adds. Every code is stable across releases; the `message` is not.
- `ReportUnavailableError` already exposes `.code` as a typed attribute — no need
to reach into `.body` for it.
server adds. Every code is stable across releases; the `message` is not. `code` is `None`
only when the response has no JSON error body, for example a proxy error page.
- `PaymentRequiredError` (402) has the code `WORKSPACE_INACTIVE`. `PermissionDeniedError`
(403) has `FORBIDDEN`, `PLAN_API_ACCESS_DENIED` or `LICENSE_INVALID`.
- `BadRequestError.field` names the first field that failed validation (`details.field`).
- `RateLimitError.code` is `RATE_LIMIT_EXCEEDED` (heavy-route limit, monthly API request
limit), `SUBMISSION_LIMIT_EXCEEDED` (daily or concurrent submission limit) or
`AI_REPORT_QUOTA_EXCEEDED`. `RateLimitError.retry_after` is parsed from the `Retry-After`
response header, or from the `retryAfter` body field when the header is absent or is not a
number. It is `None` when both are absent.
- The create routes raise `QuotaUnavailableError` (503, `QUOTA_UNAVAILABLE`, a subclass of
`InternalServerError`) when the server cannot check the submission quota. Retry after
response header, then from the `retryAfter` body field, then from
`details.retryAfterSeconds`. It is `None` when all are absent.
- Every `503` raises `ServiceUnavailableError` (a subclass of `InternalServerError`) with
`retry_after`. The create routes raise its subclass `QuotaUnavailableError`
(`QUOTA_UNAVAILABLE`) when the server cannot check the submission quota. Retry after
`retry_after` seconds.
- The SDK already retries `GET`, `PUT` and `DELETE` requests on `429` and `503` when the wait
is at most 60 seconds, so these errors reach your code only after the retries or for a
longer wait. It never retries a `POST` after a timeout or a `429` or `503`.
- The API never sends `SUBMISSION_PRIVATE` or `INVALID_UUID`. Code that checked them must
check `NotFoundError` with `NOT_FOUND`, and `BadRequestError` with `INVALID_QUERY_PARAM`.

Expand Down
9 changes: 5 additions & 4 deletions examples/recipe_09_discriminate_errors_by_code.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
AuthenticationError,
BadRequestError,
NotFoundError,
PaymentRequiredError,
PermissionDeniedError,
RateLimitError,
ReportUnavailableError,
Expand All @@ -34,15 +35,15 @@ def main(client: ThreatZone, submission_uuid: str) -> str:
response = client.get_indicators(submission_uuid)
except AuthenticationError:
return "AUTHENTICATION_ERROR"
except PaymentRequiredError as exc:
return f"WORKSPACE_INACTIVE:{exc.code or 'unknown'}"
except PermissionDeniedError as exc:
code = (exc.body or {}).get("code") if exc.body else None
return f"PERMISSION_DENIED:{code or 'unknown'}"
return f"PERMISSION_DENIED:{exc.code or 'unknown'}"
except NotFoundError:
# A private submission of another workspace also arrives here, not at PermissionDeniedError.
return "NOT_FOUND"
except BadRequestError as exc:
code = (exc.body or {}).get("code") if exc.body else None
return f"BAD_REQUEST:{code or 'unknown'}"
return f"BAD_REQUEST:{exc.code or 'unknown'}"
except ReportUnavailableError as exc:
return f"REPORT_UNAVAILABLE:{exc.code or 'unknown'}"
except RateLimitError as exc:
Expand Down
21 changes: 16 additions & 5 deletions src/threatzone/_async_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@
YaraRulePendingError,
)
from ._lists import list_page
from ._params import behaviour_filter_params, behaviour_match_params, syscall_search_params
from ._params import (
behaviour_filter_params,
behaviour_match_params,
syscall_search_params,
warn_singular_behaviour_filters,
)
from ._streaming import AsyncDownloadResponse
from .types import (
AiReport,
Expand Down Expand Up @@ -1150,10 +1155,13 @@ async def get_behaviours(

Args:
uuid: Submission UUID.
type: Filter by event type (e.g. registry, file, network, process, mutex).
pid: Filter by process ID.
operation: Filter by operation name.
process_name: Filter by process name (exact match).
type: Deprecated: use ``types``. Filter by event type (e.g. registry, file,
network, process, mutex).
pid: Deprecated: use ``pids``. Filter by process ID.
operation: Deprecated: use ``operations``. Filter by operation name.
process_name: Deprecated: use ``process_name_contains``. Filter by process
name (exact match). The four singular filters warn with
``DeprecationWarning`` and keep working.
page: 1-indexed page number.
limit: Maximum items per page (1-500).
event_ids: Return only the events with these IDs (at most 500), for example
Expand Down Expand Up @@ -1196,6 +1204,9 @@ async def get_behaviours(
above the heavy-route limit. ``retry_after`` gives the seconds to wait.
ValueError: When a list filter is empty. The SDK does not send the request.
"""
warn_singular_behaviour_filters(
type=type, pid=pid, operation=operation, process_name=process_name
)
params: dict[str, Any] = {}
if type is not None:
params["type"] = type
Expand Down
19 changes: 19 additions & 0 deletions src/threatzone/_params.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,28 @@

from __future__ import annotations

import warnings
from collections.abc import Mapping, Sequence
from typing import Any

_SINGULAR_BEHAVIOUR_FILTERS = (
("type", "types"),
("pid", "pids"),
("operation", "operations"),
("process_name", "process_name_contains"),
)


def warn_singular_behaviour_filters(**filters: object) -> None:
"""Warn once for each deprecated singular filter of ``get_behaviours()`` that is set."""
for name, replacement in _SINGULAR_BEHAVIOUR_FILTERS:
if filters.get(name) is not None:
warnings.warn(
f"get_behaviours({name}=...) is deprecated. Use {replacement}=... instead.",
DeprecationWarning,
stacklevel=3,
)


def _join(name: str, values: Sequence[int | str] | None) -> str | None:
if values is None:
Expand Down
21 changes: 16 additions & 5 deletions src/threatzone/_sync_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,12 @@
YaraRulePendingError,
)
from ._lists import list_page
from ._params import behaviour_filter_params, behaviour_match_params, syscall_search_params
from ._params import (
behaviour_filter_params,
behaviour_match_params,
syscall_search_params,
warn_singular_behaviour_filters,
)
from ._streaming import DownloadResponse
from .types import (
AiReport,
Expand Down Expand Up @@ -1147,10 +1152,13 @@ def get_behaviours(

Args:
uuid: Submission UUID.
type: Filter by event type (e.g. registry, file, network, process, mutex).
pid: Filter by process ID.
operation: Filter by operation name.
process_name: Filter by process name (exact match).
type: Deprecated: use ``types``. Filter by event type (e.g. registry, file,
network, process, mutex).
pid: Deprecated: use ``pids``. Filter by process ID.
operation: Deprecated: use ``operations``. Filter by operation name.
process_name: Deprecated: use ``process_name_contains``. Filter by process
name (exact match). The four singular filters warn with
``DeprecationWarning`` and keep working.
page: 1-indexed page number.
limit: Maximum items per page (1-500).
event_ids: Return only the events with these IDs (at most 500), for example
Expand Down Expand Up @@ -1193,6 +1201,9 @@ def get_behaviours(
above the heavy-route limit. ``retry_after`` gives the seconds to wait.
ValueError: When a list filter is empty. The SDK does not send the request.
"""
warn_singular_behaviour_filters(
type=type, pid=pid, operation=operation, process_name=process_name
)
params: dict[str, Any] = {}
if type is not None:
params["type"] = type
Expand Down
1 change: 1 addition & 0 deletions tests/integration/flows/test_flow_dynamic_lifecycle.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ def test_processes_and_tree(fake_api: FakeThreatZoneAPI, sync_client: ThreatZone
assert isinstance(tree, ProcessTreeResponse)


@pytest.mark.filterwarnings("ignore:get_behaviours:DeprecationWarning")
def test_behaviours_without_os_parameter(
fake_api: FakeThreatZoneAPI, sync_client: ThreatZone
) -> None:
Expand Down
28 changes: 28 additions & 0 deletions tests/integration/test_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@

from pathlib import Path

import httpx
import pytest

from examples import (
recipe_01_submit_and_wait,
recipe_02_url_analysis,
Expand Down Expand Up @@ -179,3 +182,28 @@ def test_recipe_12_mitre_cross_reference(
assert "indicator_attack_codes" in result
assert "intersection" in result
assert "T1055" in result["intersection"]


@pytest.mark.parametrize(
("status", "code", "expected"),
[
(402, "WORKSPACE_INACTIVE", "WORKSPACE_INACTIVE:WORKSPACE_INACTIVE"),
(403, "PLAN_API_ACCESS_DENIED", "PERMISSION_DENIED:PLAN_API_ACCESS_DENIED"),
(400, "INVALID_QUERY_PARAM", "BAD_REQUEST:INVALID_QUERY_PARAM"),
],
)
def test_recipe_09_reads_the_current_codes(status: int, code: str, expected: str) -> None:
def handler(_: httpx.Request) -> httpx.Response:
return httpx.Response(
status, json={"statusCode": status, "message": "Refused", "code": code}
)

client = ThreatZone(
api_key="test-key",
base_url="https://fake.threat.zone/public-api",
http_client=httpx.Client(transport=httpx.MockTransport(handler)),
)
assert (
recipe_09_discriminate_errors_by_code.main(client, "00000000-0000-4000-8000-000000000001")
== expected
)
Loading
Loading