Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- `Indicator.summary` (`str | None`) carries the one-sentence analyst summary of Windows V2 schema 2.1 rows. It is `None` on legacy rows and when an older server omits it. On rows with a summary, `Indicator.description` holds the long analyst text.
- `Indicator.score_maximum` (`10` legacy, `100` Windows V2) gives the scale of `Indicator.score`.
- `NetworkSummary` adds `counts_available`, `availability` (new `NetworkAvailability` model
with `state`, `reason`, `coverage`) and `source_scope`. `DnsQuery`, `HttpRequest`,
`Connection` and `NetworkThreat` add `source_scope`. Legacy reports send `counts_available=True`, `availability=None` and `source_scope=None`. Every new field is `None` when an older server omits it. Enum-like fields accept unknown values, so a new server value does not fail the call.
`SummaryNetwork` (`OverviewSummary.network`) adds the same three fields.
The `X-Network-Evidence-Availability` header of the list endpoints is not exposed, because
the list methods return plain lists. Use `get_network_summary().availability`.
- `create_url_submission()` accepts `metafields` and `configurations`. Use
`metafields={"timeout": ..., "device_preset": ...}` to set the interactive session
duration and the emulated device. Use
Expand Down
22 changes: 15 additions & 7 deletions docs/TYPES.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ in this group is envelope-wrapped (`items`, `total`) for server-side pagination.

| Type | Purpose | Endpoint |
|------|---------|----------|
| `Indicator` | A single rule hit with cross-reference IDs. Fields: `id`, `name`, `description`, `category`, `level`, `score`, `pids`, `attack_codes`, `event_ids`, `syscall_line_numbers`, `author`. | `GET /submissions/:uuid/indicators` |
| `Indicator` | A single rule hit with cross-reference IDs. Fields: `id`, `name`, `description`, `summary`, `category`, `level`, `score`, `score_maximum`, `pids`, `attack_codes`, `event_ids`, `syscall_line_numbers`, `author`. | `GET /submissions/:uuid/indicators` |
| `IndicatorsResponse` | `{items, total, levels}` envelope. `levels` is a `{malicious, suspicious, benign}` severity rollup. | same |
| `IndicatorLevels` | Severity counts: `malicious`, `suspicious`, `benign`. | nested |
| `IoC` | `{type, value, artifacts}`. `type` is the 11-value literal below. | `GET /submissions/:uuid/iocs` |
Expand All @@ -74,12 +74,16 @@ in this group is envelope-wrapped (`items`, `total`) for server-side pagination.

- `Indicator.level` is `IndicatorLevel = Literal["malicious", "suspicious", "benign"]`.
- `Indicator.author` is `Literal["system", "user"]`.
- `Indicator.score_maximum` is `int | None`: the scale of `score` (10 legacy, 100 Windows V2). `None` when the API does not send it. Divide `score` by `score_maximum` before comparing rows from different scales.
- `Indicator.summary` is `str | None`: a one-sentence plain-language summary, sent for Windows V2 schema 2.1 rows only. `None` on legacy rows and when the API does not send it. On rows with a summary, `description` holds the long analyst text.
- `IoC.type` is `IoCType = Literal["ip", "domain", "url", "email", "sha512", "sha256", "sha1", "md5", "registry", "path", "uuid"]`.
- `YaraRule.category` is `YaraRuleCategory = Literal["malicious", "suspicious", "benign"]`.
- `Artifact.type` is `ArtifactType` (14-value literal including `sample`, `dropped_file`,
`memory_dump`, `pcap`, `yara_rule`, …).
- `OverviewSummary.network` is `Optional` — `None` on submissions without a
completed dynamic report.
completed dynamic report. `OverviewSummary.network` also carries `counts_available`,
`availability` and `source_scope` (same types as `NetworkSummary`). When `counts_available`
is `False` the counts are 0 and do not mean the sample had no network activity.

Cross-references: `Indicator.pids` points at `Process.pid` (see Dynamic Scan),
`Indicator.attack_codes` at `MitreResponse.techniques`, `Indicator.syscall_line_numbers`
Expand Down Expand Up @@ -107,18 +111,22 @@ report. Every endpoint is guarded by `ReportStatusGuard('dynamic')`; read the co
| `BehavioursResponse` | `{items, total, page, limit, total_pages}`. | same |
| `BehaviourOs` | Deprecated compatibility alias; the endpoint derives the OS from the submission. | same |
| `SyscallsResponse` | `{items: List[str], total}`. `items` are raw unparsed log lines. | `GET /submissions/:uuid/syscalls` |
| `NetworkSummary` | `{dns_count, http_count, tcp_count, udp_count, threat_count, pcap_available}`. | `GET /submissions/:uuid/network/summary` |
| `DnsQuery` | `{id, host, type, status, records, timeshift}`. | `GET /submissions/:uuid/network/dns` |
| `HttpRequest` | `{id, host, ip, port, country}` — slimmed-down shape, no method/URL/headers. | `GET /submissions/:uuid/network/http` |
| `Connection` | `{id, protocol, destination_ip, destination_port, domain, asn, country, packets, timeshift}`. Used for both TCP and UDP. | `GET /submissions/:uuid/network/tcp`, `GET /submissions/:uuid/network/udp` |
| `NetworkSummary` | `{dns_count, http_count, tcp_count, udp_count, threat_count, pcap_available, counts_available, availability, source_scope}`. Legacy reports send `counts_available=True`, `availability=None`, `source_scope=None`; all three are `None` when an older server omits them. | `GET /submissions/:uuid/network/summary` |
| `NetworkAvailability` | `{state, reason, coverage}`. Network evidence state of a Windows V2 report. | nested in `NetworkSummary` |
| `DnsQuery` | `{id, host, type, status, records, timeshift, source_scope}`. | `GET /submissions/:uuid/network/dns` |
| `HttpRequest` | `{id, host, ip, port, country, source_scope}` — slimmed-down shape, no method/URL/headers. | `GET /submissions/:uuid/network/http` |
| `Connection` | `{id, protocol, destination_ip, destination_port, domain, asn, country, packets, timeshift, source_scope}`. Used for both TCP and UDP. | `GET /submissions/:uuid/network/tcp`, `GET /submissions/:uuid/network/udp` |
| `ConnectionPackets` | `{sent, received, empty}`. | nested |
| `NetworkThreat` | Suricata alert: `signature`, `description`, `severity`, `protocol`, `app_proto`, `destination_ip`, `destination_port`, `timeshift`, `metadata`, `details`, optional `tls`. | `GET /submissions/:uuid/network/threats` |
| `NetworkThreat` | Suricata alert: `signature`, `description`, `severity`, `protocol`, `app_proto`, `destination_ip`, `destination_port`, `timeshift`, `metadata`, `details`, optional `tls`, `source_scope`. | `GET /submissions/:uuid/network/threats` |
| `NetworkThreatTls` | TLS fingerprint block: `version`, `sni`, `ja3_hash`, `ja3s_hash`, `ja3_string`, `ja3s_string`. Present iff `app_proto == "TLS"`. | nested in `NetworkThreat` |

**Discriminative fields.**

- `DnsQuery.type` is `DnsRecordType` (16-value literal: `A`, `AAAA`, `CNAME`, `MX`, …).
- `DnsQuery.status` is `DnsStatus` (DNS RCODE literal, e.g. `NOERROR`, `NXDOMAIN`, `TIMEOUT`).
- `NetworkAvailability.state` is `NetworkAvailabilityState` (`disabled`, `unavailable`, `verified_zero`, `observed`); `coverage` is `NetworkCoverage` (`none`, `partial`, `complete`). When `NetworkSummary.counts_available` is `False`, all counts are 0 and do not mean the sample had no network activity.
- `source_scope` is `NetworkSourceScope` (`sample_attributed`, `submission_ip_scoped`, `submission_unattributed`, `none`) or `None` on legacy reports. Unknown values from a newer server are kept as plain strings.
- The list endpoints send the `X-Network-Evidence-Availability` response header on Windows V2. The `get_*` list methods return plain lists, so the SDK does not surface it. Read `get_network_summary().availability` instead.
- `Connection.protocol` is `ConnectionProtocol = Literal["tcp", "udp"]`.
- `NetworkThreat.protocol` is `ThreatProtocol = Literal["TCP", "UDP"]` (uppercase!).
- `NetworkThreat.app_proto` is `ThreatAppProto = Literal["HTTP", "TLS", "DNS"]`.
Expand Down
2 changes: 2 additions & 0 deletions src/threatzone/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
Metafields,
MitreResponse,
ModuleInfo,
NetworkAvailability,
NetworkConfigListItem,
NetworkConfigListResponse,
NetworkSummary,
Expand Down Expand Up @@ -155,6 +156,7 @@
"SubmissionLimits",
"ModuleInfo",
# Types - Network
"NetworkAvailability",
"NetworkSummary",
"DnsQuery",
"HttpRequest",
Expand Down
8 changes: 8 additions & 0 deletions src/threatzone/types/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@
DnsRecordType,
DnsStatus,
HttpRequest,
NetworkAvailability,
NetworkAvailabilityState,
NetworkCoverage,
NetworkSourceScope,
NetworkSummary,
NetworkThreat,
NetworkThreatTls,
Expand Down Expand Up @@ -203,6 +207,10 @@
"DnsRecordType",
"DnsStatus",
"HttpRequest",
"NetworkAvailability",
"NetworkAvailabilityState",
"NetworkCoverage",
"NetworkSourceScope",
"NetworkSummary",
"NetworkThreat",
"NetworkThreatTls",
Expand Down
7 changes: 7 additions & 0 deletions src/threatzone/types/indicators.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@

from pydantic import BaseModel, ConfigDict, Field

from .network import NetworkAvailability, NetworkSourceScope

IndicatorLevel = Literal["malicious", "suspicious", "benign"]
IndicatorAuthor = Literal["system", "user"]

Expand All @@ -33,9 +35,11 @@ class Indicator(BaseModel):
id: str
name: str
description: str
summary: str | None = None
category: list[str]
level: IndicatorLevel
score: int
score_maximum: int | None = Field(default=None, alias="scoreMaximum")
pids: list[int]
attack_codes: list[str] = Field(alias="attackCodes")
event_ids: list[int] = Field(alias="eventIds")
Expand Down Expand Up @@ -219,6 +223,9 @@ class SummaryNetwork(BaseModel):
tcp_count: int = Field(alias="tcpCount")
udp_count: int = Field(alias="udpCount")
threat_count: int = Field(alias="threatCount")
counts_available: bool | None = Field(default=None, alias="countsAvailable")
availability: NetworkAvailability | None = None
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")


class OverviewSummary(BaseModel):
Expand Down
29 changes: 29 additions & 0 deletions src/threatzone/types/network.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,28 @@
ThreatAppProto = Literal["HTTP", "TLS", "DNS"]
ThreatSeverity = Literal["high", "low"]

NetworkAvailabilityState = Literal["disabled", "unavailable", "verified_zero", "observed"] | str
NetworkCoverage = Literal["none", "partial", "complete"] | str
NetworkSourceScope = (
Literal[
"sample_attributed",
"submission_ip_scoped",
"submission_unattributed",
"none",
]
| str
)


class NetworkAvailability(BaseModel):
"""Network evidence state of a Windows V2 report."""

model_config = ConfigDict(populate_by_name=True)

state: NetworkAvailabilityState
reason: str | None = None
coverage: NetworkCoverage


class NetworkSummary(BaseModel):
"""Summary of network activity captured during dynamic analysis."""
Expand All @@ -67,6 +89,9 @@ class NetworkSummary(BaseModel):
udp_count: int = Field(alias="udpCount")
threat_count: int = Field(alias="threatCount")
pcap_available: bool = Field(alias="pcapAvailable")
counts_available: bool | None = Field(default=None, alias="countsAvailable")
availability: NetworkAvailability | None = None
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")


class DnsQuery(BaseModel):
Expand All @@ -80,6 +105,7 @@ class DnsQuery(BaseModel):
status: DnsStatus
records: list[str]
timeshift: float
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")


class HttpRequest(BaseModel):
Expand All @@ -92,6 +118,7 @@ class HttpRequest(BaseModel):
ip: str
port: int
country: str | None = None
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")


class ConnectionPackets(BaseModel):
Expand All @@ -118,6 +145,7 @@ class Connection(BaseModel):
country: str | None = None
packets: ConnectionPackets
timeshift: float
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")


class NetworkThreatTls(BaseModel):
Expand Down Expand Up @@ -149,3 +177,4 @@ class NetworkThreat(BaseModel):
metadata: dict[str, list[str]]
details: dict[str, str | int]
tls: NetworkThreatTls | None = None
source_scope: NetworkSourceScope | None = Field(default=None, alias="sourceScope")
Loading
Loading