Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/cpython-frame-walking.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Reading CPython's frame chain from outside the process

Notes gathered while building the BPF-side Python unwinder (issue #83).
Notes gathered while building the BPF-side Python unwinder.
Everything here was checked against CPython source and, where a number is
quoted, measured against real interpreter binaries on this host. It is
written down because most of it is not in any CPython document, and three
Expand All @@ -11,7 +11,7 @@ Supported range is CPython 3.12, 3.13 and 3.14, on amd64 and arm64, glibc
only. Each architecture carries its own measured glibc pthread-TSD offsets
and its own machine-code shapes for the `autoTSSkey` reference; an
architecture with neither is refused by name rather than walked with
guessed values ([#206](https://github.com/dpsoft/perf-agent/issues/206)).
guessed values.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/debuginfod-symbolization.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ fetch. M2 will add an explicit cache-invalidation flag.

- **Metrics exporter integration.** Counters exist internally (see
`symbolize/debuginfod/stats.go`) and are accessible via the `Stats()`
method and the agent prints them at shutdown (#109), but the snapshot is
method and the agent prints them at shutdown, but the snapshot is
still not wired through the `metrics.Exporter`
interface. Prometheus/OpenTelemetry export of cache hit rate, fetch
latency, and error counts is deferred to M2.
Expand Down
13 changes: 3 additions & 10 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,12 @@ own README, source workload, and driver script.
| [`rust-pgo/`](rust-pgo/) | Rust AutoFDO PGO. Build → profile → `create_llvm_prof` → rebuild → strip → measure speedup. |
| [`cpp-pgo/`](cpp-pgo/) | C++ AutoFDO PGO. Same shape via clang's `-fprofile-sample-use`. |
| [`flamegraph/`](flamegraph/) | Render a Brendan-Gregg flame graph from a perf-agent capture. |
| [`kubernetes/`](kubernetes/) | Profile an unmodified PyTorch CUDA pod in k8s via CUPTI injection, as a sidecar. **Not runnable yet — see #121.** |
| [`kubernetes/`](kubernetes/) | Profile an unmodified PyTorch CUDA pod in k8s via CUPTI injection, as a sidecar or a node collector. Needs a cluster with an NVIDIA GPU node. |

The first three depend on `perf-agent` being built and on PATH with the standard
capability set (`setcap cap_bpf,cap_perfmon,cap_sys_ptrace,cap_checkpoint_restore,cap_syslog+ep`).

## All of these run today

`kubernetes/` used to be the exception, because the shim could not load into
the images it would be injected into (#121) and nothing was published to name
in a pod spec (#139). Both are closed, and v1.3.0 publishes
`ghcr.io/dpsoft/perf-agent` and `ghcr.io/dpsoft/perf-agent-shim` as multi-arch
images, so the manifests there pull. What it still needs from you is a
cluster with an NVIDIA GPU node.
`kubernetes/` needs a cluster with an NVIDIA GPU node instead; the images its
manifests name are published, so there is nothing to build for it.

## Why these are here

Expand Down
21 changes: 14 additions & 7 deletions examples/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,16 @@ flame graph.
> **Both images are multi-arch** (amd64 and arm64), built natively on each
> architecture rather than under emulation.

## Two deployments: sidecar and collector
## Prior art

The injection pattern here is not ours. [Polar Signals' `parcagpu`][ps]
established it: an init container drops a CUPTI-based `.so` onto a shared
volume, and the application container loads it through
`CUDA_INJECTION64_PATH`. We took that directly, and the rest of this document
is what we do inside it rather than a claim to have invented it. Where the
two differ is privilege, compared below.

[Polar Signals' `parcagpu`][ps] established the injection pattern both of ours
follow: an init container drops a CUPTI-based `.so` onto a shared volume, and
the application container loads it via `CUDA_INJECTION64_PATH`. That part we
take directly.
## Two deployments: sidecar and collector

There are two manifests here, and the difference is **where the agent runs**,
which decides **how many shim inodes exist**:
Expand All @@ -48,7 +52,10 @@ we noticed it. One inode per node removes all three — measured on an RTX 3090,
with rediscovery disabled, was sampled anyway; a byte-identical copy at another
path was not.

Against parcagpu, the difference that survives is privilege, not topology:
### What differs from parcagpu

Topology aside, one difference is worth stating because it is the reason to
prefer this shape:

| | parcagpu | perf-agent collector |
|---|---|---|
Expand Down Expand Up @@ -121,7 +128,7 @@ Brendan Gregg toolchain path.
### Reading the result without being misled

A GPU flame graph is not a CPU flame graph with extra frames, and three of its
properties will mislead a reader who assumes otherwise (issue #123):
properties will mislead a reader who assumes otherwise:

- **A CPU frame under `[gpu:launch]` is as wide as the GPU time launched *from*
it** — not the CPU time spent in it. The Python and C++ frames you recognize
Expand Down
2 changes: 0 additions & 2 deletions examples/rust-pgo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,6 @@ rustc beating clang — the two examples' workloads and iteration counts
differ, and neither is a benchmark. Both exist to show that perf-agent's
output drives the AutoFDO toolchain.

See [#168](https://github.com/dpsoft/perf-agent/issues/168).

## Why this works

The dispatch loop hits the `Add` arm 99% of the time. Without PGO, the
Expand Down
Loading