Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

GO-eBPF-Proxy

A sidecar that transparently redirects an unmodified application's outbound IPv4 TCP egress to a local pass-through relay using eBPF (cgroup/connect4, no iptables/NAT), passively extracts the app's TLS session keys via an LD_PRELOAD interposer on its OpenSSL library, and captures the pod's network traffic to a DSB-embedded pcapng so the still end-to-end encrypted TLS 1.2/1.3 traffic can be decrypted and inspected offline with Wireshark/tshark. The Podman scripts under deploy/podman/ bring up a complete app+sidecar pod end-to-end (see Quick start).

No TLS termination. No MITM. No CA. The app's handshake stays end-to-end with the real server; its certificate validation is never weakened, so certificate pinning is a non-issue — the sidecar only reads the ephemeral keys the app's own TLS library already derives.

Why

Inspecting an app's encrypted egress usually means one of three intrusive options: instrumenting the app, relying on iptables/NAT (often blocked by policy), or a MITM CA (defeated by certificate pinning, and TLS 1.3 forward secrecy makes static-key decryption impossible anyway). go-ebpf-proxy avoids all three: transparent redirection at the socket layer, and TLS keys read passively from the process that already holds them.

How it works

  1. An eBPF cgroup/connect4 program intercepts connect(), records the original destination, and rewrites it to a local relay (127.0.0.1:15001).
  2. A sockops program re-keys that record by source tuple so the relay can recover it race-free after accept().
  3. The Go relay resolves the original destination and raw-pipes bytes both directions — it never parses or terminates TLS.
  4. A small LD_PRELOAD interposer (preload/libkeylogpreload.so), loaded into the app container's environment, wraps SSL_CTX_new/SSL_CTX_new_ex and registers an OpenSSL keylog callback, so it observes client_random and the TLS 1.2/1.3 session secrets as the app's own libssl derives them, and ships each line over a Unix domain socket to the sidecar.
  5. The sidecar's socket server reads those lines, validates and deduplicates them, and writes them as an NSS keylog (on tmpfs, 0600/0700). It keeps them only while capture runs: once capture stops, it still reads every line, so the app never waits, and discards it. The keylog file lives one run: the sidecar deletes it when it starts and when it stops, --retain or not.
  6. The sidecar also captures every frame on --capture-iface, both directions and unfiltered (the relay's outbound leg and all other pod traffic, ingress included), to a pcapng file, in-process (gopacket reading an AF_PACKET socket — no libpcap or external capture tool). Each key line is written into it as a Decryption Secrets Block as soon as it arrives, while every packet waits one second before it is written, so the keys land ahead of the handshake they decrypt. The capture is therefore self-decrypting while it is being written, even if the sidecar is killed. A clean stop only drains and closes it. Each run writes its own dump-<epoch>.pcapng, never overwriting a capture an earlier run kept. Capture is bounded: it stops, keeping its file, once the captures in the directory reach --max-bytes, and finished captures are deleted past --max-age. It is ephemeral by default: pod teardown wipes the capture directory unless the pod was brought up with RETAIN=1.
  7. Offline, tshark/Wireshark decrypts the pcapng from its embedded DSBs alone (or paired with the keylog) to show the app's real traffic in plaintext — proving the whole pipeline end-to-end.
flowchart LR
    app["app process<br/>(unmodified, LD_PRELOAD set)"] -- "connect()" --> kern["connect4 + sockops<br/>(eBPF)"]
    kern -- "redirect" --> relay["pass-through relay<br/>127.0.0.1:15001"]
    relay -- "raw pipe (end-to-end TLS)" --> eth0["pod eth0"]
    eth0 --> upstream["Upstream server"]
    eth0 -. "every frame, both directions" .-> capture["capture writer<br/>pcapng, bounded + ephemeral"]
    app -. "libssl handshake" .-> interposer["LD_PRELOAD interposer<br/>(SSL_CTX_new + keylog callback)"]
    interposer -- "NSS lines over unix socket" --> keylog["keylog socket server<br/>-> NSS keylog"]
    keylog -. "streamed in as DSBs" .-> capture
    capture -- "dump-epoch.pcapng" --> tshark["tshark / Wireshark<br/>offline decrypt"]
Loading

Pod view

go-ebpf-proxy runs as a second container in the same pod as the target app, sharing its network, IPC and UTS namespaces plus the host's cgroup namespace (no shared PID namespace — the interposer needs no PID/inode resolution). Every other container in the pod gets transparent redirection; one that dynamically links OpenSSL 3.x's libssl and has the interposer's LD_PRELOAD/GOEBPF_PRELOAD_SOCKET env vars set also gets TLS key extraction — no app code change, no rebuild, no trust-store edit.

flowchart TB
    subgraph pod["Pod (shared net/ipc/uts ns + host cgroup ns)"]
        subgraph appc["app container — LD_PRELOAD=libkeylogpreload.so"]
            app["app process<br/>connect(dst:any port)"]
            libssl["libssl (TLS 1.2/1.3)"]
            interposer["LD_PRELOAD interposer"]
        end
        subgraph sidecar["go-ebpf-proxy container (UID 1337)"]
            loader["eBPF loader"]
            relay["pass-through relay<br/>127.0.0.1:15001"]
            keylog["keylog socket server"]
            capture["capture writer<br/>pcapng + embedded DSB<br/>bounded, ephemeral retention"]
        end
        eth0["eth0 (shared pod network ns)"]
    end
    kernel["Kernel: connect4 + sockops<br/>(attached at the pod's common parent cgroup)"]

    app -- "connect() intercepted" --> kernel
    kernel -- "redirect to relay" --> relay
    libssl -. "handshake secrets read via SSL_CTX keylog callback" .-> interposer
    interposer -- "NSS lines over<br/>shared unix socket volume" --> keylog
    loader -. "attach cgroup programs" .-> kernel
    relay == "raw pipe (real cert validated by the app)" ==> eth0
    eth0 ==> upstream["Upstream server"]
    eth0 -. "every frame, both directions<br/>(AF_PACKET, unfiltered)" .-> capture
    keylog -. "streamed in as DSBs" .-> capture
    capture -- "dump-epoch.pcapng" --> disk[("/var/log/sidecar<br/>(host-visible volume)")]
    disk -. "tshark (keys from the embedded DSBs)" .-> decrypted(["decrypted plaintext"])
Loading

Swap in a different app container image — same or different language, any process that dynamically links OpenSSL 3.x's libssl and carries the interposer env vars — and the sidecar keeps working unchanged: the connect4/sockops programs redirect at the socket layer regardless of what wrote the syscall, and the interposer hooks the library, not any particular app binary.

Requirements

  • Linux kernel ≥ 5.10 (BTF/CO-RE) with cgroup v2 (the eBPF programs attach to a cgroup v2 path)
  • CAP_BPF, CAP_NET_ADMIN, CAP_SYS_RESOURCE (raises RLIMIT_MEMLOCK on eBPF load) and CAP_NET_RAW (opens the capture socket) — on the sidecar container only
  • Go ≥ 1.25, clang/llvm ≥ 14 and bpftool (for building the eBPF programs), a C compiler + OpenSSL 3.x headers (for building the LD_PRELOAD interposer)
  • Rootful Podman (local dev setup)
  • Supported platforms: Ubuntu 24.04 LTS is the reference platform — see the technical design document for the full supported-platform matrix
  • tshark (optional, only needed to run the offline decrypt step in Quick start)

Quick start

Brings up the real app+sidecar pod, proves a real end-to-end TLS 1.3 request through the relay, and decrypts the resulting capture offline. Requires a rootful Linux host with podman (see Requirements above); the pod scripts must run as root.

# 1. Build the sidecar binary (statically by default, for the musl/alpine sidecar
#    image) and the LD_PRELOAD interposer
make build
make build-preload

# 2. Bring up the pod: sidecar (eBPF redirect + keylog + capture) then the app,
#    wired with LD_PRELOAD/GOEBPF_PRELOAD_SOCKET automatically. To keep the capture
#    after teardown, bring it up with: sudo RETAIN=1 deploy/podman/pod-up.sh
sudo deploy/podman/pod-up.sh

# 3. Prove it end-to-end: a real HTTPS request through the relay, real cert validated
sudo deploy/podman/smoke.sh
# smoke: https://example.com -> HTTP 200

# 4. Decrypt the capture offline. It carries the session keys itself, written as
#    Decryption Secrets Blocks while it is captured, so it decrypts with no keylog
#    file: mid-run, after a clean stop, or after the sidecar was killed. Each run
#    writes its own dump-<epoch>.pcapng; ls -t picks the newest, this run's
CAPTURE="$(ls -t "$(podman volume inspect go-ebpf-proxy-sidecar-logs --format '{{.Mountpoint}}')"/dump-*.pcapng | head -n 1)"
tshark -r "$CAPTURE" \
  -o "tls.keylog_file:" -o tcp.reassemble_out_of_order:TRUE \
  -Y 'http or http2' -V
# -o "tls.keylog_file:" clears any keylog file set in your Wireshark preferences, so
# only the keys inside the capture are used. To pair the separate keylog instead, copy
# it off the sidecar's tmpfs while the sidecar runs (it deletes the keylog when it
# stops) and pass its path:
#   podman cp go-ebpf-proxy-sidecar:/var/log/sidecar-keylog-tmpfs/keylog/sslkeylog.log /tmp/sslkeylog.log
#   ... -o "tls.keylog_file:/tmp/sslkeylog.log" ...
# -Y 'http or http2' matches whichever protocol the session negotiated over ALPN:
# curl picks HTTP/2, which tshark dissects with a separate "http2" dissector that
# a plain "http" filter never matches.
# -o tcp.reassemble_out_of_order:TRUE reassembles TCP segments the capture recorded
# out of sequence, which tshark does not do by default.
# -V (full protocol detail) is required to see the actual decrypted body text --
# -Y alone only prints a one-line protocol summary per matching packet.

# 5. Tear down — wipes /var/log/sidecar unless the pod was brought up with RETAIN=1
sudo deploy/podman/pod-down.sh

Keeping the artifacts is decided once, when the pod comes up. RETAIN=1 on pod-up.sh alone keeps the capture through teardown: the sidecar then keeps its capture when it stops, and the pod records the choice as the label go-ebpf-proxy.retain=1, which a plain pod-down.sh follows. A pod brought up again on the kept volume writes its own capture beside the kept ones. If they already reach --max-bytes, pod-up.sh warns pod-up: capture not started and the pod runs without capture. The kept capture decrypts with no keylog file, and the tmpfs keylog file is never kept: the sidecar deletes it when it stops, RETAIN=1 or not, as the capture already carries every key it held, and a sidecar that starts deletes any keylog an earlier run left. RETAIN=0 pod-down.sh still wipes a retained pod. pod-down.sh --retain on a pod brought up without RETAIN=1 exits 1 and removes nothing, because that sidecar wipes its own capture when it stops.

POD_NAME, APP_IMAGE, CAPTURE_IFACE and friends are all overridable env vars on pod-up.sh (see the script header); the smoke test targets URL=${URL:-https://example.com}.

Configuration

go-ebpf-proxy has no user interface: no TUI, no web UI and no interactive mode. Its configuration, read once at startup, sets everything it does. At runtime it reports only through its structured JSON log (including a periodic stats line) and through the capture and keylog files. To change a setting, restart the sidecar with the new configuration.

Each setting can come from four places. Later ones override earlier ones:

  1. its built-in default;
  2. a config file named by --config (.yaml/.yml, .toml or .json);
  3. an environment variable: GOEBPF_SIDECAR_ plus the flag name in upper case with - turned into _, for example GOEBPF_SIDECAR_MAX_AGE=12h;
  4. its command-line flag.

The config-file key is the flag name without its leading dashes (max-age: 12h). An unknown setting in the file or in a GOEBPF_SIDECAR_* variable stops the sidecar at startup with an error naming it, and so does a key with no value or a value of the wrong type, such as an empty max-bytes or a duration without a unit (max-age: 3600). A typo never falls back silently to a default. pod-up.sh passes the flags it needs for you:

Flag Default Purpose
--cgroup-path none (required) Pod common-parent cgroup v2 path the eBPF programs attach to
--relay-listen 127.0.0.1:15001 Address the pass-through relay listens on. Leave it at the default: connect4 always redirects to 127.0.0.1:15001 (compiled in), so any other address leaves redirected connections with no listener
--pin-dir /sys/fs/bpf/go-ebpf-proxy bpffs directory for the pinned maps
--keylog-socket /var/run/sidecar/keylog.sock Unix socket the LD_PRELOAD interposer connects to
--keylog-path /var/log/sidecar-keylog-tmpfs/keylog/sslkeylog.log NSS keylog file; its directory must be on tmpfs, or the sidecar refuses to start. It holds one run's keys: the sidecar deletes it when it starts and when it stops
--capture-iface eth0 Interface to capture: every frame, both directions, unfiltered (the relay's outbound leg and all other pod traffic)
--capture-path /var/log/sidecar/dump.pcapng Template for each run's DSB-embedded pcapng capture: a run writes dump-<epoch>.pcapng (its start time in Unix seconds, then -1, -2, … if taken) in that directory, never overwriting a file
--retain false Keep this run's capture, with its embedded TLS secrets, on teardown instead of deleting it; the keylog is never kept
--max-bytes 104857600 (100 MiB) Size limit on the captures in the capture directory, this run's included: reaching it stops capture and keeps the file, while the relay keeps serving the app; a run that finds it already reached starts without capture. Nothing is deleted for size; 0 disables it
--max-age 24h Each retention pass deletes finished captures older than this, never the running capture or any other file; 0 disables it
--retention-interval 5m How often retention is enforced; 0 disables it
--stats-interval 1m How often the stats line is logged, plus once at shutdown; 0 disables it
--log-level info Lowest level logged: debug, info, warn or error, optionally offset like warn+2; warn drops the per-connection connection relayed lines
# sidecar.yaml, used with --config=sidecar.yaml
cgroup-path: /sys/fs/cgroup/machine.slice/machine-libpod_pod_abc.slice
max-age: 12h
retain: false

Configuration is loaded with koanf.

Build & test

make build             # fmt -> vet -> build
make build-preload      # build the LD_PRELOAD keylog interposer (.so)
make test               # unit tests (go test -race ./...)
make test-integration   # kernel/eBPF-backed tests (build tag `integration`; skip cleanly
                        # without the required capabilities)
make lint               # golangci-lint, covers unit and integration-tagged files
make check              # lint + build + test + test-integration

eBPF objects are pre-generated and committed (bpf/*_bpfel.go, *_bpfeb.go). To regenerate them after editing a .bpf.c file:

make generate

Project layout

bpf/             eBPF C programs (connect4, sockops) + generated bindings
internal/ebpf/   loader: load/attach/pin the eBPF programs, map codec
internal/proxy/  original-dst resolver (fail-closed) + pass-through L4 relay
internal/keylog/ keylog socket server, NSS keylog validation/writer
internal/capture/ live AF_PACKET capture, pcapng writer + embedded DSB, retention
internal/capture/capturetest/ tshark offline-decryption check (test support only)
preload/         LD_PRELOAD interposer (C): hooks SSL_CTX_new/SSL_CTX_new_ex, ships keylog lines
internal/shared/ structured logging, shared infra
cmd/app/         sidecar entrypoint (loads configuration; wires loader, relay, keylog socket server, capture)
deploy/podman/   rootful Podman dev environment (pod scripts)

Current scope

  • Interception: connect4 + sockops redirect every IPv4 TCP connect() from any container in the pod, on any destination port (not only 443), to the relay. Only the sidecar's own UID 1337 and loopback destinations are skipped. For now the relay only forwards raw TCP to whatever IP and port the app dialed; a future release will analyze and inspect that traffic, passively.
  • TLS keys: the LD_PRELOAD interposer logs the session keys of every TLS handshake, for any protocol over TLS (not only HTTPS), client or server side, in any app that dynamically links OpenSSL 3.x (not Go crypto/tls, Java, or a statically linked TLS library).
  • Capture: the pcapng records every frame on --capture-iface in both directions, unfiltered (TCP on any port, UDP, ICMP, ingress included), not only HTTPS, with the TLS keys embedded as Decryption Secrets Blocks.

Project Documentation

GO-eBPF-Proxy - Public Docs

Skills reference

License

GPL-3.0 — see LICENSE.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages