Synth perf: wire protocol is a synchronous ping-pong (~2x headroom); JSON is not the bottleneck — pipeline via client-allocated object refs

[mrgrain]

Summary

For large CDK (Python) apps, synthesis time scales roughly linearly with app size, and the host↔runtime wire protocol is a prime suspect. I profiled a real synth on both sides of the protocol. The finding: the wire format (JSON) is not the bottleneck — the synchronous, one-round-trip-per-operation structure is. Each side sits idle roughly half the time waiting for the other. There is a credible ~2× headroom, but capturing it requires pipelining the protocol (overlapping round-trips), which in turn requires optimistic / client-allocated object references.

This is the remaining big lever after assembly-load and kernel hot-path improvements (see related items below).

Method

• App: a synthetic CDK Python stack, ~2,200 resources → 16,827 kernel requests.
• Captured the full request stream; classified responses by replaying through KernelHost.
• Instrumented the Node child runtime (InputOutput) to bucket time into idle (blocked on readLine) / JSON.parse / JSON.stringify / pipe write / work (remainder).
• Instrumented the Python client send() to bucket time into serialize / write / wait (blocked on the runtime) / structure, and to attribute wait by api.
• Numbers are from one app on one machine; treat magnitudes as indicative, not exact. (Happy to share the harness.)

Message mix

api	share	count
create	38.2%	6,421
invoke	33.3%	5,601
get	14.3%	2,400
sinvoke	9.5%	1,600
sget	4.8%	800
set / sset / del	0%	0

CDK configures objects through constructor props, so there are no property sets — the volume is creates and (value-returning) invokes.

Where the time goes

Node / child runtime (full synth, ~5.7 s wall):

bucket	ms	share
work (kernel dispatch + construct JS + load)	~2,880	~50%
idle (blocked on readLine, waiting for host)	~2,700	~47%
JSON.parse	~38	0.7%
JSON.stringify	~7	0.1%
pipe write	~74	1.3%

Python client send(): total 4.31 s, of which wait (blocked on runtime) 4.00 s (93%), serialize (cattrs + json) 0.19 s, structure 0.08 s, write 0.04 s. Wait by api: invoke 1.90 s, create 1.42 s, load 0.39 s, get 0.14 s, sinvoke 0.10 s, sget 0.06 s.

Conclusions

1. Serialization/format is not the bottleneck. JSON encode+decode+write is ~2% of Node time and ~6% of Python time. A binary protocol (protobuf/msgpack/etc.) would chase single-digit percentages at the cost of cross-language complexity. Not worth it.
2. It's a synchronous ping-pong. Wall time ≈ python_work + node_work + handoff, fully serialized; each side is idle ~half the time. With perfect overlap the floor is ~max(python_work, node_work) ≈ Node's 2.88 s vs today's ~5.7 s → ~2× ceiling. Of the 4.0 s the host waits, ~2.88 s is the runtime actually working and ~1.1 s is pure handoff across 16,827 round-trips (~65 µs/msg) — partly attributable to the extra parent→child fd3 relay hop in bin/jsii-runtime.js.
3. The pipelineable volume is creates and invokes. In this app, 100% of invokes/sinvokes (7,200) return object references that are used structurally (passed to later calls) and never inspected — like creates. So ~3.4 s of the 4.0 s host wait is on operations whose result is an objref the host doesn't actually need to read; only get/sget (~0.2 s) are hard synchronization points.

Proposed direction: pipeline the protocol via client-allocated object references

The trivial "fire-and-forget void ops" idea does not apply (no void volume). To overlap creates/invokes, the host must be able to use a returned objref before the runtime confirms it. The robust, non-fragile mechanism is client-allocated object ids:

• The host mints the object id and sends it with create (and objref-returning invoke); the kernel honors the provided id instead of allocating from its #nextid counter.
• The host does not block on the response; it continues, using the id it just minted.
• Acks are drained lazily, with a cap on outstanding requests to avoid pipe-buffer deadlock; the host blocks (drains) only at true sync points: get/sget, callbacks, and end of synth.

Tradeoffs to design for:

• Error attribution shifts: a failed pipelined op surfaces at the next sync point, not the call site. Requests would need tagging to map deferred errors back.
• Callbacks (runtime → host overrides) are hard sync points and must drain/coordinate the pipeline.
• Ordering / id-space: the kernel must accept and trust client ids (in a defined range) without collisions.
• Two-sided change (kernel + each language runtime); ideally introduced behind a capability negotiated at the load/hello handshake so old and new peers interoperate.

Suggested next step

Prototype behind a flag: kernel accepts a client-supplied objid; Python client allocates ids, pipelines create/invoke, and drains lazily at sync points (relaxed error handling for the spike). Measure real wall-time against the ~5.7 s baseline to confirm the ceiling.

Related

• Assembly loading: lazy per-package runtime cache index (PR perf(kernel): ~67% faster assembly loading via a runtime index #5164).
• Kernel per-message hot path (PR fix(kernel): improve performance by ~10% by deferring error-context and cache serialization values #5162).
• Python runtime type-checking per-call cost (Make Python runtime type checking cheaper / opt-in (cache get_type_hints; kernel errors now detailed) #5165).

This issue tracks the wire-protocol/round-trip dimension specifically.

[rix0rrr]

Hello Kiro! 👋

The following is incorrect reasoning:

100% of invokes/sinvokes (7,200) return object references that are used structurally (passed to later calls) and never inspected — like creates. So ~3.4 s of the 4.0 s host wait is on operations whose result is an objref the host doesn't actually need to read; only get/sget (~0.2 s) are hard synchronization points.

It is not about the jsii method used to obtain a value, it's what is being done with that value. Correct, a create can only return an object handle (can trivially be pipelined), but:

• An invoke could equally well return an object handle as a primitive value, in which case it becomes a synchronization point.
• Similarly, a get could also return an object handle, in which case it doesn't need to be a synchronization point.

Synchronization points happen when Python code needs to observe a jsii value.

[mrgrain]

In practice, pipelining only create doesn't help. They are almost always followed by an invoke, so the pipeline never grows beyond a couple of messages.

Pipelining invoke is much more difficult because of the issue @rix0rrr noted and also currently the runtime proxies need the type being returned from the kernel. We can maybe replace the latter by using the declared typed. Though this might limit the performance benefits.

[mrgrain]

Posted by Kiro (AI agent), on @mrgrain's behalf.

Follow-up: what is the runtime ("Node side") actually spending time on?

The earlier breakdown showed the runtime is ~50% "work" / ~47% idle. Half of a ~5.7 s synth in "work" for ~2,200 constructs seemed high, so I decomposed it with a CPU profile of the kernel driven over a captured request stream (real KernelHost + real aws-cdk-lib construct JS; warm package cache; single app; (idle) here is a file-read artifact of the replay harness, not real). Proportions are the takeaway, not absolute ms.

Self-time by area (warm replay)

area	share	nature
aws-cdk-lib + constructs	~29%	real construct JS (token resolution, Intrinsic/Reference, synthesis) — a native TS app pays this
file reads / module load (readFileUtf8, internalModuleStat, wrapSafe)	~big	loading the used aws-cdk-lib JS + module resolution — also paid by native
.jsii assembly parse (followRedirect = gunzip + JSON.parse)	~9%	jsii-only; removed on warm loads by the lazy runtime index (#5164)
jsii kernel + runtime dispatch/serialization	~14%	jsii-only per-op boundary tax
CDK metadata stack-trace capture	~5%	aws-cdk-lib/core/lib/stack-trace.js — CDK, not jsii (disableable; helps native too)
GC	~5%

Findings

1. Lazy module loading works correctly. The run loads 13 submodules / 193 .js files, not the full library — exactly the services the app uses (core, aws-iam, aws-sqs, aws-lambda, aws-sns, aws-dynamodb, aws-s3, aws-sns-subscriptions, + cx-api/cloud-assembly-schema). core (82 files) dominates and is unavoidable. So the load cost is proportional to usage, not an over-loading bug.

2. The kernel per-op tax is lean: ~321 ms total ≈ ~19 µs/op over 16,823 ops, split roughly evenly between object-table bookkeeping (registerObject/tagObject, ~35 ms), value serialization (process/deserialize/mapValues/propertiesOf, ~72 ms), and dispatch + type lookups (findSymbol/typeInfoForFqn, ~58 ms). The kernel is not where the time goes.

3. captureStackTrace (~87 ms, ~5%) is CDK, not jsii — aws-cdk-lib captures a call stack per construct for metadata. A native CDK app pays it too, but it's disableable and would speed up every synth.

Two micro-levers I measured — both negligible

• Compiling the kernel to ES2022 (native #private instead of the __classPrivateFieldGet WeakMap helper): the helper disappears entirely, but replay wall was 1594.7 ms vs 1587.0 ms baseline — within noise. V8 already optimizes the helper comparably.
• Memoizing the per-create interface-set computation: of ~35 ms object-table self-time, only ~10 ms is the per-class-redundant part (removeRedundant/addFrom*); the rest is irreducible per-instance registration. ~0.6% of the run, and a prior attempt at this cache landed within noise.

Conclusion

The runtime "work" is mostly the same work a native TS CDK app of this size would do (loading core + used services, executing construct/token/synthesis logic, CDK metadata stack traces, GC). The jsii-specific surcharge is ~¼ of the work: the .jsii assembly parse (being removed by #5164) plus a lean ~19 µs/op dispatch+serialization tax. The kernel itself isn't a hot spot — the headroom is in load (#5164) and the synchronous round-trip structure (this issue), not kernel CPU.