From b04880e2ddc94733baf944409919f8d37760b4b9 Mon Sep 17 00:00:00 2001 From: magi Date: Fri, 18 Sep 2026 22:56:25 +0530 Subject: [PATCH] Replace unsubstantiated performance claims with real benchmark numbers What: the docs site's "Performance" section and PostPyro_documentation.md's Performance Notes previously implied PostPyro is faster than asyncpg/ psycopg (star ratings in the original page, unqualified "why PostPyro is faster" framing) without any measurement behind it. Ran benchmarks/bench_vs_alternatives.py against a live Postgres 16 and replaced the claims with the actual numbers: | Benchmark | PostPyro | asyncpg | psycopg3 | |-------------------------------|---------:|--------:|---------:| | Round trip (200x SELECT 1) | 29.86ms| 20.74ms| 17.19ms| | Bulk insert (1,000 rows) | 4304.05ms|4475.53ms| 4171.60ms| | Transaction (4 statements) | 8.90ms| 4.85ms| 4.13ms| | Concurrency (20 tasks) | 2.81ms| 1.16ms| 3.91ms| Read plainly: PostPyro is currently slower on round-trips and transactions, roughly on par on bulk inserts, mixed on concurrency - not a "PostPyro is faster" story. No optimization pass has happened yet on the async rewrite; correctness and API stability came first. Both docs keep the caveat that this is one machine, one point in time, and link to benchmarks/README.md to reproduce it. Also softened the "Rust backend" reason card, which implied an advantage over asyncpg/psycopg3 specifically from avoiding Python interpreter overhead - both of those are also compiled (Cython/C), so that framing was misleading in a driver-comparison context even though the underlying fact (Rust, not Python) is true. --- PostPyro_documentation.md | 13 ++++++++++++- docs/index.html | 24 +++++++++++++++++++++++- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/PostPyro_documentation.md b/PostPyro_documentation.md index feb2f2a..76f89fa 100644 --- a/PostPyro_documentation.md +++ b/PostPyro_documentation.md @@ -498,7 +498,18 @@ await users.save("Alice", "alice@example.com") - **Async I/O**: `sqlx`'s async networking releases the GIL during I/O instead of blocking the whole process - **Binary protocol**: query results come back over Postgres's binary wire protocol, parsed in Rust -The performance comparisons published for earlier (pre-rewrite, synchronous) versions of PostPyro no longer apply to this async driver and have not been re-benchmarked yet; treat any such numbers you find elsewhere as stale. `benchmarks/` has a harness that runs real timing against PostPyro, asyncpg, and psycopg on your own Postgres - run it yourself rather than trusting a number posted here, see `benchmarks/README.md`. +The performance comparisons published for earlier (pre-rewrite, synchronous) versions of PostPyro no longer apply to this async driver. `benchmarks/` has a harness (`bench_vs_alternatives.py`) that runs real timing against PostPyro, asyncpg, and psycopg on your own Postgres. + +Most recent maintainer run (one machine, one Docker Postgres 16, one point in time - not a claim about your hardware or workload; `REPEATS=5`, median reported): + +| Benchmark | PostPyro (ms) | asyncpg (ms) | psycopg3 (ms) | +| ----------------------------------------- | -------------: | ------------: | -------------: | +| Round trip (200x `SELECT 1`) | 29.86 | 20.74 | 17.19 | +| Bulk insert (1,000 rows, one at a time) | 4304.05 | 4475.53 | 4171.60 | +| Transaction (4 statements) | 8.90 | 4.85 | 4.13 | +| Concurrency (20 tasks via `asyncio.gather`) | 2.81 | 1.16 | 3.91 | + +Read plainly, not favorably: PostPyro is currently slower than both on plain round-trips and transactions, roughly on par on bulk inserts, and behind asyncpg but ahead of psycopg3 on concurrency. This is not a "PostPyro is faster" story - no optimization pass has happened yet on the async rewrite; stabilizing the API and correctness came first. Run the harness yourself for numbers that matter to a real decision; see `benchmarks/README.md`. ## Advanced Usage diff --git a/docs/index.html b/docs/index.html index f7648c9..185b75b 100644 --- a/docs/index.html +++ b/docs/index.html @@ -607,11 +607,19 @@

Performance

+
+ +
+

Not a speed claim

+

The architecture below is real, but asyncpg and psycopg3 are compiled too (Cython/C), so "no Python interpreter overhead" isn't a PostPyro-only advantage. The numbers underneath are from one machine, one Docker Postgres 16, one point in time - not a claim about your hardware or workload. Run benchmarks/bench_vs_alternatives.py yourself before trusting any number here - see benchmarks/README.md.

+
+
+

Rust backend

-

Native performance without Python interpreter overhead.

+

Query execution and type conversion happen in compiled Rust, not interpreted Python.

@@ -630,6 +638,20 @@

Zero dependencies

+

Measured results

+
+ + + + + + + + +
BenchmarkPostPyro (median ms)asyncpg (median ms)psycopg3 (median ms)
Round trip (200× SELECT 1)29.8620.7417.19
Bulk insert (1,000 rows, one at a time)4304.054475.534171.60
Transaction (4 statements)8.904.854.13
Concurrency (20 tasks via asyncio.gather)2.811.163.91
+
+

PostPyro is currently slower than both on plain round-trips and transactions, roughly on par on bulk inserts, and behind asyncpg but ahead of psycopg3 on concurrency. No optimization pass has happened yet - see benchmarks/README.md for the exact methodology and how to reproduce this.

+

Driver comparison