Repository navigation
River Rust port - #1442
Merged
Merged
River Rust port#1442
Conversation
Start a Rust workspace under `rust/` with `riverqueue-migrate`, which applies, lists, previews, and validates River's PostgreSQL and SQLite migration lines from Rust. It shares the `river_migration` history with River Go, so either language can migrate a database the other uses, and it accepts any quotable PostgreSQL schema name. The crate can't read files outside its package, so it carries mirrors of Go's canonical migrations. `syncrustmigrations` writes the mirrors and the hashes in the conformance migration inventories from Go's migration directories, and `make verify/rust-migrations`, which the conformance artifacts CI job now runs, fails when they drift, so Rust never gains an independent schema history.
Add `riverqueue-macros` with `#[derive(JobArgs)]`, which requires a stable `#[river(kind = "...")]` and can declare kind aliases, the default queue, max attempts, priority, pending state, tags, and default unique options, including the fields that make up a `by_args` key. Invalid attributes fail at compile time with spans that point at them. Applications receive the macro through `riverqueue`.
Add `riverqueue`, a Rust and Tokio implementation of River that shares the database schema and job protocol with River Go on PostgreSQL and SQLite, so Rust and Go clients can insert and work jobs in the same database. `Client` takes a caller-owned SQLx pool and isn't generic over the database or a driver trait. Workers are typed and async and get a `CancellationToken`; request builders cover insertion (including transactional, batch, unique, scheduled, and pending jobs), job and queue management, and transactional completion from inside a worker. Like River Go's `*Tx` methods, a request given the caller's transaction runs directly in it without a savepoint, so the caller rolls back on an error. The runtime fetches, completes in bounded concurrent batches, retries, snoozes, cancels, and rescues jobs the way Go does, publishes events only after their database update commits, and runs leader election and maintenance services: the rescuer, cleaners, scheduler, reindexer, and periodic jobs with Go-compatible cron parsing. Hooks, middleware, error handlers, retry policies, and a hidden extension module for lockstep add-on crates mirror Go's extension points. Persisted values follow Go exactly where another engine reads them: unique keys, metadata, attempt errors, notifications, and timestamps. Unit tests check unique keys and cron schedules against the conformance fixtures, which the fixture generator now also copies into the crate because published tests can't read files outside it.
Now that `riverqueue` exists, test the derive macro end to end: derived kinds, aliases, insert options, and unique keys, plus compile-fail cases for each attribute error, checked with `trybuild`.
Add integration tests for the storage layer: the backend contract, job and queue CRUD, list filters and cursors, exact metadata, the protocol fixtures, and parity with the rows River Go writes on each database. A test also fails if a dependency enables `serde_json` features that would change its behavior for the rest of an application. PostgreSQL tests build only with `--cfg river_postgres_tests` and run in a freshly migrated schema with a unique name, so test binaries can share one disposable database. They fail rather than skip when `RIVER_RUST_DATABASE_URL` is unset.
Add integration tests for running clients: start and stop lifecycles, runtime configuration, producer lifetimes, stuck jobs, fetching only known kinds, poll-only cancellation, running without leader election, insert notifications, the job and queue handles, and the full SQLite runtime.
Add integration tests that run clients against malformed rows and database faults on PostgreSQL and SQLite: proxied connections that are dropped and refused while clients run, after which producers, completion, notifications, and leadership must recover. Clients must also detect a PostgreSQL server that looks like YugabyteDB and fall back to polling.
Add integration tests for what add-on crates build on: work middleware, hooks, and error handlers and their ordering; extension services and the client's stop order; producer sessions and the checks River applies to their claims; peer attempts that River owns until each outcome persists; prepared insertion of stored jobs; filtered deletion of finalized jobs; and batched insert interception. Tests also cover requests run in a caller's transaction: they open no savepoint, so an extension step, insert middleware, or decode failure after River's write leaves that write in the transaction for the caller to roll back, a failed statement aborts a PostgreSQL transaction, and every write carries the caller's transaction ID.
Add runnable examples for a basic worker, graceful shutdown, cancellation, transactional completion, unique and periodic jobs, event subscriptions, custom schemas, SQLite, and a deployment where Go and Rust clients share one database.
Add `riverqueue-test`: assertions that check which jobs a test's code inserted, with optional expected properties and variants that read through an open transaction, and helpers that run a worker once with or without a database.
Add `riverqueue-cli`, which installs a `riverqueue` binary that migrates PostgreSQL and SQLite databases like `river migrate-*` and benchmarks worker throughput and end-to-end latency like `river bench`.
Have Dependabot update the Rust workspace's dependencies weekly, grouping minor and patch updates after a seven-day cooldown, and the GitHub Actions the workflows use.
Add a workspace README listing the crates and how to run their checks, benchmark, and PostgreSQL tests, and a changelog for the crates, which are versioned and released together.
From Codex: > It assumed two database inserts would finish within a **50 ms > cooldown**. Under CI load, the cooldown expired, so emitting two > notifications was correct.
Contributor
|
This is the same Rust code as #1436 without the conformance pieces, so let's carry this one forward. Two changes ours had locally that aren't here yet:
Both are commits on |
Contributor
Author
|
@bgentry Thx! I had to push a couple other fixes already, so let me cherry pick those in so we don't run into contention here. |
The root module's zip, which is what proxy.golang.org serves for every `go get github.com/riverqueue/river`, includes every file under the module root except nested modules. Without a module boundary in `rust/`, it ships the whole Rust workspace (187 files) to Go users, and `go test ./...` and other `./...` patterns walk into `rust/target`. Add a stub `go.mod` that makes `rust/` its own module so the Go tooling and the published module zip leave it out.
Rust and JavaScript previews are versioned as a prerelease of the next River Go release. River v0.49.0 has shipped, so the crates move from `0.49.0-alpha.1` to `0.50.0-alpha.1`, a version that semver orders after the release they're compatible with. Update the workspace version, the exact requirements between workspace crates, `Cargo.lock`, the README install snippets, and the mixed-deployments note on which River Go release a preview works with.
brandur
force-pushed
the
brandur-rust-port-no-conformance
branch
from
October 5, 2026 21:17
3c6696f to
7d794fd
Compare
Contributor
Author
|
OMG, GHA finally managed to run the whole thing. @bgentry Okay, going to merge this, we can probably invert the order and get a conformance suite in after. This might be better to have testing against multiple languages anyway. |
This was referenced Oct 5, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Equivalent to this other version except that we drop out the conformance suite
to cut down on 30k LOCs and keep the build running quickly:
#1436