Skip to content

Fix null reactor in WebClient destructor (websocket_port segfault) - #2171

Open
profinch wants to merge 3 commits into
BeamMW:masterfrom
profinch:fix/websocket-handler-null-reactor
Open

profinch wants to merge 3 commits into
BeamMW:masterfrom
profinch:fix/websocket-handler-null-reactor

Conversation

@profinch

@profinch profinch commented Nov 2, 2026 •

Copy link
Copy Markdown

Fixes #2170.

A node started with websocket_port segfaults at 0x418 in Reactor::cancel_tcp_connect: ~WebClient calls io::Reactor::get_Current(), and that returns null on the thread where the handler actually gets released.

  • websocket/reactor.cpp: SafeReactor::callAsync copied the callback into its queue. The caller's copy kept a reference to the captured handler, so if the reactor thread ran first, the last reference could be dropped in the asio thread. The callback is now moved into the queue and out of it, so the queue entry is the only owner.
  • beam/cli.cpp: WebClient keeps the Reactor::Ptr it connected with and uses it in the destructor. This covers shutdown, where queued handler releases are destroyed with SafeReactor after io::Reactor::Scope has ended.

Testing: a node with an empty DB and --websocket_port, 500 WebSocket clients sending frames every 1 ms, SIGINT after about 2 s. Release 7.5.14548 crashed at the same address in 3 of 85 runs, this branch in 0 of 85. The same two commits on top of beam-7.5.14548 run on our mainnet pool node, built on Debian 13 with GCC 14.

Also: plain WebSocket sessions closed after 30 s

  • websocket/sessions.h: DetectSession sets a 30 s expiry on the tcp_stream while it detects TLS. SecureWebsocketSession::on_handshake turns it off (expires_never()), but the plain path never did, so with --websocket_use_tls=0 (TLS done by a reverse proxy) every session was closed 30 s after it opened, whatever it sent: read: The socket was closed due to a timeout. Browser wallets then reconnect about every 35 s, all day, and each of those closes goes through the handler release above, which is why the crash came about once a day. WebsocketSession::run now turns the expiry off before it sets the websocket's own timeouts, as the secure session does.

Testing: an idle WebSocket to a node with --websocket_port --websocket_use_tls=0: without this commit it is closed at 30.0 s with the log line above, with it it is still open at 90 s (same build, the commit reverted and reapplied, GCC 13 on Ubuntu/WSL). On our mainnet node, behind nginx, the wallets' sessions were closed every ~35 s.

callAsync() pushed a copy of the callback into the queue, so the caller's
parameter kept its own reference to whatever the callback captured until
the function returned. ~WebsocketSession hands the client handler to the
reactor thread this way. If the reactor thread popped and ran the callback
before the asio thread destroyed its copy, the last reference was dropped
in the asio thread, and ~WebClient (beam/cli.cpp) called
io::Reactor::get_Current().cancel_tcp_connect() with no current reactor.
That is a null dereference: segfault at 0x418 in
Reactor::cancel_tcp_connect.

Seen on a mainnet node with websocket_port enabled for browser wallets:
about one crash a day, each at the moment a WebSocket session times out.

Move the callback into the queue and out of it, so the queue entry is the
only owner and captured objects are released in the reactor thread.
…he destructor

~WebClient called io::Reactor::get_Current().cancel_tcp_connect(). The
current reactor is thread-local and is only set inside io::Reactor::Scope,
but a WebSocket client handler can be released outside it. At shutdown
the WebSocket server queues the release of its handlers into SafeReactor
after the reactor has stopped; nobody drains that queue, and it is
destroyed with SafeReactor in main() after the Scope has ended. The
handlers die in the main thread with get_Current() == null, and the node
segfaults at 0x418 in Reactor::cancel_tcp_connect right after
"Node stopped".

Take a Reactor::Ptr in the constructor (the handler is created in the
reactor thread) and use it in the destructor. SafeReactor destroys its
queue before its reactor, so the reactor is still alive at that point.
profinch added a commit to profinch/bumblebeam that referenced this pull request Nov 2, 2026
…p ap-nodes

The official 7.5.14548 node segfaults about once a day with
websocket_port enabled (BeamMW/beam#2170). Until the fix (BeamMW/beam#2171)
is released, the node runs a build of 7.5.14548 with that PR from
/data/bin/beam-node (bind-mounted with the node data).

ap-nodes.mainnet.beam.mw no longer resolves and is gone from Beam's own
default peer list; eu-nodes and us-nodes remain.
…detection

DetectSession sets a 30 s expiry on the tcp_stream while it detects TLS. The
secure session turns it off after its handshake (expires_never in
SecureWebsocketSession::on_handshake), but a plain session kept it, so every
plain WebSocket session was closed 30 s after it opened, whatever it sent
("read: The socket was closed due to a timeout"). The plain session now turns it
off before it sets the websocket's own timeouts, as the secure one does.
profinch added a commit to profinch/bumblebeam that referenced this pull request Nov 6, 2026
…connections per address, wsb.bumblebeam.org

- beam-node runs 7.5.14548 + BeamMW/beam#2171: besides the null-reactor segfault, plain WebSocket
  sessions were closed 30 s after they opened (a timer set for the TLS detection, never turned off on the
  non-TLS path), so browser wallets reconnected every ~35 s all day (#2170)
- wss.conf: 12 connections per address (the wallet runs one engine per Beam wallet, plus a short probe)
- deploy/cloudflare/wsb: a Cloudflare Worker, the wallet's backup door to Beam nodes over TCP, independent
  of our server: nodes checked to serve a light wallet at the tip, tried in random order, ours last;
  binary messages arrive as Blobs there
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

beam-node segfaults in Reactor::cancel_tcp_connect (null reactor) when websocket_port is enabled

1 participant