[resetius]

Problem

The driven awaitables (Accept, Connect, ReadSome, WriteSome of TPollerDrivenSocket and TPollerDrivenFileHandle, used by uring and IOCP) post an operation with the coroutine handle and keep no record of it. If the coroutine is destroyed while waiting (e.g. an unfinished TFuture dropped by Any), the late completion resumes the freed frame, and the kernel can still write into buffers in that frame. The same happens when the coroutine is destroyed after its completion is collected but before it is resumed (e.g. from a timer in the same iteration), and in the window where a timed out Connect waits for its cancelled operation.

Fix

• The driven awaitables keep their handle while suspended and call poller->Cancel(handle) from the destructor. Accept closes a connection accepted meanwhile, Connect also removes its timer. The Connect timeout path uses the same Cancel instead of cancel by fd and a second suspension.
• TUring::Cancel(handle) (existing but unused; it also left the cancel SQE's user_data unset):
  • a completion already collected, or reaped by an earlier Cancel, is dropped;
  • an operation whose SQE is not submitted yet is turned into a NOP, so it never reaches the kernel and consumes no data or connection;
  • otherwise it submits IORING_OP_ASYNC_CANCEL for that user_data and reaps completions until the cancelled one is posted, so the kernel is done with the buffers when it returns. Completions reaped meanwhile are delivered by the next Wait(), which then does not block.
  • It does not throw (it runs in destructors). It stops waiting only on an error that persists.
  • The wait ends because every completion of a pending operation is in the SQ, the kernel, the CQ, the reaped list or ReadyEvents_. For that Wait() now keeps completions that a previous Wait() collected but nobody delivered (neither WakeupReadyHandles() nor Result()); before, two Wait() calls in a row lost them, and Cancel then waited forever. A plain Wait()/Result() loop, as in the TUring example, still gets each result once. It does not stop on the cancel's -ENOENT: a buffered read waiting for a page is not found by ASYNC_CANCEL while it is still in flight.
• TIOCp::Cancel(handle) drops a collected or already dequeued completion. Otherwise it marks the TIO, calls CancelIoEx for its OVERLAPPED and waits until the operation is complete: its packet is dequeued, or the OVERLAPPED is no longer STATUS_PENDING. So the buffers (including the AcceptEx output buffer in the awaitable) are no longer used when it returns. The second condition covers handles not associated with the port, which get no packet: TPollerDrivenFileHandle does not call Register, and registering it is not safe while the Windows pipe() returns CRT descriptors rather than handles. A packet that comes after the wait is freed by Poll() without resuming anything. Other completions dequeued meanwhile are delivered by the next Poll(). The TIO is found by walking the list of pending TIOs: O(pending operations), but only when a coroutine is destroyed mid-operation; avoiding it would need a token returned by every posting call or an index keyed by handle. TIOCp::Read/Write had their error check inverted (an immediate failure was not reported, an immediate success could free the TIO and throw); it is fixed, so an operation that fails to start leaves no pending TIO.
• Both pollers keep results by index next to ReadyEvents_ instead of a queue, and their WakeupReadyHandles() skips dropped completions, so the next coroutine still gets its own result.
• The class docs of TPollerDrivenSocket and TPollerDrivenFileHandle say that the poller must outlive coroutines suspended on their operations (the same contract as Fix stale handles in poller #140 for the readiness awaitables).

Hot path cost: a handle store on suspend and resume, on IOCP a list link and unlink per operation, and on uring a loop in Wait() over undelivered events (empty when WakeupReadyHandles() ran). No maps or extra allocations.

Tests

• test_destroyed_driven_reader_not_resumed, test_destroyed_driven_accept_and_connect_not_resumed (TUring; TIOCp on Windows): readers, Accept and a Connect kept pending by a full accept queue, destroyed before submission, while pending, after completion but before collection, after collection, and from a coroutine resumed in the same batch. They check that nothing is resumed, a byte sent after the destroy stays in the socket and out of the buffer, the next result in a batch is correct, and a connection accepted meanwhile is closed.
• uring only: test_uring_wait_result_without_wakeup (the example's Wait()/Result() pattern), test_uring_destroyed_before_submit (a byte or a connection already there is left for the next reader or Accept), test_uring_destroyed_file_reader_and_writer (pipe), test_uring_destroyed_socket_writer, test_uring_destroyed_after_repeated_wait, test_uring_destroyed_reader_with_full_cq (more completions than the CQ holds).

Linux (aarch64, kernel 7.3, liburing 2.15) with ASan:

• master: all seven destroy tests fail, six with heap-use-after-free in TPollerBase::WakeupReadyHandles, one on the Results_.empty() assert in Wait();
• the first version of this PR (3fddaf6): before_submit and file_reader_and_writer lose the byte, after_repeated_wait hangs;
• bc4ba24: wait_result_without_wakeup gets the previous result again (it passes on master and 3fddaf6);
• now: the whole suite passes (146 tests), also merged with Fix stale handles in poller #140. macOS suite passes.

IOCP is checked only by Windows CI. The tests cannot force a completion to land after Cancel returns.

#140 adds TEvent::CANCEL changes from RemoveEvent(fd, h); TUring::Wait() asserts on them. The driven awaitables do not use it. On uring only inherited readiness awaitables (Monitor) would queue one, and they already trip the same assert when they register.

Fixes #145

[github-actions]

Unit Test Results (MacOS)

146 tests  ±0   146 ✅ ±0   21s ⏱️ ±0s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit f4f1336. ± Comparison against base commit cae2aef.

♻️ This comment has been updated with latest results.

[github-actions]

Unit Test Results (Linux)

187 tests  +8   187 ✅ +8   17s ⏱️ +2s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit f4f1336. ± Comparison against base commit cae2aef.

♻️ This comment has been updated with latest results.

[github-actions]

Unit Test Results (Windows)

137 tests  +2   137 ✅ +2   42s ⏱️ +3s
  4 suites ±0     0 💤 ±0 
  4 files   ±0     0 ❌ ±0 

Results for commit f4f1336. ± Comparison against base commit cae2aef.

♻️ This comment has been updated with latest results.

[resetius]

Independent review of head 3fddaf6. Verdict: the uring part works as described and I found no P1. There is one P2 (IOCP still frees buffers the kernel may write into) and a few P3s.

What I ran

• macOS (kqueue/select/poll): Debug build of 3fddaf6, ctest 4/4 pass.
• Linux (rpi5, aarch64, kernel 7.3, liburing 2.15, g++ 15), ASan build like CI: ctest 4/4 pass (ut_tests: 140 tests). I ran the two new tests 30 more times with no failures.
• Master cae2aef with this PR's tests/tests.cpp, under ASan: both new tests fail with heap-use-after-free in TPollerBase::WakeupReadyHandles (the frames were freed in ReadInto / AcceptInto). Reproduced.
• I traced which TUring::Cancel paths the tests reach (gdb dprintf at uring.cpp:97/103/108):
  • Socket ReadSome hits all four: not submitted yet, in the kernel, collected in ReadyEvents_, and in Reaped_.
  • Accept hits in the kernel and collected.
  • Connect hits not submitted yet.
  • The Connect timeout tests hit in the kernel (test_connection_timeout, result -ECANCELED) and collected (test_connection_timeout_cleanup). The exception type and code are unchanged.
• Extra repros (Linux, ASan), all pass on this head:
  • Destroy from inside WakeupReadyHandles. A resumed coroutine destroys one whose completion is later in the same batch, and one that was resumed earlier in the batch and is waiting again. Neither is resumed, and the next coroutine gets its own result. Master: heap-use-after-free.
  • CQ overflow during the drain. TUring(4) (8 CQEs), 32 completed reads, then I destroy Simplify includes #30 and Move poller base to separate file #2. Cancel reaps 31 completions, so the overflow is flushed. Move poller base to separate file #2 is then dropped from Reaped_, and the other 30 readers each get their own byte.
  • How long destroying a TFileHandle reader blocks:
    • pipe, FIFO, pty, tty, blocking unix socket: 0.00-0.03 ms;
    • a 256 MiB cold-cache regular-file read: 627 ms. The read runs in io-wq and cannot be cancelled. The wait is needed for the buffer and is documented on Cancel.
• Merge with Fix stale handles in poller #140 (0b81655): git merge-tree reports no conflicts. The merged tree passes on Linux with ASan (158 tests) and on macOS.
• uring bench (Release, bench -m uring -n 400 -a 40 -w 4000, 3 rounds each): p50 is 4190-4236 us on master and 4230-4256 us here, so no measurable difference. Results_ and Reaped_ keep their capacity across clear(), so there are no per-iteration allocations.
• CI is green on Linux, macOS and Windows. The Windows JUnit count goes from 135 to 137, so the TIOCp tests ran.
• Commit subject, PR body and code comments follow the repo rules: bare subject, no trailers, ASCII only, no issue refs in code.

P2

1. On IOCP, Cancel(THandle) lets the frame be freed while the kernel may still write into it (coroio/backends/iocp.cpp:174-180).
   • CancelIoEx only requests cancellation and does not wait. The operation can still complete normally; a regular-file read already sent to the disk usually does.
   • Windows requires the OVERLAPPED and the data buffer to stay valid until the completion is dequeued. The TIO is kept, but these buffers live in the frame that is freed right after Cancel returns:
     • the WSARecv/ReadFile buffer passed to ReadSome (usually in that frame);
     • the AcceptEx output buffer (addr inside the awaitable, coroio/socket.hpp:631).
   • So on IOCP the "kernel writes into the freed buffer" half of TUring does not cancel in-flight operations when the awaiting coroutine is destroyed #145 is still there, and Fixes #145 would close the issue.
   • Suggestion: wait the way TUring::Cancel does. After CancelIoEx, dequeue from the port until this TIO's packet arrives, and keep the other packets for the next Poll() (an equivalent of Reaped_). At minimum, wait until HasOverlappedIoCompleted(&tio->overlapped). Otherwise keep TUring does not cancel in-flight operations when the awaiting coroutine is destroyed #145 open for IOCP and say so in the Cancel(THandle) doc in iocp.hpp.
   • From reading and the Windows docs; I cannot reproduce it here.

P3

2. On uring, a reader or Accept destroyed before its SQE is submitted loses data or a connection (coroio/backends/uring.cpp:108-110). If the operation's SQE is still in the SQ, Cancel submits it together with the cancel:

   • a recv for data that has already arrived consumes it, and the result is dropped;
   • an Accept with a queued connection accepts it and closes it.

   Verified with this repro:

   • Read: the reader is suspended with its SQE not yet submitted, a byte arrives, the reader is destroyed, then a new reader starts. On epoll the new reader gets the byte; on uring it never does.
   • Accept: a connection is queued, an unsubmitted Accept is destroyed, then a new Accept starts. On epoll the next Accept gets the connection; on uring the client reads EOF.

   Suggestion: rewrite the not-yet-submitted SQE into a NOP with null user_data (scan sq.sqe_head..sqe_tail). That avoids both the loss and the syscall. A read that completed in the kernel before the cancel loses its bytes anyway, which is worth a sentence in the doc.

   repro

   TFuture<void> ReadOnce(TUring::TSocket& s, int* got) { char buf[16]; *got = co_await s.ReadSome(buf, sizeof(buf)); }
   
   TLoop<TUring> loop;
   int sp[2]; socketpair(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK, 0, sp);
   TUring::TSocket s(sp[0], loop.Poller());
   int got = -1, got2 = -1;
   std::optional<TFuture<void>> r{ReadOnce(s, &got)}; // SQE not submitted yet
   write(sp[1], "x", 1);
   r.reset();                                         // submits the recv with the cancel: it reads "x"
   auto next = ReadOnce(s, &got2);
   for (int i = 0; i < 5; i++) loop.Step();           // next.done() == false, got2 == -1 (TEPoll: 1)

3. The poller must outlive coroutines suspended on the driven awaitables, because the new destructors call into it (socket.hpp:605, 658, 712, 767, 869, 924).

   • Destroying such a future after the loop is now a use-after-free. Repro: a future declared before TLoop<TUring> gives ASan stack-use-after-scope in TUring::Cancel (uring.cpp:95). On master the same program is clean.
   • The contract is the same as in Fix stale handles in poller #140, but the TLoop note that Fix stale handles in poller #140 adds mentions only sockets. TPollerDrivenFileHandle awaitables now call into the poller too.
   • Verified.

4. TUring::Cancel's wait loop (uring.cpp:114-140) has two robustness gaps.

   • It never stops if the operation is no longer in the kernel. Example: a completion collected by a Poll() that is not followed by WakeupReadyHandles() (two Poll() calls in a row) makes the destructor hang; my repro was killed by a 10 s timeout. Stopping when the cancel's own CQE reports -ENOENT and the target CQE has not been seen would bound the loop. The hang is verified, though it needs that misuse to trigger.
   • Its throw (uring.cpp:116-117) runs inside noexcept destructors, so any io_uring_submit_and_wait error ends in std::terminate. From reading.

5. On IOCP, Cancel(THandle) walks the whole Pending_ list (iocp.cpp:174). That is O(pending operations) per cancellation, for example per timed-out read on a server with many idle connections. Linking and unlinking per operation is O(1), which is fine. From reading.

6. Test gaps:

   • No test covers TPollerDrivenFileHandle ReadSome/WriteSome, socket WriteSome, a destroy from inside WakeupReadyHandles, or CQ overflow during the drain. The last two pass in my repros.
   • "Destroyed while connecting" (tests/tests.cpp:1544) does not cancel a pending connect: on loopback the connect completes at submission (Cancel returned 0 in the trace).

Not from this PR (verified on master and on this head): on uring, TSocket::Close() closes the descriptor and only queues Cancel(fd) for the next Wait(). If a new socket reuses the descriptor number in the same iteration, its pending read is cancelled with "Operation canceled". I did not find an issue for this.

[resetius]

Independent review of head bc4ba24. Verdict: no P1 or P2 left. The four new commits fix my earlier findings (verified on uring; IOCP by reading plus Windows CI). Two new P3s below.

Status of the earlier findings

• P2 (IOCP did not wait). Fixed, from reading plus Windows CI.
  • Cancel(THandle) now waits for the TIO's own packet and keeps the other packets in Reaped_. Poll() delivers them first, without blocking.
  • The wait cannot miss its packet. The packet is dequeued once and then Complete() frees the TIO. TIOs are freed only there or on the immediate-failure paths, which post no packet. Since the ReadFile/WriteFile check is no longer inverted, an immediate success no longer frees a TIO that still has a packet queued.
  • Complete() is the old per-packet code moved as is, for AcceptEx, ConnectEx and recv/read, plus closing the socket of a cancelled AcceptEx.
  • When Cancel runs from ProcessTimers() or WakeupReadyHandles(), Poll() has already finished with Entries_. The packets it dequeues go to Reaped_, which Poll() cleared before its own dequeue.
  • Windows CI is green with 137 tests, the same set as before, including both TIOCp tests.
• P3 (data or connection lost before submit). Fixed and verified.
  • My repro now leaves the byte for the next reader and the connection for the next Accept, as epoll does.
  • The SQ scan starts at *khead, not sqe_head. So it also finds entries already published to the kernel but not consumed, e.g. those left by an SQE that failed to prep. Rewriting them is safe because the ring has no SQPOLL. With SQPOLL only sqe_head..sqe_tail could be touched; a comment saying so would help.
• P3 (poller lifetime). Documented on both classes. The repro still gives the expected ASan error once the future outlives the loop.
• P3 (wait loop could hang or throw). Fixed.
  • My two-Poll() repro now returns, because Wait() keeps undelivered completions.
  • Nothing throws from the destructor path any more.
  • My suggestion to stop on -ENOENT was wrong. I verified the author's point with raw liburing on ext4/NVMe, using a cold-cache buffered read and then ASYNC_CANCEL by user_data. The cancel CQE arrives first, at 0.04-0.34 ms, with -ENOENT. The read completes later and writes into the buffer: 4 KiB at 0.15 ms, 64 KiB at 0.26 ms, 1 MiB at 3.2 ms, 16 MiB at 37 ms.
• P3 (O(N) scan of Pending_). Kept. The justification in the PR body is fine.
• P3 (test gaps). Closed. The new tests cover file reader and writer, socket writer, same-batch destroys, a full CQ, a repeated Wait(), and a connect that really pends.

What I ran

• macOS: Debug, ctest 4/4 pass.
• Linux (rpi5, aarch64, kernel 7.3, liburing 2.15), ASan build like CI: ctest 4/4 pass, ut_tests 145 tests. The 7 new or changed tests ran 30 more times with no failures.
• Author's table, reproduced with this head's tests/tests.cpp:
  • on master all seven fail: six with heap-use-after-free, and after_repeated_wait on the Results_.empty() assert;
  • on 3fddaf6, before_submit and file_reader_and_writer fail (the byte is lost) and after_repeated_wait hangs;
  • on this head all pass.
• gdb trace of the Cancel paths in the new tests: ready, reaped, NOP (unsubmitted), and in the kernel (including the full-backlog connect) are all hit. The carry-over in Wait() is hit. The SQ-full branch (uring.cpp:125-131) is not.
• Repro for the SQ-full branch: TUring(4) with 12 submitted readers, 11 of them completed (so the CQ overflows), and the SQ filled with 4 more. I destroyed Bench res #11 (in the kernel) and Kqueue #13 (submitted by that Cancel): 0.12 ms in total. Both bytes stayed in their sockets, and all 14 live readers got their own byte.
• Earlier repros, rerun:
  • same-batch destroys: OK;
  • CQ overflow during the drain: OK, 30/30;
  • before-submit: fixed;
  • two Poll() calls: no longer hangs;
  • destroy times: pollable fds 0.01-0.03 ms, a 256 MiB cold regular-file read 630 ms, as before.
• Merge with Fix stale handles in poller #140 (0b81655): git merge-tree reports no conflicts. The merged tree passes on Linux with ASan (163 tests) and on macOS.
• uring bench (Release, bench -m uring -n 400 -a 40 -w 4000, 3 rounds each): p50 4153-4259 us on master and 4203-4238 us here, so no measurable difference.
• CI: Linux 186, macOS 146 and Windows 137 tests, all green.
• Commit subjects, PR body and code comments follow the repo rules.

P3

1. Raw Wait() + Result() without WakeupReadyHandles() now returns a stale result (coroio/backends/uring.cpp:179-184). The class example in uring.hpp:41-57 shows exactly this pattern.
   • An undelivered completion is carried over and placed first. Repro: read 1 byte with handle h1, Wait(), Result() gives 1. Then read 3 bytes with handle h2, Wait(), and Result() gives 1 again. On master and on 3fddaf6 it gives 3. Verified.
   • Nothing in the repo uses this pattern, but it is public API.
   • Suggestion: let Result() mark its entry as delivered, e.g. ReadyEvents_[NextResult_].Handle = {} before returning, so a consumed completion is not carried over. Alternatively, document the new behaviour on Wait() and fix the example.
2. The IOCP wait (iocp.cpp:189-205, INFINITE) assumes that every pending TIO gets a packet on this port. TPollerDrivenFileHandle never associates its handle with the port: its constructor (socket.hpp:858-861) does not call Register, unlike the socket constructors. So its completions never reach the port, and destroying a coroutine waiting on one would now block forever. Before, the coroutine was only leaked. File handles on IOCP are already broken (the operation never resumes; TIOCp::Read/Write always use offset 0 for regular files #156/IOCP: inverted ReadFile/WriteFile error check, completion status ignored, IPv4-only AcceptEx #158), so this changes the symptom more than the risk. Calling Register in that constructor would close it. From reading.

Nit, harmless: if the first Reap() in the cancel path already finds the target (a completion posted after the last Wait()), the prepared cancel SQE stays in the SQ and the next Wait() submits it. It can only match operations queued after it, so it finds nothing (-ENOENT, null user_data, ignored).

[resetius]

Independent review of head f4f1336. Verdict: no P1/P2, and no new findings. Both P3s from my previous comment are resolved.

uring Result() (d255fc1). Fixed, verified.

• My stale-result repro now gives 1, then 3. That matches master and the new test_uring_wait_result_without_wakeup.
• The carry-over still fixes the two-Poll() hang: my repro returns.
• I also mixed the two delivery paths in one batch: a raw completion carried over into a batch with two coroutine reads, Result() called by hand, then WakeupReadyHandles() and three more Wait()/WakeupReadyHandles() rounds.
  • Result() returned the raw completion.
  • WakeupReadyHandles() skipped it.
  • Each coroutine got its own result (2 and 3) and was resumed exactly once.
  • So there is no double delivery and no Results_ misalignment.

IOCP exit condition (f4f1336). Resolved, from reading plus Windows CI.

• The loop now ends when the operation's packet is dequeued or when HasOverlappedIoCompleted(&tio->overlapped) is true.
• Internal is the operation's I/O status block. The Win32/Winsock wrappers set it to STATUS_PENDING before issuing the operation, and the system writes the final status only after the data and the AcceptEx addresses are copied. That holds for WSARecv/WSASend, AcceptEx, ConnectEx and ReadFile/WriteFile. So once it is no longer pending, the frame's buffers are not touched again. This is the documented use of the macro after ERROR_IO_PENDING.
• Immediate synchronous success: Internal is already final, so Cancel returns without dequeuing. The packet is still queued, and Poll() later frees the TIO once in Complete() (there is no skip-on-success mode). No double free and no new leak.
• A packet that arrives after Cancel returned takes the !handle branch in Complete(). That branch only reads the TIO, closes the accept socket and frees the TIO. It never dereferences addr or len; GetAcceptExSockaddrs runs only for live handles.
• A handle not associated with the port now ends the wait through Internal instead of hanging. Its TIO stays in Pending_ with a null handle, which is a negligible leak for an operation that could never complete anyway.
• No busy spin: each iteration either blocks up to 10 ms or dequeues at least one packet. Reaped_ order is unchanged: every dequeued batch is fully processed before the condition is checked again.
• The SQPOLL comment at the SQ scan reads correctly.

What I ran

• Windows CI (MSYS2 MinGW) compiles HasOverlappedIoCompleted/STATUS_PENDING. All checks are green: Windows 137 tests, Linux 187, macOS 146.
• Linux (rpi5, aarch64, kernel 7.3, liburing 2.15), ASan build like CI: ctest 4/4 pass, ut_tests 146 tests. The uring and destroyed-waiter tests ran 15 more times with no failures.
• My earlier repros rerun on this head, all OK: same-batch destroys, CQ overflow during the drain, SQ full, before-submit, the two-Poll() case, raw Wait()/Result(), and the mixed Result()/WakeupReadyHandles() case above.
• macOS: ctest 4/4 pass.
• Merge with Fix stale handles in poller #140 (0b81655): git merge-tree reports no conflicts. The merged tree passes on Linux with ASan (164 tests) and on macOS.
• Commit subjects, PR body and code comments follow the repo rules.