Skip to content

[v13.0.3] HAProxy 3.4.5 crashes repeatedly in mux_quic.c when receiving QUIC via rpxy-l4 UDP/443 #5576

Closed

Description

@minchit

Environment

  • Hiddify Manager: 13.0.3
  • Ubuntu: 24.04
  • HAProxy: 3.4.5-0+ha34+ubuntu24.04u1
  • HAProxy built with QUIC support (USE_QUIC=1)

Problem

On a fresh Hiddify Manager 13.0.3 server, HAProxy repeatedly crashes
under normal production traffic.

Users experience intermittent connectivity, stalls and very slow
connections while this happens.

The HAProxy worker crashes with:

FATAL: bug condition "((&qcs->el_send)->n != (&qcs->el_send))" matched at src/mux_quic.c:2365

followed by:

Current worker exited with code 132 (Illegal instruction)

A worker process unexpectedly died and this can only be explained
by a bug in haproxy or its dependencies.

This occurred repeatedly, sometimes only several seconds apart.

Example:

Sep 28 16:39:09 haproxy[111572]: FATAL: bug condition "((&qcs->el_send)->n != (&qcs->el_send))" matched at src/mux_quic.c:2365
Sep 28 16:39:09 haproxy[111570]: Current worker (111572) exited with code 132 (Illegal instruction)
Sep 28 16:39:09 haproxy[111570]: A worker process unexpectedly died and this can only be explained by a bug in haproxy or its dependencies.

Sep 28 16:39:15 haproxy[111609]: FATAL: bug condition "((&qcs->el_send)->n != (&qcs->el_send))" matched at src/mux_quic.c:2365
Sep 28 16:39:16 haproxy[111607]: Current worker (111609) exited with code 132 (Illegal instruction)

Sep 28 16:39:22 haproxy[111619]: FATAL: bug condition "((&qcs->el_send)->n != (&qcs->el_send))" matched at src/mux_quic.c:2365
Sep 28 16:39:22 haproxy[111617]: Current worker (111619) exited with code 132 (Illegal instruction)

Generated QUIC configuration

The generated HAProxy configuration contains:

bind quic4@127.0.0.1:902 ssl crt /opt/hiddify-manager/data/ssl/ alpn h3

and:

http-after-response set-header Alt-Svc 'h3=":443"; ma=2592000'

The generated rust-rpxy-l4.toml contains:

udp_target = ["127.0.0.1:902"]
tcp_target = ["127.0.0.1:901"]

[protocols."quic_service"]
protocol = "quic"
target = ["127.0.0.1:902"]

Therefore the public QUIC path appears to be:

Internet UDP/443
      |
      v
   rpxy-l4
      |
      v
127.0.0.1:902
      |
      v
HAProxy HTTP/3/QUIC

Disabling QUIC in Hiddify did not disable this path

I disabled QUIC from:

Global Configs -> Modifying Proxies -> QUIC

and applied the configuration.

However, after Apply:

  • rpxy-l4 was still listening on UDP/443
  • HAProxy was still listening on UDP 127.0.0.1:902
  • udp_target still pointed to 127.0.0.1:902
  • HAProxy config still contained bind quic4@127.0.0.1:902 ... alpn h3
  • HAProxy continued crashing with the same mux_quic.c assertion

So the Hiddify global QUIC switch does not appear to disable this
HAProxy HTTP/3 path.

Workaround / isolation test

I then removed ONLY inbound UDP/443 from firewall.

TCP/443 remained open.

I did not remove the HAProxy :902 listener and did not modify the
generated Hiddify configuration.

Before blocking UDP/443, HAProxy was crashing repeatedly, including:

16:44:55
16:45:01
16:45:11
16:45:17
16:45:24
16:45:31
16:45:38

After blocking inbound UDP/443, I reset/restarted HAProxy:

sudo systemctl reset-failed hiddify-haproxy
sudo systemctl restart hiddify-haproxy

Repeated checks after the change show:

NRestarts=0
ActiveState=active
SubState=running

and:

sudo journalctl -u hiddify-haproxy \
  --since "2026-09-28 16:46:00" \
  --no-pager \
  | grep -cE 'FATAL:|unexpectedly died'

returns:

0

The crash loop therefore stops when public UDP/443 is prevented from
reaching the rpxy-l4 -> HAProxy QUIC path.

Expected behavior

  1. HAProxy should not crash when receiving QUIC traffic.
  2. Disabling QUIC in Hiddify should disable the corresponding public
    QUIC/HTTP3 path, or at least prevent rpxy-l4 from forwarding UDP/443
    to the HAProxy QUIC listener.
  3. A malformed/problematic QUIC connection should not repeatedly kill
    the HAProxy worker and disrupt unrelated connections.

Current workaround

For now:

  • Hiddify global QUIC: OFF
  • UDP/443: BLOCKED
  • TCP/443: OPEN

With this configuration HAProxy remains stable.

I have not manually modified generated haproxy.cfg or
rust-rpxy-l4.toml.

Activity

  1. added
    usefulTriaged: real bug or improvement worth keeping
    need_attentionImportant: broken current release, security or widespread impact
    on Oct 25, 2026
  2. hiddify-com commented on Oct 25, 2026

    @hiddify-com
    Collaborator

    @hiddify-com need attention: HAProxy 3.4.5 keeps crashing in mux_quic.c on QUIC traffic forwarded by rpxy-l4 on UDP/443, which drops users under normal production load. Needs a pinned/fixed HAProxy build or a way to keep QUIC away from HAProxy until upstream fixes it.

  3. hiddify-com commented on Oct 26, 2026

    @hiddify-com
    Collaborator

    This should be fixed in v14. If it still happens on the latest v14, please reopen this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    need_attentionImportant: broken current release, security or widespread impactusefulTriaged: real bug or improvement worth keeping

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions