[eyoon1131]

Summary

rtlsim aborts a stalled run after VX_DBG_STALL_TIMEOUT cycles (100k, scaled by the L2/L3 count) and says what stalled. SimX does not check this, so a deadlock in the model spins forever.

This PR mirrors both RTL watchdogs in the SimX core:

RTL	trigger	SimX
VX_scheduler.sv:568-603	every active warp stalled for STALL_TIMEOUT cycles	scheduler timeout: active_warps=…, then each active warp's PC and tmask
VX_scoreboard.sv:234-257	a staged instruction not issued for STALL_TIMEOUT cycles	issue timeout: wid=…, PC=…, tmask=…, cycles=…, inuse={…}, then the instruction

The messages carry the same information as RTL. There is a minor divergence since thread masks print little-endian in SimX (BitVector order) while RTL prints big-endian, so each message clarifies.

Both read VX_DBG_STALL_TIMEOUT from VX_config.h.

Symptom

An example is that a cluster that can't be co-resident deadlocks, which is noted in docs/designs/cta_dispatch_architecture.md §10.2. A one-line change to tests/regression/cta_cluster/main.cpp turns it into 2-warp CTAs, so a K = NUM_WARPS cluster needs twice the core's warps:

-    const uint32_t elems_per_cta  = (uint32_t)num_threads;
+    const uint32_t elems_per_cta  = (uint32_t)num_threads * 2;

CONFIGS="-DVX_CFG_EXT_DXA_ENABLE" ./ci/blackbox.sh --driver=simx --app=cta_cluster

• Before: no output and no progress. The host prints [VXDRV] Warning: CP has not retired a command in 10s and spins until killed.
• After (9 s):

  *** cluster0-socket0-core0 scheduler timeout: active_warps=1111 (big-endian), cycles=100000
      wid=0, PC=0x80000068, tmask=1111 (little-endian)
      wid=1, PC=0x800000a0, tmask=1111 (little-endian)
      wid=2, PC=0x80000068, tmask=1111 (little-endian)
      wid=3, PC=0x800000a0, tmask=1111 (little-endian)
  Aborted
  

• rtlsim, same build: VX_scheduler.sv:603: ... timeout: active_warps=1111, stalled_warps=1111. By default rtlsim prints only that line. Its per-warp detail requires EXT_SCHED_TIMEOUT_DUMP, which dumps the per-warp PCs and thread masks that SimX prints unconditionally.

Change

sim/simx/core.cpp:

• schedule(): counts consecutive cycles in which active_warps == stalled_warps (with at least one active warp) and aborts at the scheduler timeout, dumping every active warp. As in VX_scheduler.sv, the counter is armed by the first decode unlock, so cold-start fetch misses don't count. It resets when no warp is active or some warp is unstalled, and EXT_SCHED_STALL_TIMEOUT overrides the limit. Warps parked at a barrier are stalled so barrier deadlocks are caught.
• issue() (VX_scoreboard.sv): for each warp with a staged instruction, measures the wait from max(last_issue_, ibuf_time) and aborts at VX_DBG_STALL_TIMEOUT. It fires on any reason the instruction can't issue, not only the scoreboard.

The thresholds are the same as RTL, so anything that trips the SimX watchdog would also trip rtlsim's. Deliberate differences from RTL:

• The per-warp dump is always printed. The RTL prints it only under EXT_SCHED_TIMEOUT_DUMP.
• The issue check lists the busy registers and their producer uuids, where the RTL prints a per-operand busy mask. Both identify the same dependencies.
• The issue check prints the core-wide wid followed by (issue<iw>, w=<w>). w is the slice-local index the RTL prints as wid=, and iw is the slice that the RTL carries in its instance path (g_slices[iw]).
• Both checks are always compiled. The RTL's scoreboard check is SIMULATION-only.

Testing

xlen32, default 4x4 SimX:

• Deadlock caught, same content as rtlsim: the cta_cluster change above aborts as shown. Another non-co-resident cluster, 4 one-warp CTAs × 8 KB LMEM against a 16 KB LMEM, aborts with active_warps=0011. In both cases the masks, PCs and thread masks match rtlsim's -DEXT_SCHED_TIMEOUT_DUMP output.
• Full SimX catalog (pytest ci -m simx, 267 cases, xlen32): all 267 pass on the v3.0.1 toolchain with the Vulkan host packages installed, and the watchdog fires on none of them.
• No timing change: demo, sgemm, dotproduct, vecadd, sort, fence, diverge, sgemm2, conv3 and cta_cluster (DXA) report identical instruction and cycle counts with and without the change, and none trips the watchdog.
• Issue check, with scoreboard dependents: with -DVX_DBG_STALL_TIMEOUT=40 -DEXT_SCHED_STALL_TIMEOUT=100000000, sgemm aborts on its inner-loop FMA waiting on the previous one (rd and rs3 = f15):

  *** cluster0-socket0-core0 issue timeout: wid=2 (issue0, w=2), PC=0x800000f0, tmask=1111 (little-endian), cycles=40, inuse={f15 (#0), f15 (#0)}, cid=0, wid=2, cta_id=1, tmask=1111, PC=0x800000f0, wb=1, rd=f15, rs0=f10, rs1=f0, rs2=f15, ex=FPU (#0)
  

  rtlsim with the same flags reports the same kind of stall: VX_scoreboard.sv:257: Assertion failed in …g_scoreboard[0]: timeout: wid=0, PC=0x80000118, tmask=1111, cycles=40, inuse=0100 (#0).
  It trips on a different instruction because at a 40-cycle limit the first stall to cross it depends on cycle-level timing. uuids are only assigned in debug builds; release builds always set them to 0.