Repository navigation
Conversation
VX_mem_unit.sv statically rejects two LMEM sizes: one that the LMEM base address is not a multiple of, and, with WGMMA, one above 64 KB, the most the 16-bit smem descriptor offset can address. SimX did not do this, so it built and ran configurations such as VX_CFG_LMEM_LOG_SIZE > 16. On xlen32 this caused silent failures. At LMEM_LOG_SIZE=17, CTA slots above 64 KB start at 0xFFFF0000 + 64 KB, which wraps to 0x0000xxxx. Those CTAs' shared-memory traffic lands on the console rings and the exit-code word. sgemm2 still reports PASSED but prints garbage and half of its LMEM requests leave LMEM. rtlsim correctly fails to run. SimX now mirrors all four VX_mem_unit LMEM asserts as static_asserts in the Core constructor so an invalid configuration fails to build.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SimX currently builds and runs with any
VX_CFG_LMEM_LOG_SIZEwhileVX_mem_unit.svstatically rejects two LMEM sizes: one that the LMEM base address is not aligned with, and, with WGMMA, one above 64 KB, the most the 16-bit smem descriptor offset can address.VX_MEM_LMEM_BASE_ADDR(0xFFFF0000 / 0x1FFFF0000) is a multiple of 64 KB but not of anything larger, soLMEM_LOG_SIZE > 16is an invalid configuration. rtlsim fails under this config at startup, but SimX builds and still runs, leading to silent errors.This PR mirrors those asserts in SimX, so an invalid configuration fails to build in both models.
Symptom
On xlen32 SimX with
LMEM_LOG_SIZE=17, the CTA dispatcher grants slots up to 128 KB but slot bases above 64 KB (0xFFFF0000 + 64 KB = 0x1_0000_0000) truncate to 0x0000xxxx, which is the IO window. Those CTAs' shared-memory traffic overwrites the console rings.Repro from a configured xlen32 build (rerun SimX with
LMEM_LOG_SIZE=16after):-c256gives 8 KB of LMEM per CTA. With 16 threads each CTA is one warp, so 8 CTAs are resident at 64 KB LMEM and 16 at 128 KB.[#N: lost <fp32 bits> bytes]console linesVX_mem_unit.sv:69: Assertion failed ... invalid parameterNote how SimX still reports a PASS at 17, because slots 8-15 wrap to the IO window which behaves like ordinary global memory. This explains the significant increase in cycles and the halving in LMEM reqs, as well as the console output.
Change
sim/simx/core.cpp:static_asserts mirroringVX_mem_unit.sv:59-69:TCU_WGMMA_ENABLE: LMEM is enabled, andLMEM_LOG_SIZE <= 16(descriptor offset field);
LMEM_ENABLE: the LMEM size is a multiple ofMEM_BLOCK_SIZE, andVX_MEM_LMEM_BASE_ADDRis aligned to the LMEM size.They are in the
Coreconstructor besideLocalMem::Create. SimX has no mem-unit object, so the core is where the RTL'sVX_mem_unitsub-blocks (LMEM, LMEM switch, LSU adapters, coalescers) are instantiated.Testing
xlen32,
build/, SimX:-DVX_CFG_LMEM_LOG_SIZE=17: fails to build withcore.cpp:133: static assertion failed: LMEM base address must be aligned to the LMEM size.-DVX_CFG_EXT_TCU_ENABLE -DVX_CFG_TCU_WGMMA_ENABLE -DVX_CFG_LMEM_LOG_SIZE=17: fails withcore.cpp:129: static assertion failed: LMEM exceeds the 16-bit smem descriptor offset field.-DVX_CFG_LMEM_LOG_SIZE=16: build.sgemm2passes as before.NUM_THREADS=32, which gives 64 KB, so no catalog configuration trips an assert.