veir-sqlite-leadership

A cached corpus of sqlite3 as generic MLIR, one module per function and per defined global, and a leaderboard of what veir makes of it. The chunks are committed, so scoring veir needs no LLVM/MLIR toolchain -- only the veir checkout next door.

./update-sqlite.py            # regenerate the corpus if source or toolchain moved
./update-sqlite.py --check    # is the corpus stale? (exit 1 if so)
./make-leadership.py          # score the O3 corpus, write LEADERSHIP.md

Layout

sqlite3.c                        the pinned amalgamation (3.53.3, checksummed)
chunks/<corpus>/<board>/*.mlir   one self-contained module per symbol
manifest.json                    what produced the corpus, and its digests
LEADERSHIP.md                    the standings, regenerated by make-leadership.py
.cache/                          bitcode; regenerable, not committed

<corpus> is O0 (clang -O0 then sroa, matching vcc's pipeline) or O3. <board> is functions or globals: llvm-extract -func keeps only declarations of the globals a function touches, so an initializer is only ever visible on a board of its own. A file is named for its IR symbol, which is the same string on every host, so chunks/O0/functions/absFunc.mlir names the same item as a row in veir's Test/sqlite3/O0/functions-baseline.txt.

The leaderboard

make-leadership.py puts every O3 chunk through lake exe veir-opt, run in the neighbouring veir checkout so the standings track that working tree, and sorts each into supported / parsed (accepted only with --allow-unregistered-dialect) / failed / timed out.

The section that earns the file is What to implement next: every chunk that merely parsed is attributed to the first unregistered op, type or attribute veir-opt hit in it, ranked by how many chunks each is holding up.

Only O3 is scored -- it is the harder corpus and the one worth a leaderboard. O0 stays in the repository for anyone who wants to compare against it.

Keeping it current

The leadership workflow (.github/workflows/leadership.yml) polls veir's main every ten minutes. If it has moved since the commit named in the Provenance table, the workflow builds veir-opt at the new commit, rescores and commits LEADERSHIP.md as github-actions[bot], with the change in the standings in the subject and the blockers that cleared or appeared in the body. Nothing turns red: the leaderboard records, it does not gate. A push to main here rescores at once, and the workflow can be run by hand from the Actions tab, with force to rescore an unchanged veir.

The corpus is a function of its toolchain

Chunks are not portable between toolchains. The compiler decides what gets inlined, how generated symbols are numbered, and what target metadata rides along; the MLIR tools decide how all of that is spelled. manifest.json therefore records the exact version of every tool involved and the target triple, and update-sqlite.py rewrites the whole corpus when any of them moves rather than mixing two generations of chunk in one directory.

So: a corpus is comparable with another corpus built the same way, and with nothing else. Compare digests in manifest.json before comparing results.