llm-interpretability

A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.

A toolbox for exploring the informational nature of LLMs and language

See how sure an LLM is about each word.

Real-time 3D visualisation of SAE feature activations inside GPT-2, token by token

Fast XAI with interactions at large scale. SPEX can help you understand the output of your LLM, even if you have a long context!

[Under Review] Not All Tokens Are Equally Useful for Steering: Robust Directions and Prefix Steering

Automates attribution-graph analysis via probe prompting: circuit-trace a prompt, auto-generate concept probes, profile feature activations, cluster supernodes.

Turn a knob inside a small open model instead of writing a prompt. A reproducible RepE and CAA steering harness with an honest benchmark, including the failures.

Mechanistic Interpretability Research Knowledge Base + Notes

[ICLR 2026] AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features

A minimal mech-interp project for steering LLMs

Public research on LLM internals, Jacobian lenses, SAE steering, nonlinear dynamics, evaluation, and inspectable AI systems.

Replication of 'From Reasoning to Answer' (EMNLP 2025) — Reasoning-Focus Heads + Activation Patching on DeepSeek-R1-Distill-Qwen-7B

Ouro-1.4B SAE-derived loop-boundary causal pilot: matched-control interventions, FP16/FP32 robustness analysis, reproducible figures, and scientific audit.

Personal website of Xudong Zhu, featuring research, publications, open-source projects, and technical writing on LLM and representation learning.

Studying LLM latent space topology using persistent homology and Hodge decomposition. Discovers that Hodge gradient potential recovers the animacy hierarchy — a linguistic universal — without supervision.

Code for Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting, including both controlled synthetic Transformers and open-weight models experiments.

This project explores methods to detect and mitigate jailbreak behaviors in Large Language Models (LLMs). By analyzing activation patterns—particularly in deeper layers—we identify distinct differences between compliant and non-compliant responses to uncover a jailbreak "direction." Using this insight, we develop intervention strategies that modify

Probing whether multilingual LLMs encode kinship distinctions as language-specific directions or universal shared representations

Personal profile of Xudong Zhu. PhD researcher in representation learning.