Papers and engines across constrained decoding, organized by what is constrained, where enforcement acts, and what is guaranteed.
62 papers · 62 read in full · 2017–2026
Field snapshot · Papers · Libraries · Start here · Taxonomy · Filters · Contribute
Extends Awesome-LLM-Constrained-Decoding, which lists what exists. Here each work is additionally mapped by what it constrains, how the constraint is represented, where enforcement acts, what happens after rejection, what is guaranteed, and what is measured. The full catalog is in this README; the website is an optional power-user view. Updated 2026-08-01 — see CHANGELOG.
| What gets constrained | Papers | Where enforcement acts | Papers | |
|---|---|---|---|---|
| Syntax | 47 (76%) | Token | 48 (77%) | |
| Schema | 31 (50%) | Multi-stage | 25 (40%) | |
| Semantics | 24 (39%) | Completed sequence | 13 (21%) | |
| Lexical | 15 (24%) | Field / span | 5 (8%) | |
| Types | 5 (8%) | |||
| Relations across fields | 12 (19%) | What happens to a rejected token | ||
| Runtime environment state | 5 (8%) | Masked or pruned | 54 | |
| Policy / authorization | 1 (2%) | Resampled or regenerated | 11 | |
| Repaired in place | 1 |
Catalog signal: 47/62 works constrain syntax and 48/62 intervene token by token, while only 5 model runtime environment state, 12 cross-field relations, 1 policy or authorization, and 1 in-place repair.
Boundary: 62 papers have been checked against full text; the inventory also includes provisional discovery records. Counts describe this catalog, not field-wide prevalence.
Grammar and structured-output engines: XGrammar · llguidance · Outlines · Guidance · Formatron · SynCode · transformers-CFG · GreatGramma · llama.cpp grammars · LM Format Enforcer
Control runtimes: LMQL · SGLang · AICI · genlm-control · IterGen · Litelines · constrained-diffusion
Validation and retry: Instructor
Hosted: OpenAI Structured Outputs · Gemini structured output
Benchmark: JSONSchemaBench compares coverage, quality, and performance across structured-output engines.
| Paper | Why | |
|---|---|---|
| 2021 | PICARD · EMNLP | An influential template for incremental parsing during decoding. |
| 2024 | DOMINO · ICML | Tokenization and grammars do not align. Naive masking distorts the model. |
| 2024 | XGrammar | A widely deployed structured-generation engine with reusable grammar compilation. |
| 2024 | ASAp | Shows that masking is not grammar-conditioned sampling and targets the resulting bias. |
| 2025 | CRANE | Formalizes when restrictive grammars limit reasoning; interleaves free reasoning with constrained answers. |
| 2025 | AdapTrack | matrix_rank blocked → model emits matrix_power, which it rated 0.20%. |
| 2026 | The Constraint Tax | Measures when hard schemas improve validity while reducing executable correctness. |
| 2026 | Future Validity | Quantifies the distributional bias introduced by locally masking invalid next tokens. |
† checked against full text; unmarked discovery records have metadata and link verification only. Detailed evidence status remains available in the filters and JSON data.
Works addressing state-, relation-, or policy-dependent constraints. 16 of 62 records.
| Year | Work | Constrains | Enforced at | Represented as |
|---|---|---|---|---|
| 2026 | Decode-Time Grammars † | syntax, schema, type +2 | token, field, multi-stage | CFG, token index, oracle |
| 2026 | XGrammar-2 † | syntax, schema, state | token, field | CFG, PDA, token index |
| 2025 | ChopChop † | syntax, type, semantic +1 | token | program, CFG, solver |
| 2025 | CONFETTI † | schema, semantic, state | multi-stage | undisclosed |
| 2025 | Logically Constrained Decoding † · MathNLP 2025 | syntax, semantic, state | token | program, oracle |
| 2025 | Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo † · ICLR 2025 | syntax, semantic, relation | token, sequence, multi-stage | CFG, program, oracle |
| 2024 | COLLIE: Systematic Construction of Constrained Text Generation Tasks † · ICLR 2024 | lexical, syntax, semantic +1 | sequence | CFG, program |
| 2024 | FANTAstic SEquences and Where to Find Them † | schema, semantic, relation | token, field | trie, token index |
| 2024 | IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking † · ICLR 2025 | schema, semantic, relation | field, multi-stage | CFG, program |
| 2024 | Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access † · ACL 2024 | syntax, schema, relation | multi-stage | CFG |
| 2024 | We Need Structured Output † · CHI Extended Abstracts | syntax, schema, semantic +1 | multi-stage | varies |
| 2023 | Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning † · EMNLP | syntax, schema, semantic +1 | token | CFG |
| 2023 | Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context † · NeurIPS 2023 | type, semantic, state +1 | token, field | oracle, program |
| 2022 | Synchromesh † · ICLR 2022 | syntax, schema, type +2 | token | oracle, CFG, FSM +1 |
| 2021 | PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models † · EMNLP 2021 | syntax, semantic, relation | token | CFG, oracle |
| 2019 | A General-Purpose Algorithm for Constrained Sequential Inference † · CoNLL 2019 | syntax, relation | token | FSM |
All 62 records, newest first.
| Year | Work | Constrains | Enforced at | Represented as |
|---|---|---|---|---|
| 2026 | Accelerating Constrained Decoding with Token Space Compression † | syntax | token | CFG, PDA, token index |
| 2026 | Decode-Time Grammars † | syntax, schema, type +2 | token, field, multi-stage | CFG, token index, oracle |
| 2026 | Draft-Conditioned Constrained Decoding for Structured Generation in LLMs † | syntax, schema | token, multi-stage | FSM, CFG |
| 2026 | Future Validity is the Missing Statistic † | syntax, schema | token | CFG, FSM, trie |
| 2026 | Lost in Space † · GEM 2026 | syntax | token | CFG |
| 2026 | Mitigating Bias in Locally Constrained Decoding via Tractable Proposals † · ICML 2026 | lexical, syntax, schema | token, sequence, multi-stage | FSM, token index, program |
| 2026 | StructEval † · TMLR 2026 | syntax, schema | sequence | varies |
| 2026 | The Constraint Tax † | syntax, schema | token, multi-stage | varies |
| 2026 | Thinking Before Constraining † | syntax, schema | token, multi-stage | FSM |
| 2026 | XGrammar-2 † | syntax, schema, state | token, field | CFG, PDA, token index |
| 2025 | AdapTrack † | syntax, semantic | token, multi-stage | oracle, program |
| 2025 | ChopChop † | syntax, type, semantic +1 | token | program, CFG, solver |
| 2025 | CONFETTI † | schema, semantic, state | multi-stage | undisclosed |
| 2025 | Constrained Decoding of Diffusion LLMs with Context-Free Grammars † | syntax, schema | token, multi-stage | FSM, CFG |
| 2025 | Constrained Decoding with Speculative Lookaheads † · NAACL 2025 | lexical, semantic | token, multi-stage | classifier, program |
| 2025 | Constrained Discrete Diffusion † · NeurIPS 2025 | lexical, semantic | multi-stage | program, solver, classifier |
| 2025 | CRANE: Reasoning with constrained LLM generation † · ICML 2025 | syntax, semantic | token, multi-stage | CFG |
| 2025 | DINGO: Constrained Inference for Diffusion LLMs † · NeurIPS 2025 | syntax, schema | multi-stage | FSM, program |
| 2025 | Earley-Driven Dynamic Pruning for Efficient Structured Decoding † · ICML 2025 | syntax, schema | token | CFG, FSM |
| 2025 | Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Models † · AISTATS 2025 | lexical, schema | token, sequence | token index, trie |
| 2025 | Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling † · COLM | lexical, syntax, semantic | token, sequence, multi-stage | program, oracle |
| 2025 | Flexible and Efficient Grammar-Constrained Decoding † | syntax | token | CFG, FSM, token index |
| 2025 | Generating Structured Outputs from Language Models † | schema | token | varies |
| 2025 | Grammar-Constrained Decoding Makes Large Language Models Better Logical Parsers † · ACL 2025 Industry Track | syntax, semantic | token | CFG |
| 2025 | GRAMMAR-LLM † · Findings of ACL 2025 | syntax, schema | token | CFG |
| 2025 | Logically Constrained Decoding † · MathNLP 2025 | syntax, semantic, state | token | program, oracle |
| 2025 | Pre3: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation † · ACL 2025 | syntax, schema | token | PDA, token index |
| 2025 | Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo † · ICLR 2025 | syntax, semantic, relation | token, sequence, multi-stage | CFG, program, oracle |
| 2025 | Syntactic Control of Language Models by Posterior Inference † · Findings of ACL 2025 | syntax | token, sequence, multi-stage | classifier, program |
| 2025 | The Hidden Cost of Structure † · RANLP | syntax, schema | token | varies |
| 2025 | Type-Constrained Code Generation with Language Models † · PLDI 2025 | type, syntax, semantic | token | FSM, solver, oracle |
| 2025 | Why is constrained neural language generation particularly challenging? † · TMLR | lexical, syntax, semantic | multi-stage | varies |
| 2024 | Automata-based constraints for language model decoding † · COLM 2024 | syntax, schema | token | FSM, PDA |
| 2024 | COLLIE: Systematic Construction of Constrained Text Generation Tasks † · ICLR 2024 | lexical, syntax, semantic +1 | sequence | CFG, program |
| 2024 | Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars † | syntax | token | FSM, CFG |
| 2024 | FANTAstic SEquences and Where to Find Them † | schema, semantic, relation | token, field | trie, token index |
| 2024 | Grammar-Aligned Decoding † | syntax | token | CFG, trie, token index |
| 2024 | Guiding LLMs The Right Way † · ICML 2024 | syntax, schema | token | FSM, CFG, token index |
| 2024 | IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking † · ICLR 2025 | schema, semantic, relation | field, multi-stage | CFG, program |
| 2024 | Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models † · EMNLP 2024 Industry Track | syntax, schema | token, multi-stage | CFG, varies |
| 2024 | Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access † · ACL 2024 | syntax, schema, relation | multi-stage | CFG |
| 2024 | SynCode: LLM Generation with Grammar Augmentation † | syntax, schema | token | FSM, CFG, token index |
| 2024 | We Need Structured Output † · CHI Extended Abstracts | syntax, schema, semantic +1 | multi-stage | varies |
| 2024 | XGrammar † | syntax, schema | token | CFG, PDA, token index |
| 2023 | Don't Fine-Tune, Decode † | syntax, schema | token | FSM, token index |
| 2023 | Efficient Guided Generation for Large Language Models † | syntax, schema | token | FSM, CFG, PDA +1 |
| 2023 | Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning † · EMNLP | syntax, schema, semantic +1 | token | CFG |
| 2023 | Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context † · NeurIPS 2023 | type, semantic, state +1 | token, field | oracle, program |
| 2023 | Prompting Is Programming † · PLDI 2023 | lexical, syntax, semantic | token, multi-stage | program, FSM |
| 2023 | Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs † · PADL 2023 | syntax, semantic | token, sequence, multi-stage | program, trie |
| 2023 | SGLang: Efficient Execution of Structured Language Model Programs † | syntax, schema | token, multi-stage | FSM |
| 2023 | Tractable Control for Autoregressive Language Generation † · ICML 2023 | lexical | token | solver |
| 2022 | NeuroLogic A*esque Decoding † · NAACL 2022 | lexical, semantic | token, multi-stage | program |
| 2022 | Synchromesh † · ICLR 2022 | syntax, schema, type +2 | token | oracle, CFG, FSM +1 |
| 2022 | Validating Large Language Models with ReLM † | lexical | sequence | FSM |
| 2021 | Constrained Language Models Yield Few-Shot Semantic Parsers † · EMNLP 2021 | syntax, semantic | token, multi-stage | CFG |
| 2021 | NeuroLogic Decoding † · NAACL 2021 | lexical | sequence | solver, program |
| 2021 | PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models † · EMNLP 2021 | syntax, semantic, relation | token | CFG, oracle |
| 2019 | A General-Purpose Algorithm for Constrained Sequential Inference † · CoNLL 2019 | syntax, relation | token | FSM |
| 2019 | Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting † · NAACL 2019 | lexical | sequence | program, trie |
| 2018 | Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation † · NAACL 2018 | lexical | sequence | program |
| 2017 | Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search † · ACL 2017 | lexical | sequence | program |
Add a paper ·
fix a coding ·
rules. Edit data/papers.json, then python3 scripts/generate_views.py.
Most wanted: full-text coding for discovery records and corrections to existing annotations. Unmarked paper titles are the current full-text queue; a PDF is welcome when no HTML version exists.
Curated and broad, not a systematic census. The Atlas covers 2017–2026 across formal-language decoding, probabilistic inference, serving systems, benchmarks, and runtime enforcement. Snapshot counts describe this catalog, not the whole field. Methodology · JSON data · filterable site.
The README dagger shows whether full text has been checked; the exact review state remains available in the machine-readable data and filterable site.
Seeded from Awesome-LLM-Constrained-Decoding by Saibo Geng and contributors — go there for the broader list.
Code MIT · annotations CC BY 4.0 · no paper text redistributed.