PleaSQLarify

Code for PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying (CHI '26).

Interactive demo

Natural language interfaces to databases have to deal with ambiguous input. Rather than collapsing that ambiguity into a single query, PleaSQLarify treats it as something to resolve interactively: it samples an action space of candidate SQL queries, extracts interpretable decision variables that separate them, and lets the user narrow the space one decision at a time while the interface tracks how beliefs update across turns.

Layout

backend/          FastAPI service behind the interface
  app/src/        api.py (routes), main.py (entry point)
  eval/           offline evaluation, shared with the backend
frontend/         Next.js interface
sample-generation/  prompting setup used to sample candidate SQL

The evaluation lives under backend/ rather than at the top level because the backend imports the same helpers.py and spider/process_sql.py modules at runtime.

Setup

Python 3.12 for the backend, Node 22 for the frontend.

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Copy .env.example to .env if you intend to run the sample generation; the backend and the evaluation do not need any API keys.

Data

The evaluation and the backend both read AMBROSIA (CC BY 4.0), which is not redistributed here. Download it and place it so that the databases sit under backend/data/data/:

backend/data/data/
├── scope/
├── attachment/
├── vague/
├── ambrosia.csv
└── splits.csv

Paths inside 27154505_diverse_sql_output.jsonl are relative to that directory. The evaluation writes summary_<timestamp>.csv and full_logs_<timestamp>.jsonl next to itself; both are gitignored.

Running

Backend (from backend/):

python -m uvicorn app.src.main:app --host 0.0.0.0 --port 8000

It loads the first sample on startup and caches similarity matrices into similarity_cache/, which it creates on demand.

Frontend (from frontend/app/):

yarn install
NEXT_PUBLIC_API_BASE_URL=http://localhost:8000 yarn dev

Without that variable the interface points at the deployed backend. The backend's CORS list allows http://localhost:3000 and the deployed domains, so add your own origin there if you serve it elsewhere.

Both components have a Dockerfile; the backend image expects to be built from backend/ and the frontend from frontend/.

Evaluation (from backend/eval/):

python run_eval.py

Sample generation (from sample-generation/) holds the prompting setup used to sample candidate SQL from GPT-4o. The template in evaluation/prompt is filled with the schema and the question only, at temperature 0.7:

python evaluation_multiple.py --config_file=evaluation-args-openai-multiple.json

This is the only part that calls out to hosted models, and it reads credentials from .env.

The candidate set the evaluation actually runs on is provided directly as backend/eval/27154505_diverse_sql_output.jsonl, so the generation step does not need to be re-run.

Results

Notebook	Produces
backend/eval/plot_runs.ipynb	Figure 5, per-turn gold-label entropy and functional output similarity across the three ambiguity types

The notebook is checked in with its outputs, and so the figure can be read without re-running it. The output file that it calls is not provided, but the results are reproducible using the backend/eval/run_eval.py script.

Acknowledgements

backend/eval/spider/process_sql.py is adapted from Spider (Apache 2.0); the local copy adds span tracking on top of the original parser. The benchmark data is AMBROSIA.

Citation

@inproceedings{chan2026pleasqlarify,
  title     = {PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying},
  author    = {Chan, Robin Shing Moon and Sevastjanova, Rita and El-Assady, Mennatallah},
  booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
  year      = {2026}
}