Bandogram

Sevilla, A. F. G., & Lahoz-Bengoechea, J. M. (2026). Decomposing Sign Language Movements: A Multi-Band Visualization Method for Articulatory Analysis. In Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) (pp. 9559–9568). European Language Resources Association (ELRA). https://doi.org/10.63317/32sdurbs4fio.

Video hand tracking and multi-band visualization for sign language analysis.

Uses MediaPipe hand landmark detection on 2D video, decomposes the motion into arm displacement, hand rotation, and finger movement, and displays them as a time-aligned spectrogram-style plot (a bandogram). An interactive web viewer synchronizes the plot with the video for exploration.

Getting Started

Prerequisites: Python 3.12 with uv, Node.js (for viewer).

uv sync                              # Install Python deps
npm install                          # Install JS deps
uv run bandogram download            # Download MediaPipe model

Usage

Processing videos

# Place .mp4 files in data/input/, then:
make process                         # Process all of them at once

# Or one at a time:
uv run bandogram track video.mp4 -b  # Borderless plot (for viewer)
uv run bandogram track video.mp4     # Labeled plot (for standalone use)

Output goes to data/output/: annotated video (name.mp4) and bandogram plot (name_marginless.png).

Interactive Viewer

make serve                           # Build viewer + start server → http://localhost:8000

The left sidebar lists all processed videos, click one to load it. Videos in data/input/ that haven't been processed yet appear under Untracked with a track button. Clicking it runs the pipeline and auto-loads the result when done.

In the viewer: click the video to play/pause, click/drag on the bandogram to seek or set loop markers (red = start, blue = end). The video loops between markers automatically. Use the zoom button to align the video width with the plot.

Developer mode (live reload)

Run both servers simultaneously:

uv run bandogram serve --no-open     # API server on port 8000
npm run dev                          # Vite dev server on port 5173 → open this one

Vite proxies /api, /output, and /input to the Python server automatically.

CLI Options

uv run bandogram track <video> [options]
  --output, -o <path>    Output video path
  --borderless, -b       Generate borderless plot for viewer
  --window-size, -m <N>  Detection window size (default: 10)
  --lookahead, -n <N>    Lookahead frames (default: 6)

uv run bandogram serve [options]
  --port, -p <N>         Port to listen on (default: 8000)
  --no-open              Don't open browser automatically

Authors

Antonio F. G. Sevilla antonio@garciasevilla.com

José María Lahoz Bengoechea jmlahoz@ucm.es