Finger Frame Live AI — frame a box with your fingers and step into a live, real-time AI world.
Hold up both hands and frame a box with your fingers — and see a live, real-time AI world inside the frame, generated at 30fps by Decart Lucy 2.5 (realtime video-to-video over WebRTC).
Unlike offline generation, the AI version moves with you: blink and it blinks, wave and it waves — with tens of milliseconds of model latency. Your hands become a window into a world that tracks your every move.
- Real-time AI inside your finger frame — the camera stream goes to Lucy 2.5 over WebRTC and a transformed version comes back live, drawn screen-aligned inside the quad you trace with your hands.
- Six live style effects (keys
1–6): 3D Movie, Anime, Cyberpunk, Watercolor, LEGO, and a Custom ✨ mode driven by your own prompt. - Live prompt switching — switching effects pushes a new style into the running session with no reconnect, so the world restyles mid-motion.
- Stable hand tracking — built on the original app's battle-tested pipeline: anatomical corner ordering, gesture hysteresis, teleport rejection, velocity-adaptive smoothing, and dropout hold.
- Bring your own key — paste a Decart key in the 🔑 panel; it stays in your browser and is used only for your WebRTC session.
- Keyless fallback — without a key, the window still shows a local color filter so the tracking stays demoable.
?demomode — a synthetic feed with fake hands lets you test tracking with no camera (and no AI).
| Input | Action |
|---|---|
| Both hands up, thumbs & index fingers out | Open the finger frame |
Keys 1–6 |
Switch live effects (3D Movie, Anime, Cyberpunk, Watercolor, LEGO, Custom ✨) |
🔑 button |
Open the key & custom-prompt panel |
This app is the third in a family that turns your hands into a picture frame.
| App | Generation | Latency |
|---|---|---|
| finger-frame-effect (repo) — live camera, local effects | Canvas 2D (Van Gogh, toon, glitch, …) | none |
| finger-frame-effect-ai (repo) — recorded video, AI restyle | Gemini Omni Flash (offline video edit) | minutes |
| this app — live camera, live AI | Decart Lucy 2.5 (realtime video-to-video) | ~real time |
Live capture — the AI world inside the frame moves in real time with the camera (full-quality mp4).
The webcam feed (1280×720@30) is mirrored to a full-screen canvas, and MediaPipe Hand Landmarker tracks the two hands. The finger-frame quad is derived with the pipeline from the original app:
- Anatomical corner ordering — corners are
[left.index, right.index, right.thumb, left.thumb], so the edge cycle is honest geometry: two upright "L"s trace a rectangle, and flipping one hand crosses the edges into a bowtie of two triangles (which uncrosses by itself). - Gesture hysteresis — once the frame is active it's easy to keep, so rotating or foreshortening your fingers doesn't drop it.
- Teleport rejection — a far-jumped quad must persist across frames before being accepted as a real reposition (not a mis-detect during hand overlap).
- Velocity-adaptive smoothing — dampens pixel jitter when nearly still, yet follows at high gain the moment the hands genuinely move.
- Dropout hold — crossing/overlapping hands break detection briefly; the last quad is held through a short window instead of fading out.
- Degenerate-frame gate — near-zero traced area (e.g. a legitimately crossed frame) is rejected, keeping the window well-formed.
The same camera stream is sent to Lucy 2.5 over WebRTC. The transformed stream returns live and is drawn mirrored and screen-aligned inside the quad — so the finger frame becomes a window into the AI world, staying perfectly registered as your hands move.
Each of the six effects is a live style prompt phrased per Decart's prompt
templates ("Change the style of the video to <description>." with concrete
visual specifics — vague phrasing degrades output). Switching an effect sends
the new prompt into the running session with no reconnect. Custom ✨ uses
your own prompt from the 🔑 panel.
Get a key at platform.decart.ai, then paste it into the 🔑 panel and hit Save & connect.
- The key stays in your browser (optionally remembered on the device) and is used only for the WebRTC session with Decart.
- Without a key, the window falls back to a local color filter so the tracking remains demoable.
python3 -m http.server 8125Then open http://localhost:8125 and allow camera access.
Demo mode — add ?demo to test tracking with a synthetic feed and fake
hands, no camera or AI needed:
# http://localhost:8125/?demoEverything runs client-side in your browser. Your camera feed and API key are never uploaded anywhere except the Decart WebRTC session that powers the live AI effect.
.
├── index.html # UI, toolbar, key panel, styling
├── main.js # hand tracking, Lucy WebRTC client, rendering loop
├── examples/
│ ├── lucy.gif # in-frame animation preview
│ └── lucy.mp4 # full-quality demo clip
└── README.md