Creator Engine is an open-source, multi-tenant SaaS platform that automates the end-to-end production of short-form 9:16 social media videos (30–90 seconds) from text topics.
The system orchestrates AI-generated decisions across specialized provider adapters (LLM, TTS, Asset Retrieval, and FFmpeg Rendering) to produce complete, captioned MP4 videos.
Creator Engine separates AI decision-making from deterministic code execution:
- Orchestrator: Manages state lifecycle and job pipelines.
- Provider Adapters: Loose coupling via provider interfaces (
LLMProvider,TTSProvider,AssetProvider,Renderer,Publisher). No vendor lock-in. - Graceful Degradation: Core pipelines function even when optional external services (such as TTS or stock video APIs) are unavailable, utilizing robust fallbacks (e.g., silent audio generation and colored gradient backgrounds).
- Script Generation: LLM generates structured narration and timing based on topic input.
- Storyboard Generation: LLM produces validated JSON scene breakdowns with visual asset directives.
- Text-to-Speech (TTS): Synthesizes scene audio (with silent audio fallback).
- B-roll Retrieval: Searches and downloads portrait stock footage (Pexels API integration with fallback).
- FFmpeg Rendering: Composes scenes, burns in styled captions, synchronizes audio, and concatenates segments into an optimized 1080x1920 portrait MP4.
- Framework: Next.js 16 (App Router, React 19, TypeScript)
- Styling: Tailwind CSS v4, Shadcn/UI components (
@base-ui/react,lucide-react) - Video Processing: FFmpeg (system binary with explicit font configuration and robust filter sanitization)
- AI / LLM: OpenRouter API & local Omniroute integration
- Assets: Pexels Stock Video API
├── docs/ # Architecture specs and milestone summaries
│ ├── architecture.md
│ ├── M2-COMPLETE.md
│ └── M3-COMPLETE.md
├── public/
│ └── exports/ # Rendered MP4 output files
├── src/
│ ├── app/ # Next.js App Router (API routes & dashboard UI)
│ ├── components/ # React dashboard components & UI primitives
│ └── lib/
│ ├── pipeline/ # Orchestrator and job queue state management
│ ├── prompts/ # LLM prompt templates for script & storyboard
│ ├── providers/ # Modular provider adapters (LLM, TTS, Assets, Renderer)
│ ├── store/ # In-memory project and tenant state stores
│ └── validation/ # Storyboard JSON validation and parsing
├── test-full-pipeline.js # End-to-end integration test script
└── PIPELINE_TEST_REPORT.md# Detailed test results and verification logs
- Node.js: v18+ (Node 20+ recommended)
- FFmpeg: Must be installed and available on your system
PATH(required for video composition and audio generation).
Clone the repository and install dependencies:
git clone https://github.com/biha-pixel/Content-engine.git
cd Content-engine
npm installCopy .env.example to .env.local and configure your API keys:
cp .env.example .env.localExample .env.local:
# LLM Provider Configuration ("omniroute" or "openrouter")
LLM_PROVIDER=omniroute
OMNIROUTE_API_KEY=your-omniroute-api-key
OMNIROUTE_BASE_URL=http://localhost:20128
OMNIROUTE_MODEL=default
# Alternative OpenRouter API
OPENROUTER_API_KEY=sk-or-v1-your-key-here
OPENROUTER_MODEL=mistralai/mistral-nemo
# Pexels API (required for B-roll video search)
PEXELS_API_KEY=your-pexels-api-key-here
# Kokoro TTS (optional; falls back to silent audio if unavailable)
KOKORO_BASE_URL=http://localhost:8880/v1npm run devOpen http://localhost:3000 in your browser to access the interactive dashboard.
- Start Development Server:
npm run dev - Build for Production:
npm run build - Lint Code:
npm run lint - Run End-to-End Pipeline Test:
node test-full-pipeline.js
- Storage: Currently utilizes in-memory state stores (Maps) for projects and jobs suitable for MVP evaluation. PostgreSQL persistence with RLS is planned for post-MVP.
- Concurrency: Pipeline executes synchronously within API request context (generation takes ~30–90 seconds depending on video length and scene count).
- Publishing: Direct social media API publishing is out of scope for MVP; videos are downloaded locally as MP4 files from
public/exports/.
- ✅ M1: Scaffolding & Architecture — Project structure, core interfaces, tenant auth shell.
- ✅ M2: Infrastructure — LLM integration (OpenRouter/Omniroute), script/storyboard generation, pipeline orchestration, REST API.
- ✅ M3: Asset Integration — Kokoro TTS, Pexels B-roll retrieval, FFmpeg video rendering with captions, dashboard video player modal.
- 🔄 M4: Polish & Production Readiness — Quality control flow, persistent database storage, enhanced error recovery, and retry mechanisms.
Contributions, issues, and feature requests are welcome. Please ensure that all TypeScript checks (npm run build) and linter rules pass before submitting pull requests.
This project is licensed under the MIT License.