UPDATES 🌐

AI-Powered Social Intelligence & Narrative Analytics Platform

Smart India Hackathon 2026 — Problem Statement SIH 26152: Social Media Analytics
Turning raw, noisy, multilingual public discourse into evidence-backed, explainable intelligence.

🔗 Live Deployment & Production Access

Component	Platform	Live URL	Status
Frontend Web App	Vercel	https://updates-intelligence.vercel.app (Replace with your deployed domain)	🟢 Active (Instant Edge Fallback)
Backend API Engine	Render	https://updates-backend.onrender.com (Replace with your Render web service URL)	🟢 Online / Healthy
Interactive API Docs	Swagger UI	https://updates-backend.onrender.com/docs	📖 Full OpenAPI 3.0 Specs
Health & Streaming Status	REST Probe	https://updates-backend.onrender.com/health	🩺 Database & Kafka Live Check

Zero-Downtime Architecture: The Next.js frontend implements hybrid API route handlers (frontend/app/api/[...path]/route.ts). If the upstream backend server is cold-starting or offline, the platform seamlessly serves high-fidelity cached and verified fallback topics (reservation-protest, maratha-reservation-protest-2026, tukaram-mundhe-fda-testing-surge), ensuring 100% demo uptime for presentations and evaluations.

📸 Visual Showcase & Platform Tour

1. Modern Discovery Homepage & Trending Intelligence

Features real-time breaking story carousels, category filtering (Protest, Laws, Education, Health & Safety), momentum baseline detection for emerging narratives, and the persistent Ask UPDATES assistant.

2. Comprehensive Topic Intelligence & Sentiment Dashboard

Drill down into any active narrative (e.g. /topic/reservation-protest). Displays overall sentiment distribution (55% Opposing, 27% Neutral, 18% Supportive), executive AI synthesis, demographic distributions (geography, language, probabilistic age brackets, interest groups, platforms), and primary conversation drivers.

3. Deep Conversation Analytics, Trends & Propagation Network

Visualizes timestamped volume acceleration, confidence ratings (qualified vs low-signal records), interactive citizen opinion contribution via MuRIL, topic-scoped AI queries, and community network propagation.

4. Regional Socio-Political Monitoring: Maratha Reservation 2026

Targeted tracking of regional public mobilization (Manoj Jarange's fast, Kunbi certificate validation, and judicial scrutiny) across Maharashtra with multi-platform sentiment breakdown.

5. Live Public Feed Stream

Real-time verified public discourse stream (/live) aggregating posts across X, Reddit, YouTube, and civic forums with instant stance indicators and sentiment signals.

6. Operations Control Room: Connectors & Multilingual Classification Lab

Live status of official connectors (X API v2, YouTube Data API v3, Reddit OAuth, Meta Graph API), Hugging Face model truth cards, automated ingestion loops, and the interactive MuRIL Multilingual Classification Lab.

7. Methodology, CSQE Engine & Responsible AI Governance

Transparent methodology documentation (/methodology) detailing the 4-stage pipeline, honest uncertainty principles, privacy-by-design standards, and ethical confidence boundaries.

8. Instant Multi-Entity Search

Rapid search querying stories, categories, and deep analytical topic dossiers simultaneously.

9. Persistent Settings & Dark Mode

Customizable user preferences including dark/light theme persistence and notification dispatch configurations.

💡 The Problem & Our Solution

The Challenge in Social Listening

Public social media discourse is inherently multilingual, repetitive, sarcastic, and fragmented across disparate platforms. Traditional social listening tools suffer from critical flaws:

Noisy Keyword Aggregation: Treating every one-word spam reply, bot post, or emoji flood as equal to substantive feedback.
Brittle Multilingual NLP: Failing to parse Romanised Hindi-English code-mixing (Hinglish) and regional nuances common across Indian social media.
Black-Box Causation: Asserting unfounded claims of who "started" a narrative rather than showing observed propagation networks.
Hallucinatory AI: Feeding unfiltered noise into LLMs resulting in generic, ungrounded conversational summaries.

The UPDATES Solution

UPDATES establishes an evidence-backed intelligence pipeline:

CSQE (Contextual Signal Qualification Engine): Dynamically evaluates signal density and filters spam before running expensive analytical models.
Context-Aware Multilingual NLP: Dual-layer architecture employing Google's MuRIL (airzipm/sentiment-analysis-muril-v2) and Indic abusive safety filters.
Probabilistic Demographic Scopes: Reports aggregate geographical, linguistic, and age brackets with explicit confidence badges; refuses to infer private PII (exact age, address, income).
Explainable Narrative Velocity: Classifies narratives as STABLE, RISING, FAST_RISING, or DECLINING based on timestamped volume acceleration.
Evidence-Grounded AI Analyst: Injects structured topic analytics and anonymized representative voices directly into synthesis prompts.

🏗️ System Architecture

End-to-End System Topology

flowchart TD
    subgraph DataSources["Public Platform Collectors"]
        X["𝕏 / Twitter API v2\n(Recent Search & Quotes)"]
        YT["YouTube Data API v3\n(Video Comments & Threads)"]
        RD["Reddit OAuth API\n(Targeted Subreddits)"]
        META["Meta Graph API\n(Facebook & Instagram Comments)"]
        GDELT["GDELT Global News\n(Live RSS / News Ingestion)"]
    end

    subgraph IngestionNormalization["Ingestion & Normalization Layer"]
        ADAPT["BaseCollector & Policy Gate\n(Rate-Limit, Quota & Token Auditing)"]
        NORM["Universal Schema Normalizer\n(UUID, Clean Text, Timestamp, Metadata)"]
        STREAM["Apache Kafka Event Bus\n(Optional Topic Streaming / Buffer)"]
    end

    subgraph IntelligenceEngine["Core Analytics & Intelligence Pipeline"]
        CSQE["CSQE Signal Qualification Engine\n(Noise, Repetition, Spam & Substance Scoring)"]
        MURIL["MuRIL Multilingual Pipeline\n(English · Hindi · Hinglish Stance & Sentiment)"]
        SAFETY["Indic Abusive Safety Classifier\n(Toxic abuse separation from policy debate)"]
        NX["NetworkX Interaction Graph\n(PageRank Centrality & Observed Amplifiers)"]
        TREND["Explainable Trend Service\n(Velocity & Volume Acceleration Windows)"]
    end

    subgraph StorageLayer["Data & Persistence Tier"]
        DB[(PostgreSQL / Supabase / SQLite\nTopics · Comments · Analytics · Briefs)]
    end

    subgraph ApplicationLayer["FastAPI Application Services"]
        API["FastAPI REST Application\n(/api/topics, /api/intelligence, /api/ai/ask)"]
        AI["Grounded AI Analyst\n(Deterministic Mock / Configurable LLM)"]
    end

    subgraph PresentationLayer["Frontend Presentation (Next.js 16)"]
        NEXT["Next.js 16 App Router\n(React 19 · TypeScript · Tailwind CSS 4)"]
        EDGE["Next.js Hybrid API Handler\n(Zero-Failure Proxy & Fallback Cache)"]
        UI["Interactive Intelligence UI\n(Donut Gauges · Recharts · Network · Chatbot)"]
    end

    DataSources --> ADAPT
    ADAPT --> NORM
    NORM --> STREAM
    STREAM --> CSQE
    NORM -.->|Direct Fallback| CSQE
    CSQE -->|High-Confidence Signals| MURIL
    CSQE -->|Safe Signals| SAFETY
    MURIL --> NX
    MURIL --> TREND
    SAFETY --> DB
    NX --> DB
    TREND --> DB
    CSQE --> DB
    DB <--> API
    API <--> AI
    API <--> EDGE
    EDGE <--> NEXT
    NEXT --> UI

Signal Qualification & Classification Pipeline (CSQE)

sequenceDiagram
    autonumber
    participant Raw as Raw Public Comment
    participant CSQE as CSQE Signal Engine
    participant Filter as Noise Filter
    participant MuRIL as MuRIL Indic NLP
    participant Store as Analytics DB

    Raw->>CSQE: Pass raw payload (text, timestamp, platform)
    Note over CSQE: Evaluate Semantic Substance:<br/>1. Length & word count check<br/>2. Emoji-only / Repetitive string test<br/>3. Topic relevance & keyword density
    alt Content is Low Signal (e.g. "BINOD", "first!", emoji spam)
        CSQE->>Filter: Score < 0.35 (Tag: LOW_SIGNAL)
        Filter->>Store: Store for audit, down-weight from aggregate sentiment
    else Content is High Signal (Constructive opinion / critique)
        CSQE->>MuRIL: Score >= 0.35 (Tag: HIGH_SIGNAL_EVIDENCE)
        Note over MuRIL: Multilingual Stance & Sentiment:<br/>- English, Devanagari Hindi, Romanised Hinglish<br/>- Stance: Supportive / Opposing / Neutral<br/>- Safety: Safe vs Abusive
        MuRIL->>Store: Index with confidence score & topic association
    end

⚡ Core Technical Innovations

1. CSQE (Contextual Signal Qualification Engine)

Located at backend/app/services/csqe.py, CSQE ensures analytical integrity by scoring incoming comments before sentiment analysis:

Heuristics Evaluated: Semantic density, repetition patterns (e.g., identical character repeats), emoji saturation, and topical relevance.
Non-Destructive Filtering: Low-signal records (e.g., viral copy-pasta like "BINOD" or single-emoji spam) are never discarded; instead, they are flagged and down-weighted so high-level executive sentiment reflects reasoned public sentiment.

2. Context-Aware Multilingual NLP (MuRIL)

Model: airzipm/sentiment-analysis-muril-v2 (Multilingual Representations for Indian Languages).
Safety Filter: Hate-speech-CNERG/indic-abusive-allInOne-MuRIL.
Tri-Lingual Support: Native English, Devanagari Hindi (हिंदी), and Romanised Hindi-English (Hinglish e.g., "yeh policy sahi hai aur students ko benefit milna chahiye").
Decoupled Classification: Emits independent labels for Sentiment (Positive/Neutral/Negative), Stance (Supportive/Opposing/Questioning), and Safety (Safe/Toxic).

3. Aggregate Audience & Ethical Inferences

In strict adherence to privacy-by-design standards:
- Geography: Macro-regional aggregations (e.g., Delhi NCR, Maharashtra) based strictly on explicit public post metadata.
- Age Brackets: Probabilistic brackets (e.g., 18–24 years) tagged with explicit confidence levels (Medium / Low).
- Exclusions: The platform explicitly refuses to infer exact age, personal address, income, or sensitive demographic attributes.

4. Graph Network & Narrative Spread

Leverages NetworkX PageRank and centrality metrics to map interaction nodes:
- Origin Communities: Earliest observed posting clusters.
- Amplifier Nodes: High-centrality dissemination hubs (news aggregators, key forums).
- Audience Nodes: Broader receiving public communities.
- Zero Causation Claims: Strictly reports observed interaction patterns without unverified claims of narrative origination.

5. Evidence-Grounded AI Analyst

Built with a provider protocol (backend/app/services/ai_analyst.py):
- Deterministic mock provider enabled by default (LLM_PROVIDER=mock) for zero-cost hackathon evaluation.
- Pluggable hosted LLM adapter ready for Gemini 1.5/2.0 or OpenAI endpoints.
- Grounded prompt templates inject verified representative voices and exact sentiment statistics to prevent hallucination.

🛠️ Technology Stack

Layer	Technology	Purpose
Frontend Framework	Next.js 16 (App Router)	Server Components, Turbopack, client routing, hybrid API handlers
UI Library	React 19 & TypeScript 5	Reactive state, type-safe data contracts, modern component architecture
Styling	Tailwind CSS v4	Clean, responsive design system with CSS custom properties & theme switching
Charts & Visuals	Recharts & Lucide React	Donut charts, volume acceleration area charts, network graphs, modern icons
Backend Framework	Python 3.12 & FastAPI	High-performance asynchronous REST API, OpenAPI 3.0 docs, Pydantic schemas
Database & ORM	SQLAlchemy & PostgreSQL	Production relational storage; SQLite development and demo fallback
NLP & ML	Hugging Face Transformers	Google MuRIL multilingual sentiment & Indic abusive content classifier
Graph Analytics	NetworkX & SciPy	PageRank centrality, interaction graph modeling, observed network flows
Message Streaming	Apache Kafka (Optional)	Scalable pub/sub event pipeline (`docker-compose.infrastructure.yml`)
Deployment	Vercel + Render	Frontend edge deployment paired with containerized Python web services

🚀 Deployment Guide

Option 1: Frontend on Vercel

The frontend is fully configured with frontend/vercel.json.

Push your code to your GitHub repository.
In the Vercel Dashboard, select Import Git Repository.
Configure the project:
- Framework Preset: Next.js
- Root Directory: frontend

Set Environment Variables:

NEXT_PUBLIC_API_URL=https://your-backend-service.onrender.com
ENABLE_BACKEND_PROXY=true

Click Deploy. Vercel will build and assign your production domain: https://<your-project>.vercel.app

Option 2: Backend on Render

The repository includes a ready-to-deploy render.yaml specification.

In the Render Dashboard, select New > Blueprint.

Connect your GitHub repository. Render reads render.yaml automatically:

services:
  - type: web
    name: updates-backend
    runtime: python
    rootDir: backend
    buildCommand: pip install -r requirements.txt
    startCommand: uvicorn app.main:app --host 0.0.0.0 --port $PORT
    healthCheckPath: /health
    envVars:
      - key: PYTHON_VERSION
        value: 3.12.0
      - key: DEMO_MODE
        value: "true"
      - key: CORS_ORIGINS
        value: "*"

(Optional) Add a managed PostgreSQL instance or supply your Supabase URL in DATABASE_URL.
Click Apply. Render deploys the web service at: https://updates-backend.onrender.com

Option 3: Supabase PostgreSQL Setup

To use managed cloud PostgreSQL instead of the local SQLite fallback:

Create a free project on Supabase.

Obtain your Transaction Pooler Connection String (Port 6543) or direct URI:

DATABASE_URL=postgresql://postgres:[PASSWORD]@db.[PROJECT-REF].supabase.co:6543/postgres?sslmode=require

Supply DATABASE_URL to your Render environment variables. The backend automatically initializes all required tables upon boot via app.services.database.init_database().

💻 Local Development Setup

Prerequisites

Node.js: v20.x or higher (tested on v24.x)
Python: 3.11 or 3.12
Git

1. Clone the Repository

git clone https://github.com/ManyaS-Git/Update.git
cd Update

2. Backend Setup (FastAPI)

cd backend

# Create and activate Python virtual environment
python -m venv .venv
# On Windows PowerShell:
.venv\Scripts\Activate.ps1
# On macOS/Linux:
# source .venv/bin/activate

# Install base requirements
pip install -r requirements.txt

# (Optional) Install real local MuRIL transformers runtime
pip install -r requirements-ml.txt

# Run the backend API server
uvicorn app.main:app --reload --port 8000

The API documentation will be available at http://localhost:8000/docs.

3. Frontend Setup (Next.js 16)

Open a new terminal window:

cd frontend

# Install dependencies
npm install

# Start the local development server
npm run dev

Open http://localhost:3000 in your browser.

4. Running the Test Suite

# Test frontend build and linting
cd frontend
npm run lint
npm run build

# Test backend services, CSQE, and API contracts
cd ../backend
pytest

📡 REST API Reference

The backend exposes a structured, typed REST interface. Key endpoints include:

Method	Endpoint	Description
`GET`	`/health`	System health check (database status, demo mode flag, Kafka connection).
`GET`	`/api/topics`	List all tracked conversation topics and active dossiers.
`GET`	`/api/topics/{slug}`	Fetch topic metadata, header banner, and conversation count.
`GET`	`/api/topics/{slug}/sentiment`	Retrieve sentiment breakdown (negative, neutral, positive, 6h trend).
`GET`	`/api/topics/{slug}/audience`	Aggregate demographic brackets (geography, language, age, interests).
`GET`	`/api/topics/{slug}/trends`	Timestamped volume points and sentiment fluctuations over time.
`GET`	`/api/topics/{slug}/drivers`	Leading discussion themes and emerging catalyst cards.
`GET`	`/api/topics/{slug}/voices`	Curated, representative public community quotes with stance badges.
`POST`	`/api/topics/{slug}/voices`	Submit a live public observation (classified via MuRIL in real-time).
`GET`	`/api/topics/{slug}/network`	Interaction graph nodes (origin, amplifier, audience) and weighted edges.
`GET`	`/api/topics/{slug}/confidence`	Qualified signal tally vs low-signal excluded items with level rating.
`GET`	`/api/topics/{slug}/brief`	Executive AI briefing summary and key narrative catalysts.
`GET`	`/api/emerging`	Rapid-detection snapshot of fast-rising topics before they peak.
`GET`	`/api/connectors`	Status and credential requirements for platform collectors.
`GET`	`/api/models/status`	Model cards and runtime state of MuRIL and safety classifiers.
`POST`	`/api/classify`	Test classification of single English, Hindi, or Hinglish text.
`POST`	`/api/classify/batch`	Batch classification endpoint for up to 500 public comments.
`POST`	`/api/ai/ask`	Evidence-grounded Q&A against topic analytics.
`POST`	`/api/chat`	Interactive assistant conversational query endpoint.

📁 Repository Structure

Update/
├── .env.example                     # Unified environment template
├── docker-compose.infrastructure.yml # Optional Kafka + PostgreSQL services
├── main.py                          # Root entrypoint for flexible cloud deployments
├── render.yaml                      # Render Blueprint Infrastructure-as-Code
├── requirements.txt                 # Root Python requirements redirect
│
├── frontend/                        # Next.js 16 Web Application
│   ├── app/                         # App Router pages and hybrid API routes
│   │   ├── api/[...path]/route.ts   # Edge route handler & offline demo fallback
│   │   ├── topic/[slug]/page.tsx    # Topic Intelligence dashboard
│   │   ├── live/page.tsx            # Live public signals feed
│   │   ├── sources/page.tsx         # Platform connectors & MuRIL lab
│   │   ├── methodology/page.tsx     # Methodology & CSQE documentation
│   │   ├── search/page.tsx          # Multi-entity search route
│   │   ├── settings/page.tsx        # Preferences & theme customization
│   │   └── globals.css              # Design system tokens and styles
│   ├── components/                  # Reusable UI component library
│   │   ├── TopicPage.tsx            # Full topic dossier layout
│   │   ├── TopicCharts.tsx          # Recharts volume & sentiment components
│   │   ├── NetworkPanel.tsx         # Interaction graph visualizer
│   │   ├── AIAnalyst.tsx            # Evidence-grounded AI query box
│   │   ├── Chatbot.tsx              # Floating Ask UPDATES assistant
│   │   └── SourcesPage.tsx          # Connectors management & test lab
│   ├── lib/
│   │   ├── api.ts                   # Universal fetch client with graceful fallbacks
│   │   └── demo-data.ts             # Typed high-fidelity seed datasets
│   ├── public/                      # Static media assets and story graphics
│   └── vercel.json                  # Vercel deployment specification
│
├── backend/                         # FastAPI Python Intelligence Engine
│   ├── app/
│   │   ├── api/routes/              # Modular API endpoints (topics, intelligence, content)
│   │   ├── collectors/              # X, Reddit, YouTube, Meta adapters
│   │   ├── core/                    # Security, rate-limiting, settings
│   │   ├── models/                  # SQLAlchemy ORM and Pydantic schemas
│   │   └── services/                # CSQE, MuRIL sentiment, NetworkX, Auto-ingestion
│   ├── requirements.txt             # Core Python runtime dependencies
│   ├── requirements-ml.txt          # Hugging Face Transformers & PyTorch runtime
│   └── tests/                       # Pytest test suite for CSQE, NLP, and API contracts
│
└── docs/                            # Architectural & Ethical Documentation
    ├── screenshots/                 # High-resolution platform captures
    ├── architecture.md              # Monorepo architecture specifications
    ├── methodology.md               # Scientific rationale for signal scoring
    ├── infrastructure.md            # Production deployment audit & Kafka gates
    └── api-limitations.md           # Platform policy compliance boundaries

🔒 Ethics, Privacy & Responsible AI

Strictly Public Data: UPDATES only consumes content made publicly accessible through official platform APIs.
No Profiling or PII: Personal identifying information (exact street addresses, real names, exact ages, financial status) is neither extracted nor stored.
Honest Uncertainty: Every analytical visual is paired with an explicit Confidence Rating (High / Medium / Low) and sample-size disclosures.
Sarcasm & Slang Sensitivity: Recognizes cultural idioms and linguistic sarcasm in Indian social discourse through context-aware classification rather than naive keyword matching.

🏆 Hackathon Presentation Checklist

Discovery Homepage: Live responsive news stories, trending tags, and category filters.
Topic Deep Dive: High-impact visualization on /topic/reservation-protest.
CSQE Qualification: Clear contrast demonstrated between low-value noise and high-confidence evidence.
Multilingual AI Lab: Interactive test bench on /sources parsing English, Hindi, and Hinglish comments.
Interactive Citizen Voice: Live submission form on topic pages immediately classified by the AI engine.
Grounded Assistant: Fully functional Ask UPDATES chatbot answering domain-specific inquiries.
Production Ready: One-click deployments defined for Vercel, Render, and PostgreSQL.

📄 License

This project is open-source and available under the MIT License.

Name		Name	Last commit message	Last commit date
backend		backend
docs		docs
frontend		frontend
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
docker-compose.infrastructure.yml		docker-compose.infrastructure.yml
main.py		main.py
render.yaml		render.yaml
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation