An intelligent, web-based document search and information retrieval platform that leverages a Retrieval-Augmented Generation (RAG) architecture pattern to parse unstructured documents (.txt, .pdf) and retrieve contextually relevant answers with TF-IDF similarity scoring. Built as the Capstone Final Project for CS50x: Introduction to Computer Science at Harvard University.
In modern workflows, researchers and professionals frequently deal with lengthy articles, technical reports, and multi-page documents. Manually scanning hundreds of pages to locate specific facts or answers is time-consuming and inefficient.
DocQuery addresses this problem by providing a streamlined interface where users can:
- Upload text and PDF documents (
.txt,.pdf). - Automatically parse and index raw content into modular, overlapping text chunks.
- Pose natural-language queries to retrieve exact matching excerpts along with computed relevance scores.
Watch the full project presentation explaining the RAG architecture and web application workflow:
👉 Watch DocQuery Demo on YouTube
The project follows a modular Model-View-Controller (MVC) pattern powered by Flask and SQLite:
app.py: Main Flask application logic & routinghelpers.py: Core NLP, RAG chunking & TF-IDF algorithm logicassistant.db: Relational SQLite databaserequirements.txt: Python dependenciesuploads/: Staging folder for uploaded documentstemplates/: UI Layer (Jinja2 + Bootstrap 5)layout.html: Base navigation layoutindex.html: Search interface & result presentationupload.html: Drag-and-drop document upload interfacehistory.html: Search prompt history & logsapology.html: Custom error handling page
- Sliding Window Chunking (
split_into_chunks): Slices text into 200-word blocks with a 30-word overlap. Overlapping preserves semantic continuity across chunk boundaries so search keywords aren't lost at edges. - Text Normalization (
tokenize): Cleans raw input strings using Regular Expressions (re) and normalizes case for precise token matching. - TF-IDF Vectorization (
compute_tfidf): Evaluates Term Frequency against Inverse Document Frequency across all indexed chunks to rank and retrieve the most contextually relevant passage for any natural-language query.
GET / POST /(Index): Processes user search prompts, executes TF-IDF scoring against the SQLite chunk database, logs search history, and displays relevant results.GET / POST /upload: Manages file uploads, parses.pdffiles viapypdf, breaks text into chunks, and populates the database tables.GET /history: Displays historical queries, matching outputs, source document tags, and timestamps.
documents: Stores uploaded document metadata (id,filename,uploaded_at).chunks: Contains segmented text blocks linked via Foreign Key (doc_id).history: Records user query history, returned answers, source documents, and execution timestamps.
- Deterministic TF-IDF vs. Heavy ML Embeddings: TF-IDF was deliberately chosen over heavy PyTorch/Transformer neural models to guarantee fast, low-latency execution inside lightweight environments (like CS50 Codespaces) without needing external GPU resources or third-party paid API keys.
- Relational Normalization: Slicing documents into an explicit
chunkstable linked to parentdocumentsensures clean cascade operations and prevents orphaned text fragments when files are updated or removed.
- Python 3.9+
pippackage manager
git clone https://github.com/T2004-la/docquery.gitcd docquery
pip install -r requirements.txt
flask run
Open http://127.0.0.1:5000 in your web browser to start indexing and searching documents.
Developed by Tara Latifi as the capstone final project for CS50x: CS50's Introduction to Computer Science (Harvard University).
