AI Document Assistant

Retrieval-Augmented Generation (RAG) for Document Question Answering

A modular Retrieval-Augmented Generation (RAG) application for document question answering built with FastAPI, Streamlit, FAISS, SQLite, OpenAI, and Ollama.

The application supports persistent document storage, semantic retrieval, local or cloud-based LLM inference, and source-aware answers through a simple web interface.

---

User Interface

Example

Question about the document

The assistant retrieves relevant document chunks and uses them as context to generate a source-aware answer.

Persistent Document Selection

Previously uploaded documents are persisted with SQLite and can be selected again after an application restart.

Question outside the document

When no sufficiently relevant context is retrieved, the application avoids generating an unsupported answer.

---

Architecture

                         Streamlit UI
                              │
                        HTTP Requests
                              │
                       FastAPI Backend
                              │
              ┌───────────────┴───────────────┐
              │                               │
        PDF Processing                   SQLite Storage
              │                      documents / pages / chunks
              │                               │
        Text Extraction                       │
              │                               │
           Chunking ◄─────────────────────────┘
              │
          Embeddings
              │
              ▼
       FAISS Vector Index
              │
       Semantic Retrieval
              │
       Relevant Context
              │
       Provider Factory
          ┌───┴────┐
          │        │
       Ollama    OpenAI
          │        │
          └───┬────┘
              ▼
     Answer + Source References

---

Key Features

• Retrieval-Augmented Generation (RAG) for PDF question answering
• PDF text extraction and automatic chunking
• Embedding-based semantic search
• FAISS vector indexing and similarity retrieval
• SQLite persistence for documents, pages, and chunks
• Selection and reuse of previously uploaded documents
• Automatic FAISS index rebuilding after application restart
• Local LLM inference with Ollama
• Optional OpenAI integration
• Modular LLM and embedding provider architecture
• FastAPI REST backend
• Streamlit web interface
• Source page and chunk references
• Automated tests with pytest

Current limitation

The current implementation supports machine-readable PDF documents. Support for scanned PDFs (OCR), Microsoft Word documents, plain text files, and additional data sources (e.g. databases or cloud storage) is planned for future versions.

---

LLM Providers

The application uses a provider abstraction that allows the LLM backend to be changed without modifying the API layer.

Current providers:

• Ollama – Local LLM inference without an external API.
• OpenAI – Cloud-based inference using the OpenAI Responses API.
• Mock Provider – Deterministic provider used for development and testing.

The active provider is configured through the LLM_PROVIDER environment variable.

---

Project structure

The project follows a layered architecture to separate the API layer, business logic, and LLM providers.

.
.
├── app/
│   ├── main.py
│   ├── models.py
│   ├── schemas.py
│   │
│   ├── providers/
│   │   ├── embeddings/
│   │   │   ├── base.py
│   │   │   ├── factory.py
│   │   │   └── ollama_provider.py
│   │   ├── base.py
│   │   ├── factory.py
│   │   ├── mock_provider.py
│   │   ├── ollama_provider.py
│   │   └── openai_provider.py
│   │
│   └── services/
│       ├── document_store.py
│       ├── faiss_vector_store.py
│       ├── in_memory_vector_store.py
│       ├── pdf_parser.py
│       ├── retrieval.py
│       ├── text_chunker.py
│       └── vector_store.py
│
├── ui/
│   └── streamlit_app.py
│
├── tests/
│   ├── test_api.py
│   ├── test_document_store.py
│   ├── test_embedding_service.py
│   ├── test_faiss_vector_store.py
│   ├── test_health.py
│   ├── test_in_memory_vector_store.py
│   ├── test_pdf_parser.py
│   ├── test_retrieval.py
│   └── test_text_chunker.py
│
├── requirements.txt
├── pytest.ini
├── .env.example
└── README.md

---

RAG Pipeline

Instead of sending the entire document to the language model, document chunks are converted into vector embeddings and indexed with FAISS. When a question is submitted, the most semantically similar chunks are retrieved and supplied to the LLM as context. This reduces unnecessary context, improves scalability, and helps ground answers in the uploaded document.

---

Document Persistence

SQLite persists document metadata, extracted pages, and text chunks, allowing previously uploaded documents to be reused across application restarts.

FAISS indexes remain in memory. If an existing document is selected after a restart, its chunks are loaded from SQLite, their embeddings are regenerated, and the FAISS index is rebuilt automatically.

---

Setup

1. Create a virtual environment

Windows

py -m venv .venv
.venv\Scripts\activate

macOS / Linux

python -m venv .venv
source .venv/bin/activate

2. Install dependencies

pip install -r requirements.txt

3. Configure environment variables

Copy .env.example to .env and replace the placeholder values with your own configuration.

Note

The OpenAI provider requires a valid OpenAI API key and an OpenAI Platform account with active billing or available API credits.

The Ollama Provider requires Ollama to be installed locally together with the selected model (e.g. gemma3:1b).

The .env file contains sensitive information and should never be committed to Git.

4. Run the API

python -m uvicorn app.main:app --reload

Open the interactive API documentation:

http://127.0.0.1:8000/docs

---

5. Launch the Streamlit interface

In a second terminal:

python -m streamlit run ui/streamlit_app.py

The application will be available at:

http://localhost:8501

---

Usage

1. Start the FastAPI backend and Streamlit interface.
2. Upload a PDF or select a previously stored document.
3. Ask a question about the document.
4. Review the generated answer and source references.

Previously uploaded documents remain available across application restarts through SQLite persistence.

Depending on the selected provider, the application generates answers using either:

- Ollama (local inference)
- OpenAI (cloud inference)
- Mock Provider (development and testing)

---

## Testing

The project includes automated tests using `pytest`.

Current test coverage includes:

- API endpoints
- Health endpoint
- PDF parser
- Document store
- Text chunking
- Embedding provider
- Semantic retrieval

Run all tests with:

```bash
python -m pytest -v

---

Next milestones

• Conversation memory
• OCR support for scanned PDFs
• Docker support
• Support for additional document formats