Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Default owners: the BitPolito AI/ML team.
* @BitPolito/ai-ml

# Architecture decisions are approved by the lead maintainer.
/docs/adr/ @lucaosti
25 changes: 25 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
version: 2
updates:
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
timezone: "Europe/Rome"
commit-message:
prefix: "chore(deps)"
groups:
python-minor-patch:
update-types: ["minor", "patch"]

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
timezone: "Europe/Rome"
commit-message:
prefix: "chore(ci)"
groups:
actions-minor-patch:
update-types: ["minor", "patch"]
79 changes: 79 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: CI

on:
pull_request:
push:
branches: [main]

concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- run: uv sync --locked
- name: Ruff lint
run: uv run ruff check .
- name: Ruff format
run: uv run ruff format --check .

typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- run: uv sync --locked
- name: mypy
run: uv run mypy

import-contracts:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- run: uv sync --locked
- name: import-linter
run: uv run lint-imports

test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6
with:
enable-cache: true
- run: uv sync --locked
- name: pytest
run: uv run pytest
- name: Coverage summary
if: always()
run: uv run coverage report --format=markdown >> "$GITHUB_STEP_SUMMARY"

quality-gate:
if: always()
needs: [lint, typecheck, import-contracts, test]
runs-on: ubuntu-latest
steps:
- name: Fail unless every required job succeeded
if: >-
contains(needs.*.result, 'failure') ||
contains(needs.*.result, 'cancelled') ||
contains(needs.*.result, 'skipped')
run: |
echo "Required job results: ${{ join(needs.*.result, ', ') }}"
exit 1
- name: All required jobs succeeded
run: echo "quality-gate passed"
41 changes: 41 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Instructions for automated contributors

These rules apply to any coding agent that works in this repository. Humans should read `CONTRIBUTING.md`, which these rules condense.

## Before you write code

1. Read the issue completely: Goal, Scope, Out of scope, Files, Acceptance criteria, How to verify.
2. Read `docs/specs/2026-10-16-agentic-rag-design.md`, at least sections 4, 5 and 9, and every ADR the issue references in `docs/adr/`.
3. If the issue is ambiguous or contradicts the spec, comment on the issue and stop. Do not guess.

## While you work

- Change only what the issue's Scope covers. Do not refactor, rename or "improve" anything outside it.
- Respect the architecture (ADR-0001):
- core code depends only on `agentic_rag.types` and `agentic_rag.ports`;
- backends live in `agentic_rag.adapters`;
- dependencies are injected through constructors;
- no module-level singletons;
- no `os.getenv` or `os.environ` outside `EngineConfig`.
- Never change a port or a public contract without an accepted ADR.
- Write the tests first. Every acceptance criterion needs a test that proves it.
- All code, comments, docstrings and messages are written in English. Docstrings use the Google style.
- Do not add references to AI tools or assistants, or attribution trailers, in code, docs or commit messages.

## Before you open a pull request

Run every check, and the issue's own "How to verify" commands:

```bash
uv run ruff check . && uv run ruff format --check .
uv run mypy
uv run lint-imports
uv run pytest
```

Then:

- **Branch:** `<type>/<issue>-<short-description>`.
- **Commits:** Conventional Commits, `type(scope): summary`, with a `Refs #<issue>` footer.
- **Pull request:** against `main`, filling in the template, with `Closes #<issue>`. For retrieval changes, paste the before and after `arag-eval` SUMMARY.
- Never merge your own pull request. A human reviews and merges it.
140 changes: 140 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Contributing to Agentic-RAG

This guide covers everything you need to go from cloning the repository to getting a pull request merged. The design behind these rules is in `docs/specs/2026-10-16-agentic-rag-design.md`, and the architecture decisions are in `docs/adr/`.

All code, comments, documentation, issues, commit messages and pull requests are written in **English**.

## 1. Setup

You need [uv](https://docs.astral.sh/uv/). uv installs Python 3.11 for you, following `.python-version`.

```bash
git clone https://github.com/BitPolito/Agentic-RAG.git
cd Agentic-RAG
uv sync
uv run pytest
```

Always run project commands through `uv run`, so they use the project's virtual environment.

## 2. Quality checks

These are the same checks CI runs. Run them before you push:

```bash
uv run ruff check .
uv run ruff format --check .
uv run mypy
uv run lint-imports
uv run pytest
```

| Check | What it enforces |
|---|---|
| ruff | Lint and formatting, Python 3.11, line length 100, Google-style docstrings. Docstrings are not required in tests. |
| mypy | Strict typing on `packages/`. |
| import-linter | The architecture rules from ADR-0001: types and ports never import adapters; the engine never imports `agentic_rag_code`; engine modules respect the layer order. |
| pytest | Tests with coverage. The build fails below 80%. |

CI runs every check on each pull request. The `quality-gate` job aggregates them, and it is the only required status check.

## 3. Picking up work

1. **Find an issue.** Open the [project board](https://github.com/orgs/BitPolito/projects/23) and choose an issue in the **Ready** column.
- Issues labelled `good first issue` are a good starting point.
- Do not start an issue in **Backlog**. Its blockers are not closed yet.
2. **Claim it.** Assign the issue to yourself and move it to **In progress**.
3. **Read before coding.** Read the issue's Goal, Scope, Out of scope and Acceptance criteria. Read the spec sections and ADRs it references.
4. **Ask early.** If anything is unclear, ask in the issue. Do not guess.

### Definition of Ready

An issue is **Ready** only when all of these hold:

- its acceptance criteria are testable;
- every issue it is blocked by is closed;
- its size is `S`, `M` or `L`.

Work larger than `L` is turned into an epic with sub-issues.

## 4. Branches

- `main` is protected. Never commit to it directly.
- Create one short-lived branch per issue, named `<type>/<issue-number>-<short-description>`. For example: `feat/12-hybrid-retriever`, `fix/31-bm25-delete`.
- Branch types: `feat`, `fix`, `refactor`, `docs`, `test`, `chore`, `perf`, `research`.
- A branch is deleted after its pull request is merged.

## 5. Commits

Use [Conventional Commits](https://www.conventionalcommits.org/), in the form `type(scope): summary`.

- **Summary:** imperative mood, at most about 72 characters.
- **Body:** explains *why*, when the reason is not obvious.
- **Footer:** `Refs #<issue>`.

Allowed scopes: `engine`, `ingest`, `retrieval`, `evidence`, `generation`, `agent`, `adapters`, `eval`, `code-rag`, `dataset`, `bench`, `qvac`, `ci`, `docs`.

Example:

```
feat(retrieval): add reciprocal rank fusion

Weighted fusion is sensitive to score scales across backends; RRF is
scale-free and is needed for the QVAC and local stacks to be comparable.

Refs #18
```

## 6. Pull requests

1. **Open a draft early.** Open a pull request against `main` as soon as you have something to show. Keep it a draft until it is ready.
2. **Fill in the PR template.** It covers What, Why, `Closes #<issue>`, how you verified the change, and reviewer notes.
3. **Keep it small.** Aim for about 400 changed lines or fewer. Split anything larger.
4. **Include eval results for retrieval changes.** If the PR changes retrieval behaviour (ingestion, chunking, search, fusion, reranking or the evidence pack), paste the `arag-eval` SUMMARY block from before and after the change.
5. **Request review.** A review is requested automatically from `@BitPolito/ai-ml` (see `CODEOWNERS`). Move the issue to **In review**.
6. **Address feedback with new commits.** Do not force-push a branch that is under review.
7. **Merge.** A pull request is merged when `quality-gate` is green and it has at least one approval. Always merge with a merge commit, never squash or rebase, and delete the branch:

```bash
gh pr merge <number> --merge --delete-branch
```

8. **Close the issue.** `Closes #<issue>` in the PR closes the issue automatically. If the PR only covers part of the issue, use `Refs #<issue>` and comment on the issue with the progress.

## 7. Architecture Decision Records

An ADR in `docs/adr/` is required, before the implementation, for any change to:

- a port (`agentic_rag.ports`) or a public contract, such as `EvidencePack` or the QVAC HTTP contract;
- the repository layout or the core tooling;
- a technology choice that other code will depend on, such as the vector store or the code graph store.

Copy the format of an existing ADR. Open it with status **Proposed**. It becomes **Accepted** once its pull request is merged. The lead maintainer approves ADRs.

## 8. Porting code from BitPolito Academy

The engine is extracted from `BitPolito/bitcoin-academy` (ADR-0004). When you port code:

- Copy from the pinned commit named in the backlog (`b78d661`). Record the source in the PR as `Source: bitcoin-academy@<sha> <path>`.
- Port the matching tests together with the code.
- Remove domain-specific code while porting: course and Bitcoin vocabulary, book-specific rules, user-facing strings. Pass these in as configuration instead.
- Do not modify Academy. Its migration is milestone E6.

## 9. Labels

| Label | Meaning |
|---|---|
| `type/*` | Kind of change: feature, bug, refactor, docs, test, chore, research. |
| `area/*` | Part of the system: ingest, retrieval, adapters, eval, agent, code-rag, dataset, benchmark, qvac, infra. |
| `track/*` | Foundation (M0–M2), engine (E1–E6) or research (R1–R5). |
| `priority/p0`–`p2` | p0 blocks its milestone, p1 is needed for it, p2 is nice to have. |
| `size/S`, `size/M`, `size/L` | S is up to about half a day, M is one to three days, L is about a week. |
| `status/blocked`, `status/needs-design` | Waiting on something, or needs an ADR first. |
| `epic` | Tracks the sub-issues of a later milestone. |
| `agent-ready` | Precise enough to hand to a coding agent. A human still reviews the resulting PR. |
| `good first issue` | A good starting point for newcomers. |

## 10. Cadence

- **Weekly update:** each contributor posts a short async update on their issue or in the team channel, covering done, next and blocked.
- **Milestone review:** when a milestone's exit criteria are met, the team reviews it and the next epic is broken down into issues.
18 changes: 16 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,23 @@ Agentic-RAG is a local-first Retrieval-Augmented Generation engine built by the

Everything runs locally. The project does not use frontier or hosted commercial models.

## Status
## Quick start

The project is being set up. The design is described in `docs/specs/`, and architecture decisions are recorded in `docs/adr/`.
```bash
git clone https://github.com/BitPolito/Agentic-RAG.git
cd Agentic-RAG
uv sync
uv run pytest
```

## Documentation

- [Contributing guide](CONTRIBUTING.md): workflow, conventions and how to pick up work.
- [Architecture overview](docs/architecture.md): packages, layers and data flow.
- [Design specification](docs/specs/2026-10-16-agentic-rag-design.md): goals, architecture, milestones and conventions.
- [Architecture Decision Records](docs/adr/README.md).
- [Project board](https://github.com/orgs/BitPolito/projects/23).
- Instructions for automated contributors: [AGENTS.md](AGENTS.md).

## License

Expand Down
67 changes: 67 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Architecture overview

This page is a one-page map of the repository. The full design is in [`docs/specs/2026-10-16-agentic-rag-design.md`](specs/2026-10-16-agentic-rag-design.md), and the decisions are recorded in [`docs/adr/`](adr/README.md).

## Packages

| Path | Distribution | Import | Role |
|---|---|---|---|
| `packages/engine` | `agentic-rag` | `agentic_rag` | Domain-agnostic RAG engine: types, ports, pipelines, evidence pack, evaluation, adapters |
| `packages/code-rag` | `agentic-rag-code` | `agentic_rag_code` | AST-aware code retrieval for Python and TypeScript, built on the engine's ports |
| `services/qvac-worker` | none (Node 22) | none | QVAC SDK service used by the `qvac` adapter (ported in milestone E3) |
| `research/` | none | none | Benchmark runners, dataset builders, experiments and the technical report |

## Engine layers

Each layer may import only the layers below it. import-linter enforces this in CI.

```
agentic_rag.adapters backends behind optional extras (qvac, local, vector, lexical, sql, cache, memory)
agentic_rag.ingest | retrieval | generation | agent | eval
agentic_rag.evidence EvidencePack construction
agentic_rag.ports typing.Protocol interfaces
agentic_rag.types domain-neutral data model
```

`agentic_rag` never imports `agentic_rag_code`. The dependency only goes the other way.

## Data flow

**Ingestion**

```
file -> Parser -> Document(blocks) -> Cleaner -> Chunker -> (parents, children)
-> ParentStore.put_many(parents)
-> Embedder.embed(children) -> VectorStore.upsert
-> LexicalIndex.add(children)
-> IngestResult(stats, section_tree, warnings)
```

**Query**

```
query -> QueryTransform* -> {VectorStore.search, LexicalIndex.search} -> Fusion
-> Reranker -> MMR -> ParentStore expansion -> EvidencePackBuilder
-> (optional) LLM with strict citation policy -> Answer(evidence, citations)
```

## Ports

The ports are:

- `Embedder`
- `VectorStore`
- `LexicalIndex`
- `ParentStore`
- `Reranker`
- `LLM`
- `Cache`
- `TokenCounter`

Their exact signatures are fixed by ADR-0005 (milestone M1). Every port has an in-memory fake in `agentic_rag.adapters.memory`, and a conformance suite that every adapter must pass.

## Tracks

- **Foundation (M0–M2):** workspace, contracts and the evaluation harness.
- **Engine (E1–E6):** extract ingestion, retrieval, adapters and the agent runtime from BitPolito Academy, release the engine, then migrate Academy onto it.
- **Research (R1–R5):** code ingestion, code retrieval, the benchmark dataset, the agent and its Task Completion Rate harness, and the technical report.
Loading