Mneme

The Personal Search Engine for your Digital Brain

---

📖 Introduction

Mneme (/ˈniːmiː/; Greek: Μνήμη) (named after the Greek muse of memory) is a powerful, local-first search engine designed to index and retrieve your personal documents, code, and notes with lightning speed. Unlike diverse grep tools, Mneme builds a persistent, optimized index of your content, allowing for instant search results with relevance ranking.

Whether you have thousands of markdown notes, a vast library of PDF documents, or a sprawl of code repositories, Mneme helps you recall information instantly.

✨ Features

• 🚀 Blazing Fast Indexing: Utilizes LSM-tree inspired batch processing to handle large datasets with minimal memory footprint.
• 🧠 Smart Ranking: Implements BM25 and Vector Space Model (VSM) algorithms to surface the most relevant results, not just keyword matches.
• 🔮 Fuzzy Search: Tolerates typos and misspellings using Trigram Indexing + Levenshtein Distance — so "kuberntes" still finds "kubernetes".
• 🔍 Content Awareness:
  • Automatically detects and skips binary files (images, videos, executables).
  • Supports pluggable ingestors for future expansion (e.g., Google Drive, GitHub).
• 📝 Rich Snippets: Generates context-aware snippets with accurate highlighting of search terms.
• 🛡️ Safe Storage: includes a "Tombstone" mechanism to safely handle deletions and updates without immediate data loss.
• ⚙️ highly Configurable: Customize everything from indexing paths to ranking algorithms via a simple TOML configuration.

📦 Installation

Using Go

Ensure you have Go 1.21+ installed.

go install github.com/ayanrocks/mneme@latest

Using Pre-built Binaries

If you don't have Go installed, download a pre-built binary from the GitHub Releases page.

1. Download the binary for your platform:

   Platform	Binary
   Linux (x86_64)	mneme-linux-amd64
   macOS (Intel)	mneme-darwin-amd64
   macOS (Apple Silicon)	mneme-darwin-arm64
   Windows (x86_64)	mneme-windows-amd64.exe

2. Make it executable and move it to your PATH (macOS/Linux):

   chmod +x mneme-<os>-<arch>
   sudo mv mneme-<os>-<arch> /usr/local/bin/mneme

   On Windows, move the .exe to a directory in your PATH.

   Alternatively, run the binary directly from where you downloaded it without adding it to your PATH by opening a new terminal in the same folder and then:

   ./mneme-<os>-<arch> version

Verify

mneme version

🚀 Quick Start

1. Initialize Mneme: Set up the configuration and data directories.

   mneme init

2. Configure: Add the directories you want to index.

   mneme config add /Users/username/docs

3. Index Your Data: Build the search index.

   mneme index

4. Search: Find what you need instantly.

   mneme find "project architecture"

⚙️ Configuration

Start with the default configuration generated by mneme init. Customize it in ~/.config/mneme/mneme.toml.

[sources]
# Lists of directories to index recursively
paths = ['/path/to/docs', '/path/to/code']
# Specific extensions to include (empty = all text files)
include_extensions = ['.md', '.txt', '.go', '.py']
# Specific extensions to exclude
exclude_extensions = ['.log', '.tmp']
# Folders to ignore
ignore = ['.git', 'node_modules', 'vendor']

[index]
# Skip binary files like images/videos (recommended: true)
skip_binary_files = true

[search]
# Number of results to return
default_limit = 20

[ranking]
# Customize ranking weights
bm25_k1 = 1.2
bm25_b = 0.75

📂 Data Storage

Mneme stores its index and metadata in ~/.local/share/mneme.

• segments/: Contains the active search index segments.
• tombstones/: Holds old index files that have been replaced but not yet permanently deleted.
• meta/: Stores metadata about the index state.

Run mneme clean to clear out the tombstones directory and reclaim disk space.

🛠️ Usage

mneme config

Manage configuration.

• Commands:
  • show: Display current configuration.
  • add <path>: Add a directory to index.
  • remove <path> [-a|--all]: Remove a directory from index.

mneme index

Crawls your configured paths and builds/updates the search index.

• Flags:
  • -v, --verbose: Show detailed progress.
  • -q, --quiet: Only show errors.

mneme find <query>

Searches the index for the given query.

Fuzzy search — automatically catches typos and near-matches:

mneme find "deplyment"           # still finds "deployment"
mneme find "kuberntes config"    # still finds "kubernetes" + "config"

Phrase search — wrap words in quotes to match an exact phrase:

mneme find "aws region"          # matches the exact phrase "aws region"

Mixed search — combine phrases and individual words:

mneme find "error handling" go   # matches phrase "error handling" AND word "go"

Individual words — unquoted words are matched independently:

mneme find deploy production     # matches documents with "deploy" or "production"

mneme clean

Manages the storage engine.

• Usage: mneme clean helps recover space by removing old index segments and tombstones.

🧪 Testing

Mneme includes a comprehensive test suite.

# Run all tests
go test ./...

# Run tests with coverage
go test -race -coverprofile=coverage.out -covermode=atomic ./...

For more details on the testing infrastructure, benchmarks, and CI/CD pipeline, please refer to TESTING.md.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository.
2. Create your feature branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push to the branch (git push origin feature/AmazingFeature).
5. Open a Pull Request.

📄 License

Distributed under the MIT License. See LICENSE for more information.