bbackup

Back up Docker containers and host filesystems: encrypted, incremental, and agent-ready.

Quick start · Filesystem backup · Agent integration · CLI reference · Documentation

uv tool install --force git+https://github.com/CruxExperts/best-backup.git

uv handles the isolated tool environment automatically. This one command works for first install and redeploy/update from GitHub. See Installation if you need to install uv first, or for advanced system-wide /usr/local/bin deployment.

Why bbackup

Run bbackup backup and you get an interactive container picker, a live BTOP-style dashboard while the backup runs, and a finished artifact that can be verified, encrypted, and shipped to Google Drive, SFTP, or a local path. Point it at /srv/data and it backs that up too, with gitignore-style excludes. The companion bbman command handles setup, health checks, dependency installs, and self-updates so day-to-day maintenance stays out of the way.

Every command speaks structured JSON, making it compatible with AI agents out of the box: set two env vars, run bbackup skills, and drive the entire tool with --input-json.

Tip

Use --dry-run --output json before destructive restore work or scheduled backup changes. The JSON plan is designed for both humans and automation.

The flow collects Docker and filesystem inputs, writes a manifest, encrypts the artifacts, uploads them, and verifies the manifest before restore.

Backup flow

flowchart LR
    docker[Docker containers<br/>volumes • configs • networks]
    fs[Host filesystems<br/>paths • excludes]
    manifest[backup_manifest.json<br/>sizes • SHA-256 • item results]
    encrypt[Encryption<br/>AES-256-GCM or RSA-4096]
    upload[Remote upload<br/>local • SFTP • rclone]
    restore[Restore preflight<br/>manifest verification]

    docker --> manifest
    fs --> manifest
    manifest --> encrypt
    encrypt --> upload
    manifest --> restore

Features

Feature	Description
Rich TUI	BTOP-style live dashboard with real-time transfer metrics
Docker backup	Containers, volumes, networks, and configs in one shot
Filesystem backup	Back up any host path recursively with gitignore-style excludes
Incremental backups	rsync `--link-dest` so unchanged data is hardlinked, not copied
Encryption	AES-256-GCM (symmetric) or RSA-4096 (asymmetric) at rest
Remote storage	Google Drive via rclone, SFTP, or local directory
Rotation	Time-based daily/weekly/monthly retention with quota enforcement
Solid archive	Optional single tarball and optional whole-file encryption for one-file uploads
Full restore	Containers, volumes, networks, and filesystem paths; restore from a directory or solid archive file
Backup sets	Named groups of containers defined in config for repeatable runs
Agent-friendly CLI	JSON I/O, `--input-json`, `--dry-run`, and skill discovery on every command
Management CLI	`bbman` for setup, health, updates, cleanup, and diagnostics

Requirements

Python 3.12+
Docker Engine/daemon, with socket access for the account that runs bbackup
rsync (system package - used for volume and filesystem backups)
tar (system package - used for metadata and solid-archive handling)
rclone (optional, for Google Drive and other rclone remotes)

Warning

Access to the Docker socket, including membership in the docker group, is effectively root-equivalent control of the host. Grant it only to an account that is intentionally trusted to manage containers and mounted host paths.

Installation

uv tool install and uv sync serve different workflows. The recommended isolated tool install is for running bbackup; uv sync creates the environment for a checked-out project and is where project extras are selected.

Install or redeploy from GitHub

Run this when uv is already installed:

uv tool install --force git+https://github.com/CruxExperts/best-backup.git

It creates an isolated uv tool environment and links bbackup and bbman into the uv tool bin directory. The --force flag makes the same command safe for first install, repair, and redeploy/update.

If uv is not installed yet:

curl -LsSf https://astral.sh/uv/install.sh | sh
~/.local/bin/uv tool update-shell
~/.local/bin/uv tool install --force git+https://github.com/CruxExperts/best-backup.git

Open a new shell after uv tool update-shell so bbackup and bbman are on your PATH.

Project checkout and optional extras

For development or a checked-out source tree:

git clone https://github.com/CruxExperts/best-backup.git
cd best-backup
uv sync --locked
uv run bbackup --version

The gdrive-auth optional dependency is selected in this project environment, not by running uv sync after an isolated uv tool install:

uv sync --locked --extra gdrive-auth
uv run bbman auth-gdrive --client-secrets client_secret.json --dry-run --output json

Advanced: system-wide links

Most installs should use the uv tool command above. If /usr/local/bin/bbackup and /usr/local/bin/bbman must be shared by every user or a root-owned scheduler, see INSTALL.md.

For development setup, local source installs, and uninstall instructions, see INSTALL.md.

Quick start

After installation, confirm that Docker, rsync, and tar are available and that the account running bbackup is allowed to access Docker.

# Interactive first-time setup; this creates the starter config
bbman setup

# Check Docker, system tools, config, and any configured snapshot profiles
bbman health --output json
bbman validate-config --output json

Edit ~/.config/bbackup/config.yaml before the first run. The starter template leaves encryption disabled and enables a local remote under ~/backups/docker. It also contains an example snapshot_profiles block; configure that profile's restic repository, password file, and tools before expecting bbman health to report it healthy, or remove the unused example block.

# Inspect the planned scope without creating a backup
bbackup backup --backup-set production --dry-run --output json

# Run the configured backup (or use `bbackup backup` for the interactive picker)
bbackup backup --backup-set production --output json

See QUICKSTART.md for the complete first-run walk-through, remote storage, encryption setup, manifest check, and restore boundaries.

Integrity and upload safety

Every non-cancelled backup writes backup_manifest.json under the local staging backup directory. It records the requested scope, source paths, item results, errors, file sizes, and SHA-256 hashes. A real restore verifies that manifest before mutating Docker or filesystem targets and fails on missing, changed, unlisted, or backup-root-escaping files.

The backup is assembled first under backup.local_staging (the starter default is /tmp/bbackup_staging). Each selected or enabled remote is attempted independently. Local, SFTP, and rclone uploads write to a .partial destination and promote to the final backup name only after the copy succeeds. A remote failure is recorded in per-remote status and errors; inspect those entries and keep the local staging artifact until the outcome is understood. Encryption can replace plaintext staging with an encrypted artifact.

bbackup restore --dry-run is a target-selection plan only: it does not execute a restore or verify the manifest. Do not treat it as a data-integrity check. Use it before any real restore, and reserve an actual restore for an isolated, disposable destination when you need to exercise manifest verification.

Configuration

bbackup checks these locations in order:

~/.config/bbackup/config.yaml
~/.bbackup/config.yaml
/etc/bbackup/config.yaml
./config.yaml

A fully annotated template is in config.yaml.example. The minimal setup:

backup:
  local_staging: /tmp/bbackup_staging
  backup_sets:
    production:
      containers: [myapp, mydb, nginx]
      scope:
        volumes: true
        configs: true

remotes:
  local:
    enabled: true
    type: local
    path: ~/backups/docker

For rclone remotes you can optionally set rclone_options.transfers and rclone_options.checkers (or top-level rclone.default_options) to tune upload concurrency; both accept 1–32, default 8. See config.yaml.example and docs/architecture.md.

Google Drive rclone auth

Use bbman auth-gdrive to create a dedicated Google Drive rclone remote for bbackup. The helper uses a Google Desktop app OAuth client secrets file and configures rclone; backup uploads still run through rclone. The helper targets My Drive by default; shared-drive selection is not exposed by this command.

The optional gdrive-auth dependency is a project extra. For a checked-out project, install it with uv sync and run the command through that environment:

uv sync --locked --extra gdrive-auth
uv run bbman auth-gdrive --client-secrets client_secret.json --dry-run --output json
uv run bbman auth-gdrive --client-secrets client_secret.json --remote bbackup-gdrive

For the isolated GitHub tool install, do not run uv sync in another directory. Install the helper packages into the tool environment explicitly:

uv tool install --force \
  --with google-auth-oauthlib \
  --with oauthlib \
  --with requests-oauthlib \
  git+https://github.com/CruxExperts/best-backup.git
bbman auth-gdrive --client-secrets client_secret.json --dry-run --output json

For an SSH-hosted bbackup install, forward the fixed callback port from your workstation, run the command inside that SSH session, then open its printed URL in your workstation browser:

ssh -L 53682:127.0.0.1:53682 user@server
bbman auth-gdrive --client-secrets client_secret.json --no-open-browser --port 53682

Do not pass a raw client secret value on the command line; use the downloaded client_secret.json file.

Then reference the rclone remote in config.yaml:

remotes:
  gdrive:
    enabled: true
    type: rclone
    remote_name: bbackup-gdrive
    path: /backups/docker

Filesystem backup

Add a filesystem: section to back up arbitrary host paths:

filesystem:
  home-data:
    description: "Important home directory data"
    targets:
      - name: documents
        path: /home/user/Documents
        enabled: true
        excludes:
          - "*.tmp"
          - ".cache/"
          - "node_modules/"

Run a named filesystem set:

bbackup backup --filesystem-set home-data

Or pass paths directly, no config needed:

bbackup backup --paths /home/user/docs /srv/data --exclude "*.tmp"

Native restic snapshots

Add a top-level snapshot_profiles: section for encrypted deduplicating restic snapshots. bbackup owns discovery, safety checks, state, and scheduling while restic owns the snapshot repository.

snapshot_profiles:
  essentials-daily:
    engine: restic
    host_id: WORKSTATION01
    repository: rclone:my-drive:backups/WORKSTATION01/restic/essentials-daily
    # Prefer a dedicated Google Drive rclone OAuth client. Set this only as an
    # explicit YAML boolean true when accepting rclone's shared default client.
    allow_default_rclone_drive_client: false
    cache_dir: ~/.cache/bbackup/restic/WORKSTATION01/essentials-daily
    state_dir: ~/.local/state/bbackup/WORKSTATION01/essentials-daily
    password_file: ~/.local/share/bbackup-credentials/restic/WORKSTATION01/essentials-daily.password
    repo_homes:
      - ~/Projects
      - ~/Work
    explicit_repos:
      - /home/user/my-control-repo
    include_paths:
      - ~/.config/bbackup
      - ~/Documents
    retention:
      active_repo_daily: 14
      active_repo_weekly: 8
      active_repo_monthly: 12
      path_daily: 14
      path_weekly: 8
      path_monthly: 12
    schedule:
      daily_time: "03:30"
      maintenance_time: "Sun 04:30"
      verification_time: "monthly"
      verification_read_data_subset: "5%"

active_repo_* retention is applied only to the matching active Git repo ID. path_* retention is applied only to matching configured include-path IDs. The generated restic forget command filters by host_id plus stable bbackup/profile/scope/ID tags and groups only by host. Mutable user tags are excluded, as are retired repositories and unscoped profile-wide policies.

The password file must stay outside selected backup paths and should be 0600. Escrow the password outside the backup.

Google Drive rclone remotes should use a dedicated OAuth client_id. A profile may opt into rclone's shared default Drive client with allow_default_rclone_drive_client: true; quoted strings do not enable it. Create the dedicated remote with bbman auth-gdrive --client-secrets client_secret.json --remote my-drive before using a repository such as rclone:my-drive:....

Plan, initialize, run, check, restore, retire, purge-plan, and render schedule units with:

bbackup snapshot plan --profile essentials-daily --output json

bbackup snapshot init --profile essentials-daily --dry-run --output json

bbackup snapshot run --profile essentials-daily

Deleted Git repositories are marked retired only after at least one successful snapshot. Retirement is sticky: rediscovering the same repository does not reactivate it automatically. Automated retention targets active repo IDs and configured path scopes. Retired cleanup remains manual: run snapshot purge-plan, execute its dry-run restic forget command, inspect the result, and only then run the destructive command after explicit operator confirmation.

Rendered schedule units include a shared user-runtime flock lock, so daily backup, weekly non-destructive snapshot check, and monthly verification cannot overlap. Monthly verification defaults to 5% unless schedule.verification_read_data_subset is set.

CLI reference

bbackup commands

# Docker backup
bbackup backup                                         # Interactive backup with TUI
bbackup backup --backup-set production                 # Named backup set from config
bbackup backup --containers app db                     # Specific containers
bbackup backup --incremental                           # rsync --link-dest mode
bbackup backup --config-only                           # Skip volumes
bbackup backup --volumes-only                          # Skip configs
bbackup backup --no-networks                           # Skip network configs
bbackup backup --remote gdrive                         # Upload to specific remote

# Filesystem backup
bbackup backup --paths /home/user/docs /srv/data       # Back up specific paths
bbackup backup --paths /home/user/docs --exclude "*.tmp"
bbackup backup --filesystem-set home-data              # Named set from config

# Restore
bbackup restore --backup-path /path/to/backup --all
bbackup restore --backup-path /path --containers app --rename app:app_v2
bbackup restore --backup-path /path --filesystem documents \
  --filesystem-destination /home/user/docs

# Inspect
bbackup list-containers
bbackup list-backup-sets
bbackup list-filesystem-sets
bbackup list-backups
bbackup list-remote-backups --remote gdrive

# Setup
bbackup init-config
bbackup init-encryption --method asymmetric --algorithm rsa-4096

# Agent / non-interactive (available on every command)
bbackup list-containers --output json
bbackup backup --containers app --input-json '{"incremental":true}'
bbackup backup --containers app --dry-run --output json
bbackup skills                          # discover all capabilities
bbackup skills docker-backup            # step-by-step guide + JSON schemas

bbman commands

bbman setup                                          # First-time setup wizard
bbman health                                         # Docker, tools, config health check
bbman health --output json
bbman check-deps                                     # Check dependencies
bbman check-deps --install                           # Install missing packages
bbman validate-config                                # Parse and validate config
bbman status                                         # Backup history and totals
bbman status --output json
bbman cleanup                                        # Clean staging dirs and old logs
bbman cleanup --yes                                  # Skip confirmation (agent mode)
bbman diagnostics                                    # Generate diagnostic report
bbman diagnostics --report-file /tmp/report.txt      # Save to file
bbman check-updates                                  # Check for newer version
bbman update                                         # Self-update from repo
bbman update --yes                                   # Skip confirmation (agent mode)
bbman repo-url --url URL                             # Set the update source URL
bbman auth-gdrive --client-secrets client_secret.json # Configure Google Drive rclone auth
bbman run backup --containers app                    # Run bbackup through the wrapper
bbman skills                                         # Discover bbman capabilities
bbman skills maintenance                             # Step-by-step maintenance guide

bbman setup is intentionally interactive. The --no-interactive form is accepted only to report that setup was skipped and exits with an error; it is not a successful agent-mode setup path.

Agent integration

bbackup and bbman are natively compatible with AI agents. Every command supports structured JSON I/O, progressive skill discovery, and non-interactive execution without extra configuration.

Environment variables

Set these once and every subprocess inherits them:

export BBACKUP_OUTPUT=json           # all commands emit a JSON envelope
export BBACKUP_NO_INTERACTIVE=1      # no TUI, no prompts, no pagers

Variable	Effect
`BBACKUP_OUTPUT=json`	All commands emit JSON envelope without `--output json`
`BBACKUP_NO_INTERACTIVE=1`	Suppresses TUI, prompts, and pagers system-wide

Skill discovery (JSON and Markdown)

bbackup skills                           # level-0: all skill IDs + summaries (JSON)
bbackup skills docker-backup             # level-1: steps, schemas, examples (JSON)
bbackup skills --format markdown         # full Markdown skills catalog
bbman skills                             # level-0 skills for bbman (JSON)
bbman skills maintenance                 # maintenance skill (JSON)
bbman skills --format markdown           # full Markdown skills catalog

Level-0 JSON output:

{
  "cli": "bbackup",
  "version": "1.8.6",
  "agent_hint": "Set BBACKUP_OUTPUT=json and BBACKUP_NO_INTERACTIVE=1 for fully non-interactive use.",
  "skills": [
    {"id": "docker-backup",     "summary": "Back up Docker containers, volumes, networks, and configs.", "common": true},
    {"id": "filesystem-backup", "summary": "Back up arbitrary host filesystem paths with gitignore-style excludes.", "common": true},
    {"id": "restore",           "summary": "Restore containers, volumes, networks, or filesystem paths from a backup.", "common": true}
  ]
}

JSON envelope

Every command in JSON mode emits exactly this to stdout. All progress and diagnostic text goes to stderr.

{
  "schema_version": "1",
  "command": "backup",
  "success": true,
  "data": {},
  "errors": []
}

schema_version bumps only on breaking changes — additive fields are always safe.
errors is always present; non-empty means success: false.
A non-zero exit code always accompanies success: false.

`--input-json` parameter passing

Pass all parameters as a single flat JSON object. Keys use underscores (hyphens converted). The object merges over any CLI flags already provided.

bbackup restore \
  --input-json '{"backup_path":"/tmp/bbackup_staging/backup_YYYYMMDD_HHMMSS","containers":["myapp"],"dry_run":true}' \
  --output json

Unknown keys are silently ignored — forward-compatible by design.

Restore pre-flight is separate from backup planning:

bbackup restore \
  --backup-path /tmp/bbackup_staging/backup_YYYYMMDD_HHMMSS \
  --all --dry-run --output json

The restore dry-run reports selected targets and performs no restore, but it does not verify the manifest or prove destination permissions. A real restore verifies backup_manifest.json before mutation; restoring existing containers, volumes, networks, or filesystem destinations can be destructive.

Dry-run / pre-flight

bbackup backup --containers myapp --dry-run --output json

{
  "schema_version": "1",
  "command": "backup",
  "success": true,
  "data": {
    "dry_run": true,
    "would_backup": {
      "containers": ["myapp"],
      "filesystem_targets": [],
      "remotes": [],
      "incremental": false,
      "scope": {"volumes": true, "configs": true, "networks": true}
    }
  },
  "errors": []
}

Exit codes

Code	Meaning
`0`	Fully successful
`1`	Bad argument, missing param, or invalid `--input-json`
`2`	Config not found or fails validation
`3`	Docker unreachable, rsync/rclone missing, or key generation failed
`4`	Partial: some items succeeded, some failed
`5`	Operation cancelled by user or agent

For agent workflows, set BBACKUP_OUTPUT=json and BBACKUP_NO_INTERACTIVE=1 globally, then use bbackup skills to discover what's available before issuing commands.

Encryption

Two modes are available:

The starter configuration leaves encryption.enabled: false, and the setup wizard's optional key step defaults to no. First-run artifacts are therefore plaintext (the manifest records encryption_mode: "disabled") until you generate keys and add the returned config snippet with enabled: true.

Symmetric — AES-256-GCM: One key encrypts and decrypts. Good for single-server setups.

bbackup init-encryption --method symmetric

Asymmetric — RSA-4096: Public key encrypts, private key decrypts. Better for multi-server setups where backup and restore run on separate machines.

bbackup init-encryption --method asymmetric --algorithm rsa-4096

The public key can live on GitHub:

encryption:
  enabled: true
  method: asymmetric
  asymmetric:
    public_key: github:YOUR_USERNAME/gist:YOUR_GIST_ID
    private_key: ~/.config/bbackup/backup_private.pem

Full details in docs/encryption.md.

TUI keyboard controls

Key	Action
`Q`	Quit / cancel backup
`P`	Pause / resume
`S`	Skip current item
`H`	Help

Project structure

best-backup/
├── bbackup/
│   ├── cli.py                # bbackup CLI entry point
│   ├── cli_utils.py          # JSON envelope, exit codes, shared decorators
│   ├── skills.py             # Skill descriptors for agent discovery
│   ├── config.py             # Config loading and all dataclasses
│   ├── docker_backup.py      # Docker backup via temp Alpine containers
│   ├── filesystem_backup.py  # Host filesystem backup via rsync
│   ├── backup_runner.py      # Backup workflow orchestration
│   ├── restore.py            # Restore operations
│   ├── tui.py                # Rich TUI and BackupStatus tracking
│   ├── remote.py             # Remote storage (local / rclone / SFTP)
│   ├── rotation.py           # Retention policies and quota cleanup
│   ├── encryption.py         # AES-256-GCM + RSA encryption
│   ├── logging.py            # Rotating file logger
│   ├── bbman_entry.py        # Console script shim for bbman
│   └── management/           # bbman subpackage (11 modules)
├── bbackup.py                # bbackup entry point
├── bbman.py                  # bbman entry point
├── config.yaml.example       # Annotated config template
├── pyproject.toml
└── uv.lock

Documentation

Doc	Description
QUICKSTART.md	Setup to first backup in 5 minutes
INSTALL.md	All installation methods
docs/management.md	Full `bbman` reference
docs/encryption.md	Encryption setup and key management
CHANGELOG.md	Release history
CONTRIBUTING.md	How to contribute
SECURITY.md	How to report vulnerabilities
SUPPORT.md	Where to ask questions or get help
docs/cli-skills.md	Unified CLI skills catalog for humans and AI agents
docs/VERSIONING.md	Version source of truth, hook setup, and release validation
docs/PUBLISHING_CHECKLIST.md	GitHub publishing and release readiness checklist
docs/standards/github-markdown/github-markdown-writing-standard.md	Governing standard for GitHub-facing Markdown
docs/standards/github-markdown/github-markdown-capabilities-reference.md	Supported GitHub Markdown syntax and context limits
docs/standards/github-markdown/github-markdown-review-checklist.md	Review gate for new and revised Markdown
docs/standards/github-markdown/github-markdown-source-provenance.md	Official source register and adaptation record

Roadmap

Shipped

Planned

Credits

Built with Rich, Click, and docker-py.

License

MIT

Navigation Menu

Repository files navigation

bbackup

Why bbackup

Backup flow

Features

Requirements

Installation

Install or redeploy from GitHub

Project checkout and optional extras

Advanced: system-wide links

Quick start

Integrity and upload safety

Configuration

Google Drive rclone auth

Filesystem backup

Native restic snapshots

CLI reference

Agent integration

Environment variables

Skill discovery (JSON and Markdown)

JSON envelope

`--input-json` parameter passing

Dry-run / pre-flight

Exit codes

Encryption

TUI keyboard controls

Project structure

Documentation

Roadmap

Credits

License

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
.agent/queue		.agent/queue
.githooks		.githooks
.github		.github
bbackup		bbackup
docs		docs
scripts		scripts
tests		tests
.gitignore		.gitignore
.python-version		.python-version
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile.test		Dockerfile.test
INSTALL.md		INSTALL.md
LICENSE		LICENSE
QUICKSTART.md		QUICKSTART.md
README.md		README.md
SECURITY.md		SECURITY.md
SUPPORT.md		SUPPORT.md
VERSION		VERSION
agent-repo-shape.json		agent-repo-shape.json
bbackup.py		bbackup.py
bbman.py		bbman.py
config.yaml.example		config.yaml.example
external_skills.lock.json		external_skills.lock.json
project.yaml		project.yaml
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation

bbackup

Why bbackup

Backup flow

Features

Requirements

Installation

Install or redeploy from GitHub

Project checkout and optional extras

Advanced: system-wide links

Quick start

Integrity and upload safety

Configuration

Google Drive rclone auth

Filesystem backup

Native restic snapshots

CLI reference

Agent integration

Environment variables

Skill discovery (JSON and Markdown)

JSON envelope

--input-json parameter passing

Dry-run / pre-flight

Exit codes

Encryption

TUI keyboard controls

Project structure

Documentation

Roadmap

Credits

License

About

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

`--input-json` parameter passing