Malwarehouse is a warehouse for your malware. Malwarehouse is a useful command line utility for storing, tagging, storing, and searching for malware. This is intended to help analyst manage their workflow by conducting basic triage and making it easy to look up past samples.

Requirements

• Python 3.9+

Installation

Install with pip (libmagic must be installed on the system first, e.g. apt-get install libmagic1 or brew install libmagic):

pip install .            # installs the `malwarehouse` command (also runnable as `python -m malwarehouse`)
pip install .[yara]      # with optional YARA rule matching
pip install -e .[dev]    # editable install with test tooling, for development

Released versions have a wheel attached to their GitHub release (pip install malwarehouse-<version>-py3-none-any.whl). malwarehouse --version shows the installed version, and CHANGELOG.md lists what changed and the steps to upgrade an existing install.

For development, uv sync --extra dev builds .venv from the committed uv.lock, the same versions CI tests; uv run pytest then runs the tests (see uv).

Configuration

malwarehouse init writes a starter malwarehouse.cfg to your user config directory (~/.config/malwarehouse/ on Linux, ~/Library/Application Support/malwarehouse/ on macOS). With it, samples and the index live in your user data directory, and malwarehouse works from any directory. Then create the database with malwarehouse init-db.

Malwarehouse uses the first configuration it finds:

1. --config PATH (anywhere on the command line)
2. the MALWAREHOUSE_CONFIG environment variable
3. malwarehouse.cfg in your user config directory
4. ./malwarehouse.cfg in the directory you run it from

Relative paths in the file (basedir, yararules and the SQLite database) are relative to the file's own directory. The whole file is checked when a command starts, so a missing setting or a mistyped plugin: On/Off switch (any case, or yes/no) is reported straight away. malwarehouse init --config PATH writes a starter config somewhere else, keeping the data next to it; --force replaces an existing file.

Libraries

• SQLAlchemy - Database ORM
• Typer - Command line interface
• python-magic - File type identification (requires libmagic)
• ppdeep - Fuzzy hashing
• vt-py (optional) - VirusTotal API v3 client for the virustotal plugin
• pefile (optional) - PE parsing for the pe_analyzer plugin
• yara-python (optional) - YARA rule matching

Authors

• Scott J Roberts - @sroberts
• Mina Naguib - @warebot

Setup - Databases

Malwarehouse keeps its index in a SQLite database file, set with [database] uri in malwarehouse.cfg (e.g. sqlite:///data/malwarehouse.db, relative to the config file, or sqlite:////absolute/path/malwarehouse.db). Other database servers are not supported.

For initial DB setup, run malwarehouse init-db. It creates the file and its directory, is safe to re-run, exits 1 on failure, and shows the migrations and SQL it runs with -v. malwarehouse-setup-db, its older name, still works.

Upgrading

The schema is managed with Alembic migrations. After upgrading malwarehouse, run malwarehouse init-db again: it upgrades the existing database in place and keeps its samples. Until then, load, find, recent, similar, reindex, delete, tag, note and source stop with an error asking you to run it. A database created before migrations were added is recognised and upgraded the same way. The upgrade runs as one SQLite transaction, so a failure leaves the database as it was; copying the database file first is still a good idea.

Usage

Every command has --help (or -h). Add -v/--verbose for progress output or -q/--quiet for errors only, anywhere on the line. Exit status is 0 on success, 1 on an error and 2 for bad arguments.

Basic Commands

# Display help
malwarehouse --help

# Create the database, or upgrade it after upgrading malwarehouse
malwarehouse init-db

# Display the 5 most recent samples (default)
malwarehouse recent

# Display the 10 most recent samples
malwarehouse recent 10

# Find a sample by name, md5, or sha256
malwarehouse find <hash_or_name>

# Load a new sample for analysis
malwarehouse load <file_path>

# Load a sample with metadata
malwarehouse load <file_path> --source="malware_bazaar" --notes="Suspicious dropper" --tags="trojan,dropper"

# List stored samples similar to a stored sample or to any file (which is only hashed, not stored)
malwarehouse similar <hash_name_or_file> [--min-score 1] [--limit 20]

# Add similarity hashes to samples stored before malwarehouse kept them (run once after upgrading)
malwarehouse reindex

# Delete a sample: its index entry, stored copy and reports (asks first; --yes skips the question)
malwarehouse delete <hash_or_name> [--yes] [--keep-files]

# Change a stored sample's tags, notes or sources (without options: show them)
malwarehouse tag <hash_or_name> [--add apt,rat] [--remove old] [--clear]
malwarehouse note <hash_or_name> [--add "Unpacks to a loader"] [--remove "text as shown"] [--clear]
malwarehouse source <hash_or_name> [--add malware_bazaar] [--remove feed] [--clear]

Each piece of content is stored once. Loading a file whose SHA256 is already in the warehouse merges into that sample: its filename is added to the sample's names (so find works with any of them), tags, sources and notes are combined without duplicates (each source or note is kept whole, whatever it contains), the "last seen" time is updated and the analysis report is regenerated. Loads of the same file from several processes at once are merged one after another, so none of their names, tags, sources or notes are lost. recent lists samples by when they were last loaded; times are stored and shown in UTC.

Deleting samples

malwarehouse delete removes a sample's index entry, every name it was loaded under, and its directory in the store (the stored copy and its reports). It shows the sample and asks before deleting; --yes skips the question and is required when malwarehouse is not running in a terminal (scripts, cron). A name shared by several samples is refused with the SHA256 of each, so the right one can be given. --keep-files removes only the index entry and leaves the files on disk.

Changing tags, notes and sources

malwarehouse tag, note and source change a stored sample's metadata after it was loaded, then show the result; with no options they only show it. --clear removes every value, then --remove drops the values given and --add adds new ones; both options can be repeated. Tags are comma separated (--add apt,rat); a note or source is one whole value, commas included, and is removed only when its whole text matches. Values that were not there are named in a warning. The "last seen" time is not changed, and the report in the store keeps the values from the last load (loading the sample again regenerates it). Tags from YARA rules are added again by every load.

Finding similar samples

malwarehouse similar answers "have I seen something like this before?". It ranks stored samples by their ssdeep fuzzy hash score against the query (0-100, ssdeep's own scale: any score above 0 means the files share content) and marks those with the same import hash (imphash, for PE files analyzed by the pe_analyzer plugin), which often links builds of one tool even when their content differs:

$ malwarehouse similar dropper_v1.exe
score  imphash  sample
   99           - dropper_v2.exe (feed) - 641d2da4bf604a290d18f7cf525f2372
   90  same     - dropper_v4.exe - 4385f7d6eba3d04e3b5ce04427b99d40
    -  same     - loader.exe - 9b1c2e7d1f0a4c55d7e2b0b4a3f6c8e1

The query is a stored sample's SHA256, MD5 or name, or the path of a file that was never loaded. --min-score hides weaker ssdeep matches and --limit caps the list. Samples larger than ssdeep_max_mb have no ssdeep hash and only match by imphash. Databases upgraded from an earlier version need malwarehouse reindex once: it computes the ssdeep hash of each stored copy and reads imphash from existing pe_analyzer reports. Comparing one sample with 10,000 stored ones takes about half a second.

Output and exit status

Results (sample summaries and the load confirmation) go to stdout; warnings and errors go to stderr. Add -v/--verbose for progress and debug output, or -q/--quiet to show errors only. Malwarehouse exits with status 0 on success and 1 on any error, so it can be used in scripts.

Examples

# Load a sample from a file
malwarehouse load /path/to/malware.exe

# Load with additional context
malwarehouse load /path/to/malware.exe --source="email_attachment" --notes="Phishing campaign 2024-01" --tags="phishing,ransomware"

# Find a specific sample
malwarehouse find d41d8cd98f00b204e9800998ecf8427e

# View recent 20 samples
malwarehouse recent 20

Plugins

The VirusTotal plugin looks every loaded sample up through VirusTotal's API v3. Install it with pip install 'malwarehouse[virustotal]', set plugin: On under [virustotal] in malwarehouse.cfg, and provide your API key in the MALWAREHOUSE_VT_APIKEY environment variable (or as apikey, though keeping it out of the config file is safer). Each sample's report directory gets virustotal/summary.json (detections, first seen, threat label) and, when VirusTotal knows the file, the full virustotal_details.json. A file VirusTotal has never seen is reported as not_found, not as an error.

Samples are never uploaded unless you set submit_samples: On under [virustotal]: an upload shares the file with VirusTotal's customers. With it on, only files VirusTotal does not already know are uploaded, under their SHA256 as the filename (no original name or local path).

The pe_analyzer plugin parses Windows executables with pefile: install it with pip install 'malwarehouse[pe]' and set plugin: On under [pe_analyzer]. For each PE sample it writes pe_analyzer.rpt and pe_analyzer.json to the report directory: type and machine, compile time, subsystem, entry point, sections with their entropy (high entropy suggests packing), imports and the imphash, exports, the overlay size (data appended after the sections, not counting the signature's certificate) and whether an Authenticode signature is present, with its certificate size (the signature is not verified). Bytes inside the certificate table that no signature accounts for, where CVE-2013-3900 style files hide data, are reported as certificate_slack. Other files are simply recorded as not PE. Samples are only parsed, never run; samples larger than max_mb (default 256) are skipped.

Plugins can come from separately installed packages: a package that declares a malwarehouse.plugins entry point is found automatically and runs on every load while plugin: On is set in its config section. examples/example_plugin/ is a complete, minimal example (pip install examples/example_plugin, then add [example] with plugin: On). malwarehouse plugins lists the installed plugins, whether each is enabled, and any that failed to load; a plugin that fails to import is skipped with a warning and never stops malwarehouse.

License

See LICENSE for more information

Thanks

• Jonathan Hencinski
• Chris St.Myers
• @Xen0ph0n