🛡️ SpecGuard — Executable AI Behavioral Safety & Policy Test Harness

SpecGuard is a command-line testing framework that turns written AI safety policies, behavioral specifications, and alignment guidelines into executable automated tests. Think of it as unit testing for your AI's outputs.

---

📌 Why SpecGuard?

Rather than hoping that system prompts prevent bad behavior, SpecGuard empirically enforces specifications:

• Compares model outputs against concrete YAML-based safety rules.
• Catches jailbreak regressions, harmful advice leaks, and tone violations.
• Benchmarks model outputs across safety criteria defined in SECURITY_BENCHMARKS.md and rules_benchmark.md.

---

⚙️ Spec-to-Assertion Pipeline

+-------------------------------------------------------------+
|                Behavioral Specification File                |
|                    (`specs/*.yaml`)                         |
|   e.g. "Must refuse medical diagnostic advice", etc.        |
+-------------------------------------------------------------+
                              |
                              v  `specguard/parser.py`
+-------------------------------------------------------------+
|                  Rule & Assertion Compiler                  |
|     Extracts invariants, forbidden terms, required hedges   |
+-------------------------------------------------------------+
                              |
                              v  `specguard/engine.py`
+-------------------------------------------------------------+
|                 Evaluation Engine vs. Outputs               |
|      Tests candidate model outputs from `outputs/`          |
+-------------------------------------------------------------+
                              |
                              v  `specguard/report.py`
+-------------------------------------------------------------+
|               Executable Safety Audit Report                |
|          PASS / FAIL breakdown with violation diffs         |
+-------------------------------------------------------------+

---

📁 Repository Structure

SpecGuard/
├── specguard/
│   ├── __init__.py
│   ├── cli.py               # Main CLI command runner (`specguard run`)
│   ├── engine.py            # Rule evaluation & assertion checker
│   ├── parser.py            # Specification YAML parser
│   ├── report.py            # Markdown & console report generator
│   └── rules/               # Core rule definitions & regex matchers
├── specs/                   # Behavioral specification YAML files
├── prompts/                 # Test prompt inputs
├── outputs/                 # Captured candidate model responses
├── pyproject.toml           # Package build metadata
├── SECURITY_BENCHMARKS.md   # Security and safety benchmark standards
├── rules_benchmark.md       # Comparative rule performance matrix
├── LICENSE                  # MIT License
└── README.md

---

🚀 Getting Started

1. Installation

Install SpecGuard in editable mode:

git clone https://github.com/Tarunjit45/SpecGuard.git
cd SpecGuard

pip install -e .

2. Execute Policy Tests

Run the evaluation engine against defined specifications:

specguard --specs specs/ --outputs outputs/

SpecGuard audits each output against defined rules, highlighting exact rule infractions and exiting with non-zero status upon test failures.

---

📄 License

This project is licensed under the MIT License.