README.md

SeeNOW

Smart Exam Engine & Navigated Optical Workflow

SeeNOW is an enterprise-grade autonomous browser-controlling desktop agent. Powered by Set-of-Marks (SoM) visual grounding, Playwright CDP, and OpenAI-compatible LLM routers (such as 9router), SeeNOW reads quiz screens, verifies answers analytically, and navigates web evaluations with zero human intervention.

---

🌟 Key Capabilities

• Always-on-Top Floating HUD: Lightweight native CustomTkinter desktop window floating seamlessly above any active quiz portal without obstructing view.
• 1-Click Chrome/Edge CDP Launcher: Launches your local Chrome or Edge browser on port 9222 with a persistent profile (./chrome_profile), ensuring login sessions are preserved without bot detection (navigator.webdriver bypassed).
• Set-of-Marks (SoM) Grounding: Injects high-contrast badges [1], [2], [3] onto interactive elements (radio buttons, checkboxes, action buttons) before capturing screen state.
• Iframe & Custom UI Support: Handles child <iframe> elements and non-standard radio/checkbox implementations (Vue, React, Element UI, Ant Design, Quasar).
• Optimized Vision Pipeline: Automatic Lanczos downscaling to 1280px width and JPEG 70% compression reducing frame payloads to <90 KB (>95% bandwidth savings).
• Anti-Blur Focus Shield: Injects background scripts overriding document.visibilityState to avoid tab-switch and blur detection events.
• SPA State Transition Guard: SHA-256 state hashing verifies page updates before triggering subsequent captures, preventing question-skipping.
• Global Hotkeys: Instant trigger from keyboard:
  • Alt + S: Solve Once
  • Alt + A: Toggle Auto-Loop
  • Alt + Q: Emergency Stop

---

📁 Repository Structure

seenow/
├── .github/
│   ├── workflows/
│   │   ├── ci.yml               # Multi-OS (Ubuntu, Windows, macOS) & Python 3.10-3.13 Matrix
│   │   ├── security.yml         # Bandit, Semgrep & pip-audit SAST scans
│   │   ├── codeql.yml           # GitHub CodeQL security analysis
│   │   └── release.yml          # Automated PyInstaller build on v* tags
│   ├── ISSUE_TEMPLATE/
│   │   ├── bug_report.md
│   │   └── feature_request.md
│   └── pull_request_template.md
├── assets/
│   ├── logo.png                 # Official high-resolution transparent logo
│   ├── icon.png                 # 256x256 application icon
│   └── icon.ico                 # Multi-resolution Windows executable icon
├── core/
│   ├── __init__.py
│   ├── agent_worker.py          # Background worker thread with anti-skip state transition checks
│   ├── browser_engine.py        # Playwright CDP lifecycle, frame locator, image resizing
│   ├── cache_manager.py         # Sub-20ms Question LRU cache with order-invariant matching
│   ├── config_manager.py        # Centralized config loader with URL validation & secret masking
│   ├── llm_router.py            # 9router client with persistent HTTP connection pooling
│   ├── logger_setup.py          # Structured JSON logging system
│   ├── os_screen_engine.py      # MSS + PyAutoGUI fallback engine
│   ├── quiz_notary.py           # Automated markdown session notary and evidence exporter
│   └── som_injector.js          # Injected JavaScript Set-of-Marks labeling engine
├── gui/
│   ├── __init__.py
│   ├── app_window.py            # Main CustomTkinter HUD window and Settings panel
│   └── components.py            # StatusBadge, MetricPill, and AnswerCard components
├── docs/
│   ├── README.md                # Comprehensive documentation index
│   ├── hcia-cloud-computing/    # HCIA-Cloud Computing V5.5 Master Handbook & 14 Modules
│   ├── general-knowledge-of-cloud-computing/ # General Knowledge Master Handbook & 7 Modules
│   └── development-and-basic-concepts-of-cloud-computing/ # Development & Concepts Master Guide
├── tests/
│   ├── __init__.py
│   ├── test_browser_engine.py   # CDP connection and frame locator tests
│   ├── test_cache_manager.py    # LRU caching and hash key tests
│   ├── test_config.py           # Configuration and URL validation tests
│   ├── test_gui_components.py   # UI custom widgets tests
│   ├── test_json_parser.py      # Resilient JSON extraction tests
│   ├── test_notary.py           # Session logger tests
│   ├── test_schema.py           # Output Pydantic schema validation tests
│   └── test_worker.py           # Agent worker orchestration tests
├── .env.example
├── .gitignore
├── config.json.example
├── pyproject.toml               # Build configuration, Ruff, Mypy, Pytest & Bandit rules
├── requirements.txt
├── requirements-dev.txt
├── LICENSE                      # MIT License
├── SECURITY.md                  # Vulnerability reporting policy
├── CONTRIBUTING.md              # Contribution guidelines
├── CHANGELOG.md                 # Version release history
├── build_exe.py                 # PyInstaller standalone build script
└── main.py                      # Application entry point

---

🚀 Quick Start

1. Installation

# Clone the repository
git clone https://github.com/rhaffle87/seenow.git
cd seenow

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: .\venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Configuration

Copy the example configuration:

cp config.json.example config.json

Edit config.json (or configure via the Settings tab in the desktop application) to provide your router endpoint and API key:

{
  "router_base_url": "http://localhost:20128/v1",
  "router_api_key": "YOUR_API_KEY_HERE",
  "primary_vision_model": "ag/gemini-3.8-flash-high"
}

3. Running SeeNOW

python main.py

1. Click "🚀 Launch / Attach Chrome (CDP Port 9222)" on the Live HUD.
2. Log in to your quiz portal in the launched browser window.
3. Use Alt + S to solve the current question, or Alt + A for full automated progression.

---

🛠️ Testing & Code Quality

Run the local test and security verification suite:

# Install developer tools
pip install -r requirements-dev.txt

# Run headless unit tests
pytest

# Code linting
ruff check .

# Static type checking
mypy core

# Security SAST audit
bandit -r core gui main.py -ll

---

📦 Building Standalone Executable (.exe)

Compile SeeNOW into a standalone, portable Windows binary:

python build_exe.py

The compiled binary will be located at dist/SeeNOW/SeeNOW.exe.

---

📄 License

This project is licensed under the MIT License.