build(deps-dev): update py-spy requirement from >=0.3.14 to >=0.4.2 - #22
Closed
dependabot[bot] wants to merge 191 commits into
Closed
dependabot[bot] wants to merge 191 commits into
dependabot[bot] wants to merge 191 commits into
Conversation
Repository Cleanup: - Remove redundant root MD files (moved content to docs/) - Remove backups/ folder (git history is sufficient) - Comprehensive .gitignore for Triton YOLO project - Ignore generated files: *.dali, *.plan, *.onnx, reference_repos/, test_images/ New Features: - Add shared_client query parameter to toggle gRPC connection modes - Add /connection_pool_info endpoint for A/B testing - Add test_shared_vs_per_request.sh for automated testing - Response includes "shared_client" field showing mode used Documentation: - Add comprehensive docs/ folder with technical guides - Add AUTOMATION.md for complete automation reference - Add ATTRIBUTION.md for license compliance - Add CLAUDE.md for project instructions - Update README.md with quick start guide Project Structure: - Organize into functional folders: export/, dali/, tests/, benchmarks/ - Add model configs for all 4 tracks (A/B/C/D) - Add monitoring stack configs (Prometheus, Grafana, Loki) - Add Ultralytics patches for end2end export - Add comprehensive test suite Performance Testing: - Users can now toggle: ?shared_client=true (batching) vs false (per-request) - Enables scientific testing of batching vs DALI bottleneck hypothesis - Provides tools to measure actual performance impact 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ctor ## Track E - Visual Search Pipeline (NEW) ### MobileCLIP Integration - Add MobileCLIP2-S2 image encoder export (export/export_mobileclip_image_encoder.py) - Add MobileCLIP2-S2 text encoder export (export/export_mobileclip_text_encoder.py) - Support for 512-dim embeddings with cosine similarity search - Image-to-image and text-to-image search capabilities ### Triple-Branch DALI Pipeline - Add dual DALI pipeline for YOLO+CLIP preprocessing (dali/create_dual_dali_pipeline.py) - Add simple YOLO+CLIP pipeline variant (dali/create_yolo_clip_dali_pipeline.py) - Outputs: yolo_images (640x640), clip_images (256x256), original_images (max 1920px) - Add DALI validation against PyTorch reference (dali/validate_dual_dali_preprocessing.py) ### Track E Triton Models - models/mobileclip2_s2_image_encoder/ - TensorRT image encoder - models/mobileclip2_s2_text_encoder/ - TensorRT text encoder - models/box_embedding_extractor/ - Per-object embedding extraction - models/dual_preprocess_dali/ - Triple-branch preprocessing - models/yolo_clip_preprocess_dali/ - Simple dual-branch preprocessing - models/yolo_clip_ensemble/ - YOLO+CLIP ensemble - models/yolo_mobileclip_ensemble/ - Full visual search ensemble ### Track E API Endpoints - POST /track_e/detect - YOLO detection only - POST /track_e/predict - Detection + global embedding - POST /track_e/predict_full - Detection + global + per-box embeddings - POST /track_e/embed/image - Image embedding only - POST /track_e/embed/text - Text embedding only - POST /track_e/ingest - Ingest image into OpenSearch - POST /track_e/search/image - Image-to-image similarity search - POST /track_e/search/text - Text-to-image search - POST /track_e/search/object - Object-level search ### Track E Documentation - docs/TRACK_E_GUIDE.md - Complete implementation guide - docs/TRACK_E_SUMMARY.md - Architecture overview - docs/TRACK_E_IMPLEMENTATION_STATUS.md - Progress tracking - docs/TRACK_E_DEPLOYMENT_CHECKLIST.md - Deployment checklist ## Architecture Refactor ### Modular Service Structure - src/routers/ - FastAPI route handlers (track_a, track_bcd, track_e, models, health) - src/services/ - Business logic (triton, pytorch, opensearch, track_e) - src/clients/ - Triton client management (shared vs per-request) - src/schemas/ - Pydantic models for requests/responses - src/config/ - Centralized configuration management - src/core/ - Core utilities and dependencies ### Improved Triton Client Management - Shared client pool with configurable toggle - Per-request client option for isolation - Automatic connection management and health checks ## Build System ### New Makefile (100+ targets) - Service management: up, down, restart, logs, status - Model export: export-models, export-mobileclip, export-end2end - DALI pipelines: create-dali, create-dali-dual, validate-dali - Testing: test-all-tracks, test-track-a/b/c/d/e, compare-tracks - Benchmarking: bench-quick, bench-full, bench-matrix, bench-track-* - Model management API: api-upload-model, api-export-status, api-models - Monitoring: open-grafana, open-prometheus, metrics, gpu - Reference repos: clone-refs-essential, clone-refs-all, clone-refs-list ### Pre-commit Hooks - Add .pre-commit-config.yaml with ruff integration - Automatic formatting and linting on commit ## Code Quality ### Ruff Configuration (pyproject.toml) - Comprehensive linting rules (pyflakes, pycodestyle, isort, etc.) - Per-file ignores for tests and DALI scripts - Line length: 100 characters - Python 3.10+ target ### Code Formatting - All Python files formatted with ruff - Import sorting standardized - Consistent quote style and formatting ## Model Export Improvements ### Enhanced export_models.py - YAML config file support for batch exports - Custom model export with --custom-model flag - Automatic labels.txt and config.pbtxt generation - Progress reporting and error handling - Support for all formats: onnx, onnx_end2end, trt, trt_end2end ### PyTorch Model Download - New download_pytorch_models.py script - Replaces deprecated shell script - Supports: nano, small, medium, large, xlarge ## DALI Pipeline Updates ### Modular DALI Package - dali/__init__.py - Package initialization - dali/config.py - Centralized DALI configuration - dali/utils.py - Shared utilities for pipeline creation ### Improved Letterbox Pipeline - Better error handling and validation - Configurable batch sizes and instance counts - Support for streaming, balanced, and batch variants ## Testing Improvements ### New Test Files - tests/compare_tracks.py - Cross-track detection comparison - tests/test_track_e_ensemble.py - Track E ensemble tests - tests/test_track_e_images.py - Track E image processing tests - tests/test_track_e_integration.py - Track E integration tests - tests/test_track_e_phase1_pipeline.py - Pipeline validation - tests/validate_mobileclip_triton.py - MobileCLIP model validation ### Track E Test Scripts - scripts/track_e/setup_mobileclip_env.sh - Environment setup - scripts/track_e/test_track_e_images.py - Image test suite - scripts/track_e/test_integration.py - Integration tests ## Infrastructure ### Docker Compose Updates - OpenSearch 3.3.1 integration for vector search - OpenSearch Dashboards for management - Port 4607 (OpenSearch), 4608 (Dashboards) - Improved health checks and restart policies ### Reference Repository Management - scripts/clone_reference_repos.sh - Clone attribution repos - Essential: levipereira/ultralytics (End2End NMS), apple/ml-mobileclip - Recommended: mlfoundations/open_clip, ultralytics/ultralytics - Optional: DeepStream-Yolo, triton-server-yolo, yolov8-triton ## Cleanup ### Removed Deprecated Files - benchmarks/build.sh (replaced by Makefile) - benchmarks/GO_SETUP.md, TESTING_GUIDE.md (consolidated) - export/cleanup_for_reexport.sh, download_pytorch_models.sh, export_small_only.sh - scripts/profile_api.sh, setup_padding_comparison.sh, test_shared_client.sh - src/utils/models.py, triton_end2end_client.py, triton_shared_client.py - tests/validate_models.sh ### Consolidated Documentation - Updated all README.md files with current architecture - Removed redundant documentation - Improved code comments and docstrings ## Dependencies ### New Requirements - opensearch-py>=2.3.0 - Async OpenSearch client - transformers>=4.30.0 - CLIP tokenizer - timm, huggingface_hub - Model loading - pre-commit - Git hooks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…e Refactor ## Summary This merge integrates the complete Track E visual search feature with MobileCLIP embeddings and a major architecture refactor into the main branch. ## Key Features Merged ### Track E - Visual Search Pipeline - MobileCLIP2-S2 image and text encoder integration - Triple-branch DALI preprocessing (YOLO 640x640 + CLIP 256x256 + HD cropping) - OpenSearch k-NN vector database integration - Image-to-image and text-to-image similarity search - Per-object embedding extraction for fine-grained search ### Architecture Improvements - Modular FastAPI structure (routers, services, schemas, clients) - Improved Triton client management with connection pooling - Centralized configuration management - Enhanced error handling and validation ### Developer Experience - Comprehensive Makefile with 100+ targets - Pre-commit hooks with ruff linting/formatting - pyproject.toml with modern Python tooling - Reference repository cloning scripts for attribution ### New Triton Models - mobileclip2_s2_image_encoder (TensorRT) - mobileclip2_s2_text_encoder (TensorRT) - box_embedding_extractor (Python backend) - dual_preprocess_dali (triple-branch DALI) - yolo_mobileclip_ensemble (full visual search) ## API Endpoints Added Track E endpoints on unified API (port 4603): - POST /track_e/detect - YOLO detection - POST /track_e/predict - Detection + global embedding - POST /track_e/predict_full - Detection + per-box embeddings - POST /track_e/embed/image - Image embedding - POST /track_e/embed/text - Text embedding - POST /track_e/ingest - Index image in OpenSearch - POST /track_e/search/image - Image similarity search - POST /track_e/search/text - Text-to-image search - POST /track_e/search/object - Object-level search ## Documentation - docs/TRACK_E_GUIDE.md - Complete implementation guide - docs/TRACK_E_SUMMARY.md - Architecture overview - docs/TRACK_E_DEPLOYMENT_CHECKLIST.md - Production deployment ## Statistics - 133 files changed - +24,364 insertions, -6,951 deletions - 47 new files added - 14 deprecated files removed ## Branch History - feat/mobile-clip: a78698f Add Track E visual search with MobileCLIP and major architecture refactor - main: be82302 Clean repository structure and add shared client toggle feature 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit adds infrastructure for running fair, isolated benchmarks across all tracks (A, B, C, D variants, E variants). Key changes: - Add isolated_benchmark.sh: Unloads all models, loads only track-specific models, runs benchmark, repeats for each track - Add create_benchmark_configs.py: Generates equalized instance configs (6 DALI, 4 TRT, 4 Python) for fair GPU allocation - Add aggregate_results.py: Compares results, calculates speedups vs baseline - Add Makefile targets: bench-isolated-*, bench-create-configs, etc. - Fix triton_bench.go getBaseURL() function (was incorrectly parsing URLs) - Set workers=4 to match TRT instance count for fair PyTorch comparison Fair comparison approach: - Track A: 4 uvicorn workers = 4 PyTorch model instances - Tracks B/C/D/E: 4 TRT instances in benchmark configs - DALI preprocessing: 6 instances (I/O-bound, feeds GPU) Usage: make bench-isolated-track TRACK=A DURATION=60 make bench-isolated-all make bench-isolated-results 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Benchmark Results (all tracks on GPU 0, fair comparison): - Track A (PyTorch): 45.6 RPS @ 16 clients (baseline) - Track B (TRT+CPU NMS): 57.8 RPS @ 32 clients (1.27x) - Track C (TRT+GPU NMS): 50.7 RPS @ 128 clients (1.11x) - Track D_batch (DALI+TRT): 98.0 RPS @ 128 clients (2.15x) - Track E (YOLO+MobileCLIP): 77.4 RPS @ 32 clients (1.70x) Config changes for fair benchmarking: - All TRT models: 4 instances on GPU 0 (matches 4 PyTorch workers) - DALI preprocessing: 6 instances on GPU 0 (I/O bottleneck needs more) - All models consolidated to single GPU for accurate comparison New files: - benchmarks/BENCHMARK_RESULTS.md: Comprehensive results documentation - unload_all.sh: Helper script for isolated model testing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Stress Test Results (1138 images, 60s duration, high concurrency): - Track A (PyTorch): 56.9 RPS @ 32 clients (baseline) - Track B (TRT+CPU NMS): 65.6 RPS @ 64 clients (1.15x) - Track C (TRT+GPU NMS): 52.3 RPS @ 128 clients (0.92x) - Track D_batch (DALI+TRT): 103.8 RPS @ 256 clients (1.82x) - Track E (YOLO+CLIP): 123.3 RPS @ 64 clients (2.17x) - BEST! Key finding: Track E with parallel YOLO+MobileCLIP achieves best throughput due to excellent child model batching (avg 11.61). Config changes for stress testing: - max_queue_delay_microseconds: 100000 (100ms for batch accumulation) - max_queue_size: 2048 (high capacity for stress tests) - preferred_batch_size: [16, 32, 64] - timeout: 30 seconds 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements automatic retry with exponential backoff for Triton inference: - New src/utils/retry.py utility with retry_sync/retry_async functions - Retries on: queue full, resource exhausted, timeout, unavailable - Does NOT retry on: invalid input, model not found (non-transient errors) - Default: 3 retries, 0.1s base delay, 5s max delay, jitter to prevent thundering herd Changes to TritonClient: - All inference methods now use _infer_with_retry() wrapper - Configurable retry parameters in constructor - Ensures no request is dropped under high load Test results with retry enabled: - Track D_batch @ 256 clients: 100% success (was failing before) - Track D_batch @ 512 clients: 100% success (was failing before) - Track E @ 256 clients: 100% success (was 99.5%) - Track E @ 512 clients: 100% success (was 93.5%) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track E Batch Processing: - Add /track_e/predict_batch endpoint for processing up to 64 images per request - Implement infer_track_e_batch() in TritonClient using ThreadPoolExecutor - Add batch tracks (E_batch16/32/64) to benchmark tool with sendBatchRequest() - Update middleware to allow 10GB uploads for batch endpoints Track F CPU Preprocessing: - Add Track F router with /track_f/predict endpoint - Implement CPU-based preprocessing as DALI comparison baseline - Add infer_track_f() to TritonClient and InferenceService Benchmark Results (verified): - E_batch16 @ 64 clients: 142.6 RPS, P95=564ms, 100% success - E single @ 128 clients: 117.7 RPS, P95=1462ms, 100% success - Batch mode: 21% higher throughput, 2.6x lower latency Documentation: - Document DALI GPU preprocessing advantages in CLAUDE.md - Add hot reload instructions for development workflow - Update model configs with optimized instance counts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OpenSearch Fix: - Change indexed_at from 'now' string to proper ISO 8601 timestamp - OpenSearch date fields require valid datetime format, not string 'now' - Fixes: "mapper_parsing_exception: failed to parse field [indexed_at]" HuggingFace Cache Fix: - Mount cache to /home/appuser/.cache/huggingface (container runs as appuser) - Update HF_HOME and HF_HUB_CACHE env vars to match appuser home - Fixes: PermissionError when downloading CLIP tokenizer for text search Verified all Track E features working: - Full ensemble: detections + global + box embeddings - OpenSearch: ingestion and k-NN search - Image-to-image, text-to-image, object-to-object search 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major Features:
- Multi-index OpenSearch architecture (global, vehicles, people, faces)
- FAISS IVF GPU-accelerated clustering with auto-scaling
- Cluster maintenance service with automatic rebalancing
- Album/cluster browsing API endpoints
New Files:
- src/services/clustering.py: FAISS IVF clustering service with GPU support
- src/services/cluster_maintenance.py: Automatic rebalancing based on ingestion patterns
- scripts/test_clustering.py: Clustering test with mosaic visualization
- docs/opensearch_schema_design.md: Multi-index architecture documentation
API Endpoints Added:
- POST /track_e/clusters/train/{index_name}: Train FAISS clustering
- GET /track_e/clusters/stats/{index_name}: Cluster statistics
- GET /track_e/clusters/balance/{index_name}: Balance assessment
- POST /track_e/clusters/rebalance/{index_name}: Force rebalance
- GET /track_e/clusters/{index_name}/{cluster_id}: Get cluster members
- GET /track_e/albums: List auto-generated albums
- GET /track_e/maintenance/status: Maintenance status
- POST /track_e/maintenance/run: Run maintenance check
Technical Details:
- Dynamic n_clusters calculation: sqrt(n) to n/10 for optimal separation
- GPU-accelerated training via faiss-gpu-cu12
- Incremental cluster assignment (~0.1ms per embedding)
- Configurable rebalancing thresholds by ingestion pattern
- Vehicle clusters achieve brand/color/type separation with 128+ clusters
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Features: - SCRFD 10G face detection with TensorRT acceleration - ArcFace W600K R50 face recognition/embedding extraction - Quad-branch DALI preprocessing (YOLO, CLIP, face detection, original) - Unified embedding extractor for faces within detected persons - Face-aware OpenSearch indexing with per-face embeddings - Multi-index ingestion (faces, people, vehicles, global embeddings) - Mosaic visualization for face clusters Technical changes: - Added face_alignment.py for ArcFace-compatible face preprocessing - Extended triton_client.py with unified inference pipeline - Added face embedding fields to Track E schemas - New export scripts for SCRFD and ArcFace models - Updated pre-commit config to use ruff v0.14.0 - Added outputs/ to .gitignore for mosaic/cluster outputs Scripts: - ingest_and_cluster_faces.py: Face-specific ingestion with FAISS clustering - ingest_all_indexes.py: Multi-category ingestion pipeline - create_mosaics.py: Visualization tool for clustered results 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add imohash for exact duplicate detection during ingestion (~1ms constant time)
- Add CLIP embedding similarity for near-duplicate detection (threshold=0.99, matches Immich)
- Create DuplicateDetectionService with full group management
- Add 8 new API endpoints for duplicate detection and management:
- POST /duplicates/find, /duplicates/find_by_image, /duplicates/scan
- GET /duplicates/groups, /duplicates/group/{id}, /duplicates/stats
- DELETE /duplicates/group/{id}/member/{id}
- POST /duplicates/groups/merge
- Auto-assign near-duplicates to groups during ingestion
- Add detect_near_duplicates and near_duplicate_threshold params to /ingest
- Extend OpenSearch schema with imohash, duplicate_group_id, duplicate_score
- Refactor client_ingest.py to use parameter passing instead of global state
- Update FEATURE_ROADMAP.md to reflect Phase 3 completion
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit adds three major features to the Track E visual search pipeline: ## OCR Support (PP-OCRv5) - Add PaddleOCR detection and recognition models (TensorRT) - New endpoints: /track_e/ocr/predict, /track_e/ocr/predict_batch - Text search via /track_e/search/ocr with trigram analyzer - Auto-index OCR text during image ingestion (enable_ocr param) - Export scripts for PP-OCRv5 detection and recognition models ## YOLO11-Face Detection - Add YOLO11-face as alternative to SCRFD (better batching support) - Configurable via `detector` param: 'yolo11' (default) or 'scrfd' - End2End TensorRT export with GPU NMS - Face pipeline with ArcFace identity embeddings - New Makefile targets: setup-yolo11-face, test-yolo11-face ## High-Throughput Batch Ingestion - New /track_e/ingest_batch endpoint (up to 64 images per request) - Target throughput: 300+ RPS with optimized parallel processing - Parallel hash computation, batch Triton inference, bulk OpenSearch indexing - Reduces HTTP overhead for large photo library ingestion ## Additional Changes - Face identity service for face search and 1:N identification - Updated CLAUDE.md and README.md with new endpoints and examples - Added test_results/ to .gitignore - Fixed 15 ruff linting errors (list comprehensions, unused variables, etc.) - New model configs: ocr_pipeline, paddleocr_det_trt, paddleocr_rec_trt, yolo11_face_pipeline, yolo11_face_small_trt, unified_complete_pipeline ## New Files - src/services/ocr_service.py - OCR text extraction wrapper - src/services/face_identity.py - Face identity management - export/export_paddleocr_*.py - PP-OCRv5 TensorRT export - export/export_yolo11_face.py - YOLO11-face TensorRT export - scripts/download_yolo11_face.py - Model download script - dali/create_penta_dali_pipeline.py - 5-branch DALI (YOLO+CLIP+Face+HD+OCR) - docs/OCR_*.md - OCR implementation documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…t ingest Key changes: - Replace DALI with CPU preprocessing in ingest pipeline (100% success rate) - Make YOLO11-face the default face detector (used by ingest) - Add FastFaceClient for direct gRPC calls (bypasses BLS) - Update benchmark tool with E_faces, E_faces_scrfd, E_faces_fast tracks - Archive C++ backend experiment (doesn't scale due to sync BLS) Performance results (128 concurrent clients): - E_faces (YOLO11-face): 49 RPS, 100% success - E_ingest (full pipeline): 25 RPS, 100% success New files: - src/clients/fast_face_client.py - Direct gRPC face client - src/services/cpu_preprocess.py - CPU preprocessing utilities - src/services/parallel_ingest.py - Parallel ingest workers - models/face_pipeline_cpp/ - Archived C++ backend (reference only) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- docs/INGEST_BENCHMARK_METHODOLOGY.md - Benchmark methodology documentation - models/unified_complete_pipeline_direct/ - Direct pipeline config (experimental) - models/unified_direct_ensemble/ - Direct ensemble config (experimental) - models/yolo11_face_pipeline_cpp/ - C++ face pipeline attempt (archived) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…d API BREAKING CHANGE: Complete API restructure from track-based to capability-based endpoints. ## Removed - All track naming (A/B/C/D/E/F) from code, docs, and API - DALI GPU preprocessing (entire dali/ directory) - SCRFD face detection (using YOLO11-face exclusively) - PyTorch direct inference (Track A) - 33 unused model directories (DALI ensembles, SCRFD, variants) - 57,000+ lines of legacy code ## New Clean API Structure - /detect - YOLO11 object detection - /faces - YOLO11-face + ArcFace recognition - /embed - MobileCLIP embeddings (image/text) - /search - Visual similarity search - /ingest - Data ingestion with duplicate detection - /analyze - Combined analysis pipeline - /clusters - FAISS IVF clustering - /query - Data retrieval - /ocr - PP-OCRv5 text extraction - /models - Model management - /health - Service monitoring ## Models (8 core models) - yolov11_small_trt_end2end - yolo11_face_small_trt_end2end - arcface_w600k_r50 - mobileclip2_s2_image_encoder - mobileclip2_s2_text_encoder - paddleocr_det_trt - paddleocr_rec_trt - yolo11_face_pipeline ## Code Structure - 11 modular routers in src/routers/ - Clean service layer in src/services/ - Renamed triton_client methods (infer_yolo_end2end, infer_yolo_clip_cpu, etc.) - Updated all documentation - New benchmark infrastructure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Standardize upload field names: use 'image' for single uploads, 'images' for batch - Add COCO class names to detection responses (class_name: "person", etc.) - Add ocr_pipeline and unified_complete_pipeline to Triton model loading - Fix health router to work without PyTorch-specific settings - Fix OCR router type annotation that broke OpenAPI schema generation - Pull embedding.py service from dev branch for analyze endpoint Benchmark results (all targets met): - /detect: 122 RPS (target >50) - /faces/recognize: 98 RPS (target >30) - /embed/image: 178 RPS (target >80) - /ocr/predict: 17 RPS (target >15) - /analyze: 25 RPS (~650ms per image) - /ingest: ~800ms per image (full pipeline + indexing) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…quality Major cleanup implementing industry best practices across the entire codebase. All pre-commit checks now passing with zero critical issues. **Code Quality Improvements:** - Removed all dead code (12 issues: 7 unused imports, 4 unused variables, 1 unused argument) - Refactored 8 global variables to @lru_cache and AppResources class patterns - Fixed 30+ try-except-pass blocks with proper logging - Fixed 6 code quality issues (nested ifs, manual loops, ambiguous Unicode) - Fixed 3 critical runtime errors discovered during end-to-end testing **Pre-commit Hooks Implementation:** - Comprehensive suite with 8 tools active: ruff, mypy, bandit, hadolint, gitleaks, shellcheck, pygrep-hooks - Superior configuration with newer versions than OpenTranscribe - All tools configured and passing - Conventional commits hook disabled (requires separate installation) **Shell Script Fixes:** - Fixed 21 SC2155 warnings (separate declaration from assignment) - Fixed SC2076 regex quoting - Zero shellcheck warnings remaining **Type Safety:** - Configured mypy with appropriate ML code settings - Fixed 100+ Pydantic Field definitions (Field(None) → Field(default=None)) - Fixed critical type errors in triton_pool, embedding service, models router - Proper handling of numpy/FAISS type operations **Security:** - Bandit security scanning active and passing - Gitleaks secret detection configured - Hadolint Dockerfile linting passing - All security warnings documented and justified **Bug Fixes:** - Fixed face search method names (infer_faces → recognize_faces/detect_faces) - Fixed batch analyze validation error (has_text bool wrapping) - Fixed missing model references (unified_direct_ensemble → unified_complete_from_tensors) - Fixed CLIP_MODELS attribute error (use CLIP_IMAGE_MODEL/CLIP_TEXT_MODEL) - Fixed unused function arguments in opensearch and face_identity - Fixed None checks in triton_pool for semaphore and lock - Added proper error handling for missing task retrieval **Documentation:** - Created CODE_AUDIT_REPORT.md with detailed findings - Created PRECOMMIT_COMPARISON.md comparing with OpenTranscribe - Created FINAL_AUDIT_SUMMARY.md documenting all work - Added COCO class names mapping for detection responses **Files Modified:** - 53 files changed: +2106 insertions, -891 deletions - Configuration: .pre-commit-config.yaml, pyproject.toml - Python source: 13 files in src/, 9 files in export/, 4 scripts - Dockerfiles: Dockerfile, Dockerfile.triton - Documentation: 3 new markdown files **Quality Metrics:** Before: 7 unused imports, 12 dead code, 8 globals, 30+ silent failures, 21 shell warnings After: 0 unused imports, 0 dead code, 0 globals, proper logging, 0 shell warnings All endpoints verified working. Codebase is production-ready with professional-grade code quality following industry best practices. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit addresses batch/directory ingest failures and adds automated testing infrastructure to validate all core functionality. Batch Ingest Fixes: - Fixed gRPC "too_many_pings" errors by relaxing HTTP/2 keepalive settings in triton_pool.py (min_time_between_pings: 5s, max_pings_without_data: 2) - Fixed model name reference in triton_client.py:1875 (unified_complete_pipeline_direct → unified_complete_pipeline) - Changed batch processing from tensor-based to JPEG-based inference in visual_search.py for better compatibility with ensemble models - Reduced concurrent workers to adaptive count (max 8) to prevent saturation - Result: 94% success rate on 50-image test dataset (47/50 processed, 3 malformed test images excluded) Test Infrastructure: - Added tests/test_full_system.py - comprehensive test suite for all endpoints * Tests all 10 ML models (detection, faces, CLIP, OCR) * Tests single and directory ingest pipelines * Tests OpenSearch indexing and search functionality * Clears OpenSearch data before testing for fresh runs * Supports custom timeout for long-running operations * All 32 tests passing (100%) - Added tests/validate_visual_results.py - visual validation with bounding boxes * Draws object detection boxes with class names and confidence scores * Draws face detection boxes with 5-point facial landmarks * Draws OCR text regions with extracted text * Saves annotated images to test_results/ for manual verification Documentation: - Updated CLAUDE.md Testing section with comprehensive test commands - All test results now stored in test_results/ directory (not repo root) - Test output files: test_results_final.txt, visual_validation_results.txt Test Results (32/32 passing): - Service health checks: API, Triton, OpenSearch ✅ - Object detection: 4 objects detected with class names ✅ - Face detection: 2 faces with embeddings (512-dim) ✅ - CLIP embeddings: Image and text (512-dim each) ✅ - OCR: 8 text regions extracted ✅ - Combined analysis: Full pipeline operational ✅ - Single image ingest: Successful with indexing ✅ - Directory ingest: 50 images in 7.3s ✅ - OpenSearch: Indexes populated correctly ✅ - Search queries: Image and text search operational ✅ Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
… updates
Documentation Improvements:
- Added PROJECT_STATUS.md - comprehensive current status document
* Complete code quality metrics
* Test coverage (32/32 tests passing)
* Performance benchmarks
* Recent work summary
* Known limitations
* Next steps
- Updated README.md
* Added "Recent Updates" section highlighting latest improvements
* Added "Testing" section with comprehensive test commands
* Added "Performance" section with measured latency/throughput
* Corrected API response format examples (normalized coordinates)
* Fixed detection response to show x1,y1,x2,y2 format
* Fixed ingest response to match actual API output
- Updated docs/README.md
* Added test suite commands
* Documented visual validation process
Response Format Corrections:
- Detection: box array → {x1, y1, x2, y2} with class_name
- Faces: Added landmarks array format
- Ingest: Updated to show actual response fields
- Added note about normalized coordinates (0.0-1.0)
Test Documentation:
- Full system test (32 tests)
- Visual validation with bounding boxes
- Test results location (test_results/ directory)
- Annotated image outputs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ructure Complete rewrite of src/README.md to reflect unified capability-based API: Removed: - All Track A/B/C/D/E references (old organizational system) - DALI pipeline documentation (removed from codebase) - PyTorch direct inference (removed) - Old endpoint patterns (/pytorch/predict, /track_e/*) Updated: - File structure to match current routers (detect, faces, embed, search, etc.) - API capabilities with current endpoint patterns - Service layer descriptions - Performance metrics from actual test results - Response format examples with correct structure - Configuration variables - Documentation links New Content: - Comprehensive capability-based API overview - Actual measured performance metrics (140-170ms detection, etc.) - Current router/service architecture - Real response format examples from tests - Batch processing performance (6.8 images/sec) - Link to PROJECT_STATUS.md Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Consolidated documentation by topic to eliminate redundancy and improve navigation: New Consolidated Documentation: 1. docs/ARCHITECTURE.md (19K) - System architecture and production patterns - Merged from PRODUCTION_ARCHITECTURE.md + THREAD_SAFETY_FIX.md - Comprehensive architecture overview - Production deployment patterns - Thread safety and concurrency - Scaling strategies (single GPU → multi-GPU → multi-node) 2. docs/OCR.md (16K) - Complete OCR guide - Merged from OCR_SETUP_GUIDE.md + OCR_IMPLEMENTATION_PLAN.md + OCR_DEPLOYMENT_CHECKLIST.md - Architecture and data flow - Model specifications (DB++ detection, SVTR-LCNet recognition) - TensorRT export with correct workspace syntax - API usage and Python examples - Performance tuning and troubleshooting 3. docs/PERFORMANCE.md (17K) - Performance optimization guide - Merged from PERFORMANCE_OPTIMIZATION.md + OPTIMIZATION_SUMMARY.md + GRPC_CONNECTION_SCALING.md - FastAPI optimizations (orjson, pillow-simd) - gRPC connection management and HTTP/2 multiplexing - Benchmarking methodology - Profiling with py-spy - Tuning parameters Archived Original Files: - Moved 8 files to docs/archive/ for reference - Preserved all technical content - Documented in docs/README.md Updated docs/README.md: - Reorganized into clear sections (Core, Capabilities, Performance) - Added links to consolidated docs - Listed archived files with consolidation notes - Updated API reference table - Documented project structure Benefits: - Reduced redundancy (8 files → 3 comprehensive guides) - Easier navigation (clear topical separation) - Preserved history (archived originals) - Comprehensive coverage (no content loss) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Updated: - "Model availability (Tracks B/C/D)" → "all Triton models" - Removed obsolete track_e/ subfolder section (folder no longer exists) - "FastAPI (all tracks)" → "FastAPI" in port reference Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed: - CODE_AUDIT_REPORT.md (initial code audit) - FINAL_AUDIT_SUMMARY.md (audit summary) - PRECOMMIT_COMPARISON.md (pre-commit comparison) - AUTOMATION.md (old automation notes) These were intermediate work products no longer needed. PROJECT_STATUS.md now contains comprehensive current status. Root directory now clean with only essential documentation: - README.md (main project documentation) - CLAUDE.md (AI assistant instructions) - PROJECT_STATUS.md (comprehensive current status) - ATTRIBUTION.md (third-party attributions) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed: - docs/archive/ folder (8 files) - all content consolidated into new comprehensive docs - backends/face_pipeline mount in docker-compose.yml (folder doesn't exist) Updated: - docs/README.md: Removed archive section and reference in project structure - src/utils/README.md: Removed dali and track references, fixed broken links * "Track A/B/C/D/E" → "All API endpoints" / "Embedding services" * Removed broken link to dali/README.md * Updated related documentation links Rationale: - Archived docs were 100% redundant (content in consolidated docs) - Git history preserves all original versions - backends/ and dali/ folders were deleted - Keeps documentation clean and current Files Kept: - Dockerfile.triton (required - used by docker-compose for Python backend models) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed Files: - pytorch_models/test_nano-trt.txt (test artifact) - scripts/export_results.json (export artifact) Removed from Disk (not tracked): - .mypy_cache/ (67MB - regenerated automatically) - .ruff_cache/ (3.1MB - regenerated automatically) - benchmarks/isolated/ (empty directory) - models/paddleocr_rec_trt/export/ (empty directory) - cache/huggingface/ (3.6MB - openai/clip model not used) Updated unified_complete_pipeline: - Removed all SCRFD references (model was removed in commit bfc510d) - Changed default face_model from 'scrfd' to 'unified' - Renamed _process_with_scrfd() → _process_with_unified_ensemble() - Updated model.py docstring and comments - Updated config.pbtxt header comments - Note: 'unified' mode uses yolo_unified_ensemble, not SCRFD Total cleanup: ~73MB removed from disk Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
New router src/routers/curation/settings.py wired into the curation
aggregator (__init__.py). GET returns {defaults, updated_at, updated_by};
PUT accepts a partial {defaults: {...}} body, merges it into the stored
document, and returns the full updated record. defaults is an OPEN map
keyed by axis id (not a fixed cluster/sort/detection_profile/prompt_pack
schema) so a future axis never requires a wire-format change.
PUT validates every axis/id pair against the live GET /methods registry
(never a hand-maintained copy) before writing: axis must be one of the
four axes a shared default can actually change today
(strategy_defaults.SETTABLE_DEFAULT_AXES -- cluster/sort/detection_profile/
prompt_pack; score/overlay/export have no single-selectable-id 'default'
concept a shared override could apply to, so they 422 rather than
silently accepting a dead value), and id must be a currently-advertised
id for that axis -- otherwise 422 with a message listing the valid
axes/ids.
Adds src/services/curation/strategy_defaults.py: SETTABLE_DEFAULT_AXES
and resolve_effective_default(axis, opensearch) -- the shared-settings
lookup PUT's validation depends on. This module doesn't yet change any
existing default-application behavior (that's the next commit); it only
introduces the resolver and its axis/id bookkeeping so the endpoint has
something real to validate and write against.
New models CurationSettingsResponse / CurationSettingsUpdateRequest in
_common.py, following the existing request/response model convention.
Tests: CRUD + partial-merge + 422 validation (test_curation_settings_router.py),
resolve_effective_default's own fallback/override/stale-override/broken-client
behavior (test_resolve_effective_default.py).
…ts from settings
strategy_registry.py's _cluster_strategies/_sort_strategies/
_detection_profile_strategies/_prompt_pack_strategies now take a
resolved default_id (from strategy_defaults.resolve_effective_default,
resolved once per get_registry() call against a single settings-doc
snapshot) instead of a hardcoded comparison -- GET /methods's per-axis
default:true/false flag is now genuinely derived, not independently
hardcoded per axis.
Rewires every real endpoint that currently applies a hardcoded default
when a request omits that axis's param, through the same resolver:
- cluster: clustering/orchestrator.py's cluster_residuals -- an omitted
?clustering_method now resolves the shared override before falling
back to DEFAULT_METHOD (feeds POST /pipeline/auto_label* and
POST /clusters/*'s residual-clustering stage).
- sort: review_sorts.build_sort -- an omitted/'default' ?sort on
GET /review/{tab} tries a shared override before falling back to
that tab's own hardcoded default (review.py awaits it with the
request's opensearch client). The per-tab defaults themselves are
untouched; a shared override is an additional, opt-in global choice
layered on top, not a replacement -- confirmed by the golden-body
regression tests (now async) still passing byte-identical with no
override configured.
- detection_profile / prompt_pack: GET /methods now derives their
default flag too, but no real endpoint has a per-request selection
param for either today (exactly one active profile/pack per process,
chosen via config, not per request) -- recorded as a known, deliberate
gap in the API contract doc rather than inventing a new endpoint.
Tests: test_methods_router.py's detection-profile test call site
updated for the new default_id parameter; test_review_sorts.py's
build_sort tests are now async; new
test_curation_settings_integration.py proves a PUT changes both
GET /methods's default flag AND the real cluster/sort endpoints'
behavior in the same request cycle, so the two can never drift.
Documents the settings endpoints, the resolver, and the exact
rewired call sites in docs/design/curation_api_contract.md.
…ion + durable export tracking Reconciles job-state files claiming 'running' with no live process on startup (item_scores, selection, embedding_viz, autolabel), and gives model_export.py the same file-backed durability pattern the training jobs already use, closing the two real gaps found in an owner-requested persistence audit. Developed in parallel with the shared-settings feature in a separate worktree off the same base commit; merged sequentially.
Adds a durable, single-document OpenSearch-backed settings store and GET/PUT /curation/settings, with GET /methods' per-axis 'default' flag and the real cluster/sort default-application call sites now DERIVED from the same resolver — a shared default can never drift from actual server behavior. Cross-repo API shape agreed with the Cropwright frontend team before implementation. Developed in parallel with persistence hardening in a separate worktree off the same base commit; merged sequentially.
Until now no curation write endpoint had ever been executed against a live stack: the offline suite covers all 47 of them with mocked clients, which cannot catch a mis-wired env var, a mapping that rejects a real document, a query that silently matches nothing, or a job file nobody writes. Adds a disposable compose project (`op-live-verify`) with four containers — OpenSearch, this repo's API image, a stdlib OpenAI-compatible fake VLM, and a shell fake trainer implementing the trainer half of the job-file protocol. No GPU, no Triton, no docker socket. Deliberate choices, each fixing a concrete defect in an earlier ad-hoc attempt at this: * `OPENSEARCH_URL` (what the app actually reads) rather than `OPENSEARCH_HOSTS` (the dashboards variable). * every index name `verify_`-prefixed, enforced by the first fixture in tests/live/conftest.py, which also refuses any endpoint outside the 147xx harness port band. * the registry, source images, exports, state dir and jobs dir all mounted from one host-visible directory, so assertions can read the bytes that were written instead of trusting a response envelope. * `OP_SCORES_ENABLED` / `OP_SELECT_DIVERSE_ENABLED` / `OP_VIZ_PROJECTION_ENABLED` set, since three write endpoints are disabled by default and would otherwise only ever return 400. * `--workers=1`: several cancel endpoints and the SSE hub keep in-process state, so with two workers a cancel POST can land on a different worker than the start POST. * `OP_API_PREFIX=/kb`, so the harness serves byte-identical paths to what an existing labeler frontend already calls. `docker/test/` (not the repo root) because .gitignore ignores every root-level `docker-compose.*.yml`. pyproject.toml gains a `live` marker and `-m 'not live'` in addopts, so the default suite and the offline gate are unchanged.
Seeds the harness through the application's own helpers rather than hand-rolled mappings: `_ensure_indexes` (the real bootstrap chain, with the one intentionally-unwired migration correctly skipped), `ClassRegistry.add_class` before any label write, then ~490 item docs with `_id == crop_id` covering all three cluster-id bands. Embeddings are non-degenerate by construction: one random unit centroid per intended sub-cluster, members drawn as `centroid + N(0, sigma)` and re-normalized. sigma is scaled `0.15 / sqrt(dim)` so the noise vector's norm is ~0.15 against a unit centroid at any dimensionality — a flat 0.15 per component would give a 1024-d noise vector norm ~4.8, swamping the centroid and collapsing every group into mutual near-orthogonality, which would make the refine assertions pass vacuously. Scenarios (43 tests here), all asserting on stored state rather than the response envelope: * label single/batch with `class_id_history` growth; unknown-class rejection with `_seq_no` unchanged; unlabel; review-dismiss; exclude/unexclude round trip; move; flag-new-class; and two parallel labels of one item proving OCC leaves no torn history entry. * cluster refine asserting >= 2 distinct sub-ids via a terms aggregation, refine idempotence, and the member-count floor. * auto-promote dry-run asserting `_seq_no` unchanged, then apply asserting only the high-purity cluster is validated. * region KMeans partition, per-bucket AHC refine, the permanent false-positive bucket surviving a re-partition, FP centroid build and the suspected-FP matcher. * scores compute/coverage/cancel, diverse select asserting no mutation, UMAP viz rebuild + serve. * registry create/rename/hotkey/sync, shared-settings round trip, holdout freeze, class-merge refusal when frozen holdout rows would be relabelled (before the successful merge), and merge success across both the items and confirmed-labels indexes. Two xfail(strict) tests document real defects this found rather than weakening an assertion around them: the hardcoded `op_umap_viz_state` / `op_umap_state` index names that bypass CurationConfig, and (in the following commit) the VLM verdict-key mismatch.
Covers the three cohorts that need a counterparty rather than just
OpenSearch, each driven against a real one:
VLM (cohort c) — label_batch asserting the exact fixed answer reaches
`class_name` / `class_source` / `vlm_raw_label`, the human-owned-crop
skip, the 64-item cap, single and batched region verify, and a
visibility batch whose fake returns an explicit negative (that endpoint
fails open to True, so only a negative reply distinguishes "understood"
from "ignored").
Training (cohort d) — preflight asserting NO file appears in the jobs
dir; start asserting the job file appears and that `docker ps` is
byte-identical either side of the call, which is what proves the GPU
arbiter is a genuine no-op and that nothing needs the docker socket;
the fake trainer driving queued -> running -> finished; status, runs
list, log tail, manifest; cancel-sentinel semantics; campaign submit +
cancel-campaign.
Export (cohort e) — label-count and symlink assertions read from disk,
determinism across two version tags at the same seed, the frozen
artifact serve (byte-for-byte against the file, including the
`class_registry.json` dense export-id map that used to have readers and
no producer), and 404s for a non-whitelisted artifact name.
Closes with a restart-durability check that recreates only the API
container and re-reads the classes, holdout, exports and per-item state.
One xfail(strict) records an application bug this harness found: the
built-in prompt pack asks the model for `is_region` while both region
parsers in vlm_labeler.py read `is_plate`, so a model that follows the
shipped prompt exactly is parsed as a negative verdict with
`reason='parse_failure'`. The assertion states the correct behaviour and
will fail loudly the day the mismatch is fixed.
docker/test/README.md lists what remains unverifiable without a GPU:
train/promote, DELETE models/{name}, pipeline/auto_label(+/start), and
the evaluation half of bakeoff/run.
GENERIC_ITEM_PACK.region_user/region_batch_user (vlm_prompts.py) ask the model to answer with `is_region`, but _parse_plate_response and _parse_plate_batch_response (vlm_labeler.py) read `is_plate` instead — a model following the shipped prompt exactly was parsed as a negative verdict with reason='parse_failure'. git blame shows both sides landed in the same commit (574b957); there is no comment or design note treating `is_plate` as an intentional back-compat key, so this is accidental drift, not a dual-key design. Renamed the internal VlmRegionVerdict field and every parser/call-site reference from is_plate to is_region (matching the file's existing generic naming — RegionCrop, VlmRegionVerdict) rather than accepting both keys. The HTTP wire model in _common.py (VlmVerifyRegionBatchResult .is_plate) is a separate, genuinely frozen wire field per curation_api_contract.md and is untouched; it also turned out to be dead code (the live route uses its own same-named class in vlm.py, whose field is already is_region). Updated the fake VLM harness default + the two live tests that had pinned region_verdict_key to the buggy 'is_plate' value to match, and removed the xfail from test_vlm_verdict_key_matches_the_shipped_prompt.
UMAP_VIZ_STATE_INDEX (embedding_viz.py) and UMAP_STATE_INDEX (clustering/embedding_reduce.py) were hardcoded 'op_...' literals, bypassing CurationConfig — every other index name in this codebase resolves through it, but these two silently ignored a deployment's OP_*_INDEX renames. Added two new CurationConfig fields, umap_state_index and umap_viz_state_index (defaults unchanged: 'op_umap_state' / 'op_umap_viz_state'), each overridable via its own OP_*_INDEX env var through from_env(). They stay as two separate fields rather than one shared role because the module docstring in embedding_viz.py is explicit that these are deliberately distinct indexes (the retired clustering reducer's fitted-manifold cache vs. the viz-only projection's own metadata slot) that must never share a name or state. Documented both in env.template, wired the harness compose file to verify_-prefixed overrides, and removed the now-fixed gap's xfail marker + the safety-guard allowlist in tests/live/conftest.py that used to tolerate these two indexes appearing unscoped.
OCCFinalConflictError's docstring (src/clients/occ.py) claims human write endpoints surface it as HTTP 409, and several curation router call sites (regions.py, crops.py) do a bare 'except OCCFinalConflictError: raise' specifically to bypass their own generic-exception 404 fallback — that pattern only makes sense if a dedicated handler picks the re-raised exception back up. No such handler was registered; src/main.py only had the generic Exception -> 500 handler, so an exhausted-OCC-retry conflict was surfacing as an opaque 500 instead of the documented 409. Added @application.exception_handler(OCCFinalConflictError) ahead of the generic handler, returning 409 with doc_id/retries/request_id. Docstring was accurate about intended behavior, so fixed the code rather than the docs. Added tests/test_occ_conflict_handler.py, which builds a fresh app via create_app() (no lifespan triggered) with a throwaway route raising the exception and asserts the 409 mapping.
…re contract
GET /crops/{id} returned the raw OpenSearch _source directly, so its
JSON keys followed RegionFields (region_* by default) instead of the
frozen ItemDoc plate_* contract. Route it through the same
_src_to_crop_doc() translation GET /crops (list) already uses, and
extend ItemDoc with the plate_status/plate_text/plate_detector/
plate_verified/etc fields it was missing (PATCH .../plate_meta writes
them but nothing on the GET side ever returned them).
PATCH /crops/{id}/plate_meta's `updated_fields` echoed sorted(doc.keys()) --
internal RegionFields storage keys -- instead of the request's plate_*
wire names. Track the wire names explicitly as each field is applied.
GET /review/{tab} built its per-item response dict using `fields.X` as
the literal JSON key (not just to select which storage key to read),
so every region field it returned carried a region_* key by default.
Replaced with literal plate_* keys, matching the already-correct
pattern in regions.py's _region_item().
Found via a Cropwright (labeler frontend) integration test that wrote
plate_text/plate_status via PATCH and got region_* back on re-fetch --
not a supersession of the plate_* wire-contract decision, a bug in it.
Also fixes check-no-literal-region-fields' wire-key allowlist regex,
which only matched RegionFields instances bound to F -- review.py binds
the same instance to `fields`, so its legitimate
'plate_status': src.get(fields.status) wire-serializer lines tripped a
false positive. Also recognizes the bool(...) wrapper and a
wire_fields.append('plate_foo') bookkeeping pattern.
Adds docs/CURATION.md — the previously-missing user-facing guide for the experimental curation subsystem: what it is, the four config dataclasses (CurationConfig, RegionFields, DetectionProfile, and RegionStatus which landed in Wave 5), the full OP_* env var table sourced from env.template, the class-registry schema with the non-vehicle warehouse/pallet worked example, which models a deployment must supply (BYO-model), the curation Compose profile and its workers, the seed/bootstrap path for a fresh install, and known gaps stated up front. Also updates docs/design/curation_design_rationale.md (was stale: described three dataclasses instead of four, claimed no ingest-write route existed and no worker container shipped, both since fixed by earlier waves) and docs/design/curation_api_contract.md: - Documents the real wired VlmVerifyRegionBatchResult.is_region field instead of a dead is_plate duplicate that lived unused in src/routers/curation/_common.py (deleted separately; nothing else imported it) — the doc and the code now agree on what's real. - Adds the 11 plate_* round-trip fields ItemDoc gained for the PATCH/PUT plate_meta round-trip. Verified live counts rather than trusting the plan text: 109 routes under /curation (25 route groups), not the plan's original 103/23 — the tree moved across five waves since the plan was drafted.
README.md: adds a Curation & Active Learning section (25 route groups, 109 routes verified live, not the plan's stale 103), fixes the nonexistent /health/models reference (there is no such route — /health and /ready are the real ones), links docs/CURATION.md, and stops telling readers to 'source .venv/bin/activate' in favor of calling venv binaries directly, per this project's own stated convention. CLAUDE.md: same source-activate fix in three spots, and fixes the false claim that all API endpoints have a /v1 twin — verified live that none of the 109 /curation routes do. docs/ARCHITECTURE.md: strips 'private, domain-specific reference implementation' and 'Bucket B' vocabulary, rewrites the stale 'intentionally thinner than the reference' section (ingest write routes and worker containers both exist now), adds the curation Compose profile's runtime-companion topology, and lists RegionStatus as the fourth config dataclass. docs/README.md, scripts/README.md: refresh stale claims — a nonexistent check_services.sh (openprocessor.sh status covers it), a 'make test' target that now exists (Wave 0), documents scripts/curation/ which was previously unmentioned, links the new SECURITY.md/ CONTRIBUTING.md/CURATION.md, and fixes the 2026-01-27 'Last Updated' date. src/README.md, docs/FACE_RECOGNITION_IMPLEMENTATION.md: same source-activate fix, found while sweeping the tree for the pattern.
1100+ tests and 27 pre-commit hooks had zero CI enforcement. Two jobs, triggered on PR and push to main: - pytest: installs CPU torch/torchvision from the PyTorch wheel index plus requirements-test.txt (Wave 1's CPU-installable dependency subset), then runs the import sanity check and the full offline suite (-m 'not live'). - pre-commit: runs every hook exactly as a contributor would locally. Installs golangci-lint and hadolint explicitly since two hooks in .pre-commit-config.yaml are 'language: system' and expect those binaries on PATH (golangci-lint/go-build only fire on benchmarks/*.go, but --all-files means they do fire in CI); caches pip and the pre-commit hook environments.
SECURITY.md: disclosure contact (GitHub private vulnerability reporting)
plus a prominent, explicit statement that the API has no authentication
and must not be internet-exposed — it serves DELETE /query/image/{id},
DELETE /curation/models/{model_name}, and POST /ingest/directory
(arbitrary server-side path read). Verified against the shipped
docker-compose.yml: Grafana really does ship admin/admin
(GF_SECURITY_ADMIN_USER/PASSWORD) and OpenSearch really does ship
DISABLE_SECURITY_PLUGIN=true.
CONTRIBUTING.md: dev setup (venv, pre-commit install — direct binary
calls, never source-activate), how to run the offline suite and the
Wave 6 live write-path harness, what the
check-no-literal-region-fields ratchet is and why (including the
PORTED_PATHS allowlist mechanism), conventional-commit format, and this
repo's merge-commit-never-squash convention.
CODE_OF_CONDUCT.md: standard Contributor Covenant v2.1.
.github/: ISSUE_TEMPLATE/ (bug report, feature request, a security
contact-link redirect to private reporting), PULL_REQUEST_TEMPLATE.md,
dependabot.yml (pip/github-actions/docker, weekly), CODEOWNERS
(wildcard placeholder — no existing per-path ownership convention in
this repo to preserve).
CHANGELOG.md: fleshes out [Unreleased] with every wave's user-visible
change — the curation subsystem and its experimental gating, ingest
endpoints, the export/training artifact-chain fix, the curation Compose
profile and runtime companions, class-selectable labeling assist,
shared curation deployment settings, persistence hardening, the live
verification harness, the new CI job, and the AGPL re-badge.
This repo was MIT-badged while genuinely vendoring an AGPL-3.0 Ultralytics fork under src/ultralytics_patches/ (LICENSE's own third-party section already flagged this, at the old :23-58) — a real licensing defect (D-12/D9), not just wording. AGPL-3.0's copyleft terms propagate to the combined work, so the whole repository is re-badged AGPL-3.0-or-later rather than isolating or replacing the vendored fork. - LICENSE: full AGPL-3.0-or-later text, third-party component section retained (third-party licenses — BSD, Apache-2.0, MIT, etc. — are correct as they stood and untouched). - pyproject.toml: license field and classifier updated. - README.md: adds a License section; ATTRIBUTION.md: 'This Project' section updated (third-party attribution rows for InsightFace/ OpenCLIP/etc. correctly stay MIT — those are the actual upstream licenses of code this project depends on, not a project claim). - Dockerfile, Dockerfile.triton, docker/evaluator/Dockerfile: org.opencontainers.image.licenses OCI labels updated to AGPL-3.0-or-later. Grepped the whole tree for \bMIT\b (word-boundary, not substring) and fixed every genuine first-party claim; left every third-party attribution (InsightFace, OpenCLIP, the SthPhoenix reference note) untouched since those really are MIT upstream.
…t fix test_patch_region_metadata_writes_text_and_source asserted 'region_text' in the PATCH response's updated_fields -- that was encoding the bug c7277d1 just fixed (updated_fields must report the plate_* wire contract, never the RegionFields storage key). The raw OpenSearch _source assertions on region_text/region_text_source/etc a few lines below are correct as-is (that's real storage verification, not the HTTP contract) and are left untouched.
PUT /curation/settings required every submitted value to be a currently-advertised id for that axis, so once a shared default was pinned there was no way back to "each endpoint uses its own tuned default" -- for `sort` specifically, that per-tab-default state became permanently unreachable through the API (raised by the Cropwright settings-UI integration pass). A null value for an axis now always validates and clears that axis's stored override. OpenSearch's partial-doc merge sets the nested field to a literal null rather than deleting the key, so get_curation_settings filters None entries out of the returned defaults map -- a cleared axis is indistinguishable from one that was never set.
a9d345f removed docs/security/ and docker/hardened/ wholesale as an internal DeepStream investigation unrelated to this product. On review, half of that content was genuinely Triton-specific and belongs here -- this product ships and depends on Triton directly, and the Tier 2 hardening (purge the compile-time toolchain -- not yet applied to the production Dockerfile.triton, which only does Tier 1 + Nsight removal) is real, actionable, verified (0/0 CVE scan) guidance for anyone who needs a stricter scan gate than the production image provides. Restores, trimmed to Triton only: - docker/hardened/triton/Dockerfile, docker/hardened/test/* (build_scan, test_triton, make_test_model, infer_check) -- the DeepStream Dockerfile, trivyignore, and test_deepstream.sh are NOT restored. - docs/security/triton_cve_hardening.md -- rewritten from the removed combined doc, DeepStream sections cut, Triton content otherwise intact. - README.md trimmed to the Triton-only layout/results/commands. The DeepStream half is not lost -- the run_deepstream project's own docs/security/cve-remediation.md already cites this repo's original combined doc as its source methodology and has since gone further (per-image baseline scans, its own remediation log), so nothing needed re-sending there. Also drops docker/hardened/ from .git/info/exclude (triton-api's shared excludes file) -- it had no comment explaining why it was there, was already tracked despite it, and blocked staging this restoration.
… registry gap Found during a live-harness E2E pass: GET /methods advertises exactly one built-in detection_profile, named license_plate with detector_model=lpr_nanov11_640 (cascade_detect.py's DEFAULT_PROFILE) -- correct and intentional (backward compat for the reference LPR deployment this codebase still serves), but not obviously so to a fresh generic deployment reading the BYO-model section. Also documents a real, separate gap noticed while explaining this: OP_DETECTION_* env vars configure the profile ingest actually uses, but never call register_profile() themselves, so a deployment's own profile never appears on GET /methods without a small amount of startup code -- there's no config-driven auto-registration yet.
Finalizes the OSS-hardening completion plan (waves 0-8): moves CHANGELOG's [Unreleased] section to [0.3.0] - 2026-09-21, adding the region_*/plate_* wire-contract fix, the settings clear-axis fix, and the docs/security + docker/hardened DeepStream removal (Triton hardening kept) that landed after the Wave 7 documentation pass. Bumps VERSION and pyproject.toml's version field to match, and updates the three example API-response snippets (CLAUDE.md, README.md, docs/README.md) that hardcoded the old "0.2.1" version string -- src/main.py reads VERSION at runtime, so these were about to go stale.
Full genericization of the curation subsystem plus completeness pass per docs/design/oss_main_completion_plan.md: deps/config, the ingest gap, export-to-training artifact chain, deployment (Compose profile + runtime companions), test restoration, a live write-path verification harness (docker/test/), documentation/CI/OSS furniture, and AGPL-3.0-or-later relicensing (D9, vendors an AGPL Ultralytics fork). Plus a joint end-to-end verification pass with the Cropwright labeler frontend against that live harness, which surfaced and fixed two real bugs: a region_*/plate_* wire-contract leak (GET/PATCH crop endpoints were returning internal RegionFields storage keys instead of the frozen ItemDoc wire contract) and a shared-settings axis that could be pinned but never cleared back to per-endpoint defaults. 1121 offline tests + 74 live write-path scenarios passing, 27/27 pre-commit hooks, zero leaked references across the full history diff. See CHANGELOG.md's [0.3.0] entry for the itemized list.
Updates the requirements on [py-spy](https://github.com/benfred/py-spy) to permit the latest version. - [Release notes](https://github.com/benfred/py-spy/releases) - [Changelog](https://github.com/benfred/py-spy/blob/master/CHANGELOG.md) - [Commits](benfred/py-spy@v0.3.14...v0.4.2) --- updated-dependencies: - dependency-name: py-spy dependency-version: 0.4.2 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
Author
LabelsThe following labels could not be found: Please fix the above issues or remove invalid values from |
Author
|
OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting If you change your mind, just re-open this PR and I'll resolve any conflicts on it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the requirements on py-spy to permit the latest version.
Release notes
Sourced from py-spy's releases.
Changelog
Sourced from py-spy's changelog.
... (truncated)
Commits
d230d82Bump version: 0.4.1 → 0.4.248af4e5Minor code cleanups (#844)13c8863Add support for native extensions on linux aarch64 (#779)2982dabUse Py_Version symbol for detecting the python version (#835)c478370Fix getting symbols from OSX universal binaries (#843)344703dSet github actions workflow permissions (#840)f84084aFix intermittent OSX CI errors (#842)d7d22b6Show backtrace on errors when RUST_BACKTRACE=1 environment variable is set (#...4580345Fix corruptedsubprocessoutput (#832)39bbc40Fix release drafter config (#839)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)