Skip to content

Improve AnomaVision vs Anomalib benchmark timing - #188

Merged
DeepKnowledge1 merged 2 commits into
mainfrom
fix/anomalib-model-device-placement
Oct 27, 2026
Merged

DeepKnowledge1 merged 2 commits into
mainfrom
fix/anomalib-model-device-placement

Conversation

@DeepKnowledge1

@DeepKnowledge1 DeepKnowledge1 commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

🔗 Related Issue

Fixes #

📝 Description

  • Add end-to-end latency measurement to the AnomaVision vs. Anomalib benchmark, separating model-only inference latency from full-pipeline latency.
  • Handle Anomalib ImageItem samples without relying on PyTorch's default DataLoader collation, avoiding the reported collation error during timing.
  • Include end-to-end latency in benchmark reports while preserving timing anomaly detection based on outlier fraction and coefficient of variation (CV).

🔄 Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation update
  • 🏗️ Infrastructure / CI/CD update

🧪 Hardware & Matrix Testing

I have successfully built and tested this code using uv on:

  • anomavision[cpu] (Standard/Edge)
  • anomavision[cu121] (CUDA 12.1)
  • anomavision[cu124] (CUDA 12.4)
  • anomavision[cu118] (CUDA 11.8)

Host OS used for testing:

  • Linux / Ubuntu
  • Windows (Native or WSL2)
  • macOS

✅ Developer Checklist

  • My code follows the core style guidelines of this project (Ruff/Black formatting).
  • I have run uv run pytest and all unit tests pass locally.
  • Lockfile Guard: If I added or modified a dependency in pyproject.toml, I have run uv lock --python 3.10 and committed the updated uv.lock file.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have updated the documentation accordingly (if applicable).

📸 Screenshots / Visual Proof (Optional)

Benchmark reports include the new end-to-end latency metric alongside model-only latency and timing stability results.

DeepKnowledge1 and others added 2 commits October 27, 2026 12:36
- Add end-to-end latency measurement
- Handle Anomalib ImageItem samples without default DataLoader collation
- Separate model inference latency from full-pipeline latency
- Include end-to-end latency in benchmark reports
- Preserve timing anomaly detection based on outlier fraction and CV
- Keep timing stability and pipeline overhead as separate metrics
@DeepKnowledge1
DeepKnowledge1 merged commit 89b9186 into main Oct 27, 2026
5 checks passed
@DeepKnowledge1
DeepKnowledge1 deleted the fix/anomalib-model-device-placement branch October 27, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant