Skip to content

Add signed option to ope() for percentage bias - #3319

Merged
dennisbader merged 3 commits into
unit8co:masterfrom
COMPUMAX-EC:feature/signed-ope
Oct 29, 2026
Merged

dennisbader merged 3 commits into
unit8co:masterfrom
COMPUMAX-EC:feature/signed-ope

Conversation

@COMPUMAX-EC

Copy link
Copy Markdown
Contributor

Checklist before merging this PR:

  • Read and understood the contribution guidelines including AI/agent usage.
  • Linked at least one approved issue below (an issue with label ok).
  • Summarized the updates of this PR under Summary.
  • Added an entry under Unreleased in the Changelog.

Fixes #3310.

Summary

Adds a signed: bool = False parameter to darts.metrics.ope(). With signed=True, the absolute value is not taken, so the metric returns the signed overall percentage error (percentage bias). The default behavior does not change.

  • The sign follows the existing ope formula, (Σy − Σŷ) / Σy: positive means under-forecasting and negative means over-forecasting. If you'd rather have positive mean over-forecasting, as the issue suggests, it's a one-line change.
  • Zero-division handling, quantile and stochastic predictions, multivariate series and sequences of series all work as they do for the unsigned metric.
  • A new test, test_ope_signed, covers both signs, the default, multivariate and multi-series inputs, stochastic forecasts with q, and zero division.

Other Information

  • pytest darts/tests/metrics/test_metrics.py: 487 passed locally (Windows, Python 3.12). Ruff 0.7.2 check and format --check pass.
  • AI use: I used Claude Code to draft the implementation, tests and docstring, and to run the checks. I reviewed and verified every change and can explain the design.

@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.15%. Comparing base (580358e) to head (127cbed).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3319   +/-   ##
=======================================
  Coverage   97.15%   97.15%           
=======================================
  Files         175      175           
  Lines       19468    19469    +1     
=======================================
+ Hits        18914    18915    +1     
  Misses        554      554           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dennisbader dennisbader left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR @COMPUMAX-EC 🚀

Everything looks good to me 💯
I simplified the tests to reuse the reference OPE.

@dennisbader
dennisbader merged commit c828130 into unit8co:master Oct 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a signed percentage bias metric (or signed=True option in ope) to darts.metrics

2 participants