Skip to content

Add timeh2o-v1-tiny wrapper and fev-bench submission - #201

Merged
shchur merged 1 commit into
autogluon:mainfrom
Far0n:add-timeh2o-v1-tiny
Oct 14, 2026
Merged

shchur merged 1 commit into
autogluon:mainfrom
Far0n:add-timeh2o-v1-tiny

Conversation

@Far0n

@Far0n Far0n commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Hi,

We'd like to submit TimeH2O-v1-tiny to fev-bench.
It's a 5.8M-parameter univariate time-series foundation model that runs through the h2ofm package.

On the full benchmark it gets a SQL skill score of 45.5, with a median runtime of 0.19 s per 100 series on a single RTX 5090.

This PR adds:

  • models/timeh2o-v1-tiny/: the wrapper and requirements.txt (h2ofm==0.42.0)
  • benchmarks/fev_bench/results/timeh2o-v1-tiny.csv: results for all 100 tasks
  • an entry in models.yaml

To reproduce:

uv run python models/evaluate.py -m timeh2o-v1-tiny -b benchmarks/fev_bench/tasks.yaml

A few notes:

  • The weights are gated on Hugging Face, so access has to be requested on the model page before running.
  • h2ofm requires Python 3.12 or newer (we used 3.14).
  • This checkpoint forecasts each target independently and does not use covariates.

The submission tests pass locally, and the results match our own internal fev-bench run.

Thanks for maintaining the benchmark!

Cheers,
Far0n

@shchur

shchur commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Great results, thanks for the contribution!

@shchur
shchur merged commit 83d347d into autogluon:main Oct 14, 2026
3 checks passed
@Far0n

Far0n commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Great results, thanks for the contribution!

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants