Skip to content

Standard: no --retries for an attempt that fails on the infrastructure or the agent, and --repeat is not named --attempts #152

Description

@aitianshi

Priority

extreme

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • catalogue (benchmarks, agents, capabilities)
  • the standard (ports, kits, relay, runs)
  • CI, workflows and skills

Today

An attempt is one run of a task, counted whatever its result; a retry runs an attempt again after the infrastructure or the agent failed, never because the task was failed. Harbor has both: n_attempts, 1 by default, and max_retries, 0 by default, which retries any exception but those its RetryConfig excludes (timeouts, refusals, a wrong model or key, a usage limit, the verifier's own reward errors).

The standard has --repeat K for attempts and one fixed retry: a unit that ended rate-limited runs once more after the others (#90). An agent crash, a provider error or a setup error is never retried, and once a run ends no page says an attempt was retried. The builder calls Harbor's attempts "Repeats per task" and its retries "Retries per trial", 3 by default.

Wanted

  • cli run --retries N, 0 by default, by Harbor's retry rule, and --attempts K as the name of --repeat K.
  • The builder names both settings alike for both runners: Attempts per task (or unit) and Retries per attempt.
  • A retried attempt says so on the pages.

Acceptance

  • cli run --retries N (0 to 9, default 0) runs an attempt again, up to N times, only when it ended in a retryable failure: a rate limit, a provider or network error, a setup error, an agent crash, an output-length error, a verifier error. Never an agent or verifier timeout, a refusal, a wrong model or key, a missing secret, a usage limit, the spending cap (the run's or the unit's), a stopped run, or a task the agent failed.
  • Retries run after the other units, one at a time and paced; nothing is retried once the run is stopped or halted, and the spending cap counts every try.
  • Each earlier try is kept whole under the job's retried/, numbered, where no reader takes it for a trial; the kept trial's result.json says how many retries it took, and config.json records retries.
  • cli run --attempts K runs each unit K times; --repeat K still works.
  • A standard recipe carries retries; a Harbor recipe's retries default to 0; the builder labels the settings "Attempts per task" (or unit) and "Retries per attempt" for both runners; a saved recipe opens as it was saved.
  • A retried attempt says so on the run page, the trial page, All attempts and the live board.
  • docs/standard.md, docs/cli/run.md, docs/builder.md and the field guide describe both settings.
  • Tests cover the retry rule, the rounds of retries, the kept tries, a stop, the recipe keys and the pages.

Dependencies and related

Activity

  1. self-assigned this
    on Nov 16, 2026
  2. added
    enhancementNew feature or request
    P2Planned: limited in scope, has a way around it, or a missing feature
    on Nov 16, 2026
  3. changed the title [-]Standard: no --attempts beside --repeat, and "attempt" names a repeat in the builder, recipes and driver protocol[/-] [+]Standard: no --retries for an attempt that fails on the infrastructure or the agent, and --repeat is not named --attempts[/+] on Nov 16, 2026
  4. added a commit that references this issue on Nov 18, 2026
    49d32d1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P2Planned: limited in scope, has a way around it, or a missing featureenhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions