Skip to content

FOS reports nothing to the server when imaging fails #1206

Closed

Description

@mastacontrola

Found while doing #1202. That issue is about two notification events that had no caller; this one is the reason the failure half of it can only ever be partly solved in this repository.

FOS does not tell the server when imaging fails

handleError() in Buildroot/board/FOG/FOS/rootfs_overlay/usr/share/fog/lib/funcs.sh prints a banner to the console, optionally sleeps, and exit 1s. It makes no request to the server. Verified in FOGProject/fos at a100b2d:

$ grep -c 'curl' <handleError body>      # 0

The only failure FOS does report is a storage-node problem, via fog.checkmount → service/blame.php, and that is a recoverable one — Blame records a NodeFailure and re-queues the task on another node.

So a genuine imaging failure — bad image, mount failure, partition error, disk too small, a handleError from any of the ~40 call sites across fog.download, fog.upload, fog.mount and friends — is invisible to the FOG server. Consequences:

  • HOST_IMAGE_FAIL cannot fire for the failure people actually mean. Make imaging tasks announce the outcome they actually had #1205 wires it to TaskQueue::checkout()'s catch, which covers "imaging ran and FOG then failed to record it" — a real case, but not this one.
  • Nothing in the web UI or the API can say a task failed, because nothing knows.
  • Needs confirming: what happens to the task row. The checkin timeout (Task::isExpired() / expireTaskCheckin()) governs queue position and re-queues tasks that are waiting in line; I did not establish what, if anything, acts on a task already in the Progress state whose FOS has exited. If the answer is "nothing", a failed deploy sits as In Progress indefinitely, which is a bigger problem than the missed notification.

Shape of a fix

  1. A service endpoint here — service/error.php or similar — that takes the host identity FOS already sends everywhere else (mac + sysuuid), plus the error text and the script that raised it, and:
  2. handleError() posts to it before its exit 1, best-effort — a failure to report must not change what FOS shows the person standing at the machine.

Worth deciding before building

  • This is a FOS change, so it needs a FOS build and release, not just a web-tier deploy. Servers on an older FOS get nothing until they update, which is an argument for the endpoint being additive and tolerant of never being called.
  • The endpoint is unauthenticated in the same way every other service/*.php is — identified by MAC and sysuuid, reachable by anything that can reach the web tier. It accepts free text that lands in a notification and probably in a log or a task record. That is a real surface and wants deciding deliberately rather than by analogy with the existing endpoints.
  • Does a failed task become Cancelled, stay In Progress, or get a new state? There is no Failed state today (taskStates is Queued / Checked In / Progress / Complete / Cancelled). Adding one is a schema change with UI and API consequences; reusing Cancelled loses the distinction between "an admin stopped this" and "this broke".

Not started. Filed so the gap is written down rather than living in a PR description.

Activity

  1. added
    ready-for-humanNeeds hardware/env repro, infra access, or a product/design decision
    on Aug 19, 2026
  2. mastacontrola commented on Aug 19, 2026

    @mastacontrola
    MemberAuthor

    Done on both sides.

    FOS — FOGProject/fos@a89dec1ad, "Report failures and warnings to the server, tagged with which they are" (fos#152). That commit changes funcs.sh, which is where handleError() lives, and ships tests/checks/error-report.sh alongside it. This issue was filed against a100b2d, which is the commit immediately before it.

    Server — packages/web/service/taskerror.php receives the report, and the storage and display of it landed in #1217 (a report longer than the notification it triggers), #1223 (line breaks kept, escaped on the way out), #1226 (say why imaging failed, not just that it did) and #1227 (pin the plugins release that reports it).

    So handleError now tells the server, the reason is stored typed rather than flattened into a notification, and HOST_IMAGE_FAIL fires for the failure people actually mean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugready-for-humanNeeds hardware/env repro, infra access, or a product/design decision

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions