Skip to content

[8.19] [ML] Retry 3rd-party git clones and propagate clone failures (#3233) - #3236

Open
edsavage wants to merge 1 commit into
elastic:8.19from
edsavage:ml/backport-3233-8.19
Open

edsavage wants to merge 1 commit into
elastic:8.19from
edsavage:ml/backport-3233-8.19

Conversation

@edsavage

@edsavage edsavage commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Manual backport of #3233 to 8.19.

The automatic backport could not be created because 3rd_party/CMakeLists.txt conflicts: main has a Linux-only Sandbox2/Abseil block in this file that does not exist on 8.19. That block is unrelated to this change and has been omitted; only the intended change is applied here:

  • Add COMMAND_ERROR_IS_FATAL ANY to the Eigen and Valijson execute_process calls so an exhausted-retry FATAL_ERROR in the child cmake -P script stops configuration immediately instead of surfacing later as a cryptic missing-header compile error.
  • Factor the bounded retry+backoff clone loop into a shared ml_clone_git_dependency() helper in the new cmake/clone_git_dependency.cmake module (kept out of cmake/functions.cmake so it is safe to include() from cmake -P script mode).
  • Use the helper from pull-eigen.cmake and pull-valijson.cmake.

Verified: resolved 3rd_party/CMakeLists.txt contains no sandbox/abseil references; the helper parses under cmake -P; diff limited to the four intended files.

Made with Cursor

…#3233)

The Eigen and Valijson sources are cloned at CMake configure time from
gitlab.com and github.com respectively. Those hosts occasionally return
transient errors (e.g. GitLab "currently unable to handle this request
due to load"), and a single failed clone was enough to break an entire
CI build, requiring a manual rebuild.

Wrap each clone in a bounded retry loop (5 attempts, increasing backoff)
that starts from a clean slate on every attempt, so a brief hosting
outage no longer fails the build.

Also propagate the failure from the outer execute_process() calls that
run these scripts. Previously the FATAL_ERROR raised inside the child
`cmake -P` process was swallowed: configure logged the error but
continued with an empty 3rd_party/eigen, so the failure only surfaced
much later as a cryptic "Eigen/Core: No such file or directory" compile
error. COMMAND_ERROR_IS_FATAL ANY makes configure stop immediately with
the clear message once retries are exhausted, finally delivering the
behaviour elastic#3164 intended.

(cherry picked from commit bcce4aa)
@elasticsearchmachine

Copy link
Copy Markdown

Pinging @elastic/ml-core (Team:ML)

@edsavage edsavage added v8.19.24 Release version v8.19.24 and removed v8.19.23 labels Oct 8, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>build :ml v8.19.24 Release version v8.19.24

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants