diff --git a/.github/actions/fetch-models/action.yml b/.github/actions/fetch-models/action.yml new file mode 100644 index 000000000..ab957c0e8 --- /dev/null +++ b/.github/actions/fetch-models/action.yml @@ -0,0 +1,34 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# Downloads the models a job needs from Hugging Face -- the one place a job gets its models. A job +# names the features it needs (the feature column of .github/models.csv) and gets exactly their +# files; .github/fetch-models.sh pins each to a Hugging Face commit and checks its SHA-256, so a +# missing or changed model fails here instead of letting a model-backed test self-skip. The token +# makes the downloads count against the account's rate limit instead of the runner IP's shared one +# (which answered 429); without it (fork pull requests) the download is anonymous. There is no +# GitHub Actions cache in between any more: one 6.1 GB entry restored by about 20 jobs per run went +# over the account's cache egress ("Egress is over the account limit") and the repository's 10 GB. +# Needs the repository checked out. +name: Fetch GGUF models +description: Download the models of the named features of .github/models.csv from Hugging Face and verify their SHA-256. +inputs: + features: + description: Space-separated features of .github/models.csv (draft, tts, ...); empty means every model. + required: false + default: '' + hf-token: + description: Hugging Face token (secrets.HF_TOKEN); empty downloads anonymously. + required: false + default: '' +runs: + using: composite + steps: + - name: Download and verify the models (.github/models.csv) + shell: bash + env: + HF_TOKEN: ${{ inputs.hf-token }} + FEATURES: ${{ inputs.features }} + # $FEATURES unquoted on purpose: one argument per feature. + run: bash .github/fetch-models.sh $FEATURES diff --git a/.github/actions/restore-models/action.yml b/.github/actions/restore-models/action.yml deleted file mode 100644 index 815028476..000000000 --- a/.github/actions/restore-models/action.yml +++ /dev/null @@ -1,33 +0,0 @@ -# SPDX-FileCopyrightText: 2026 Bernard Ladenthin -# -# SPDX-License-Identifier: MIT OR Apache-2.0 - -# Restores the shared GGUF model cache and validates it -- the one place a job gets its models. -# The download-models job is the cache's ONLY writer (actions/cache, same key); every consumer -# uses this restore-only action, so a job running on a cache miss can never re-save an empty or -# partial entry under the immutable key. enableCrossOsArchive matches the writer's entry version, -# so macOS and Windows restore the same ubuntu-built entry (without it the entries are per OS, and -# the unreachable Windows-side one was once found re-saved EMPTY after an eviction). The key is -# the hash of .github/models.csv, so editing the manifest creates a fresh complete entry. -# validate-models.sh then fails a partial or absent restore loudly instead of letting the -# model-backed tests self-skip. Needs the repository checked out. -name: Restore GGUF models -description: Restore the shared GGUF model cache (written by download-models) and validate every model of .github/models.csv. -inputs: - fail-on-cache-miss: - description: Fail at the restore already when the cache entry is missing ('true' in verify-model-cache). - required: false - default: 'false' -runs: - using: composite - steps: - - name: Restore shared GGUF model cache (populated by download-models; no re-download) - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 - with: - path: models/ - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - fail-on-cache-miss: ${{ inputs.fail-on-cache-miss }} - - name: Validate model files - shell: bash - run: bash .github/validate-models.sh diff --git a/.github/buildcheck/models.py b/.github/buildcheck/models.py index 2fdffe02b..ed7ebc6fa 100644 --- a/.github/buildcheck/models.py +++ b/.github/buildcheck/models.py @@ -1,24 +1,71 @@ # SPDX-FileCopyrightText: 2026 Bernard Ladenthin # # SPDX-License-Identifier: MIT OR Apache-2.0 -"""The GGUF file names publish.yml's `env:` names (for the smoke scripts, the Android emulator jobs -and the integration jobs outside the llama module), checked against .github/models.csv -- the list -the download-models job fetches and validate-models.sh requires. A name the list lacks is a model -no job downloads: its consumer would fail or self-skip. (The llama module's tests default to the -list itself; TestConstantsTest checks that side.) +"""The CI model set, .github/models.csv: its rows (filename,feature,sha256,url, the format +fetch-models.sh reads), and the names publish.yml uses from it -- the GGUF file names of its `env:` +(for the smoke scripts, the Android emulator jobs and the integration jobs outside the llama module) +and the features its jobs ask fetch-models for. A name or feature the list lacks is a model no job +downloads: its consumer would fail or self-skip. (The llama module's tests default to the list +itself; TestConstantsTest checks that side.) """ import re ENV_MODEL = re.compile(r'^ ([A-Z0-9_]+):\s*"?([^"\s]+\.gguf)"?\s*$', re.M) +FEATURES_INPUT = re.compile(r"^ +features: *(.*?) *$", re.M) +FILENAME = re.compile(r"^[A-Za-z0-9._-]+\.gguf$") +FEATURE = re.compile(r"^[a-z]+$") +SHA256 = re.compile(r"^[0-9a-f]{64}$") +# A file of a Hugging Face model repository at a commit: resolve/main would let the file change +# under the test set, and the SHA-256 would then fail every job at once. +PINNED_URL = re.compile(r"^https://huggingface\.co/[^/]+/[^/]+/resolve/[0-9a-f]{40}/(?:.+/)?([^/]+)$") + + +def rows(models_csv_text): + return [[field.strip() for field in line.split(",")] for line in models_csv_text.splitlines() + if line.strip() and not line.startswith("#")] def filenames(models_csv_text): - return {line.split(",", 1)[0].strip() for line in models_csv_text.splitlines() - if line.strip() and not line.startswith("#")} + return {fields[0] for fields in rows(models_csv_text)} + + +def features(models_csv_text): + return {fields[1] for fields in rows(models_csv_text) if len(fields) > 1} + + +def check_rows(models_csv_text): + failures, seen = [], set() + for fields in rows(models_csv_text): + if len(fields) != 4: + failures.append(f"models.csv row {','.join(fields)!r} is not filename,feature,sha256,url") + continue + name, feature, sha256, url = fields + if not FILENAME.match(name): + failures.append(f"models.csv: {name!r} is no plain .gguf file name") + if name in seen: + failures.append(f"models.csv: {name} is listed twice") + seen.add(name) + if not FEATURE.match(feature): + failures.append(f"models.csv: {name}: feature {feature!r} is not one lower-case word") + if not SHA256.match(sha256): + failures.append(f"models.csv: {name}: {sha256!r} is no lower-case SHA-256") + pinned = PINNED_URL.match(url) + if not pinned: + failures.append(f"models.csv: {name}: {url} is no Hugging Face URL pinned to a commit " + "(https://huggingface.co///resolve/<40 hex>/)") + elif pinned.group(1) != name: + failures.append(f"models.csv: {name}: the URL names another file, {pinned.group(1)}") + return failures def check(models_csv_text, workflow_text): - listed = filenames(models_csv_text) - return [f"publish.yml env {name}={value} is not a filename of .github/models.csv -- no job downloads it" - for name, value in ENV_MODEL.findall(workflow_text) if value not in listed] + listed, known = filenames(models_csv_text), features(models_csv_text) + failures = check_rows(models_csv_text) + failures += [f"publish.yml env {name}={value} is not a filename of .github/models.csv -- no job downloads it" + for name, value in ENV_MODEL.findall(workflow_text) if value not in listed] + failures += [f"publish.yml asks fetch-models for the feature {feature!r}, which no row of " + ".github/models.csv has" + for line in FEATURES_INPUT.findall(workflow_text) for feature in line.split() + if feature not in known] + return failures diff --git a/.github/buildcheck/tests/test_models.py b/.github/buildcheck/tests/test_models.py index 933bf94bf..bd85673b4 100644 --- a/.github/buildcheck/tests/test_models.py +++ b/.github/buildcheck/tests/test_models.py @@ -7,7 +7,11 @@ from buildcheck import models, natives from buildcheck.tests.helpers import REPO -CSV = "# comment\na.gguf,https://x/a\nb.gguf,https://x/b\n" +COMMIT = "0123456789abcdef0123456789abcdef01234567" +SHA = "ab" * 32 +CSV = ("# comment\n" + f"a.gguf,draft,{SHA},https://huggingface.co/o/r/resolve/{COMMIT}/a.gguf\n" + f"b.gguf,tts,{SHA},https://huggingface.co/o/r/resolve/{COMMIT}/sub/b.gguf\n") class ModelsTest(unittest.TestCase): @@ -19,6 +23,33 @@ def test_env_names_must_be_listed(self): ["publish.yml env C_MODEL_NAME=c.gguf is not a filename of .github/models.csv " "-- no job downloads it"]) + def test_requested_features_must_be_listed(self): + workflow = " - uses: ./.github/actions/fetch-models\n with:\n features: draft tts\n" + self.assertEqual(models.check(CSV, workflow), []) + self.assertEqual(models.check(CSV, workflow.replace("tts", "tts vison")), + ["publish.yml asks fetch-models for the feature 'vison', which no row of " + ".github/models.csv has"]) + + def test_a_row_has_four_fields(self): + self.assertEqual(models.check_rows("a.gguf,https://huggingface.co/o/r/resolve/main/a.gguf\n"), + ["models.csv row 'a.gguf,https://huggingface.co/o/r/resolve/main/a.gguf' is not " + "filename,feature,sha256,url"]) + + def test_the_url_is_pinned_to_a_commit_and_names_the_file(self): + unpinned = CSV.replace(f"resolve/{COMMIT}/a.gguf", "resolve/main/a.gguf") + self.assertEqual(len(models.check_rows(unpinned)), 1) + self.assertIn("pinned to a commit", models.check_rows(unpinned)[0]) + other = CSV.replace(f"{COMMIT}/a.gguf", f"{COMMIT}/c.gguf") + self.assertEqual(models.check_rows(other), ["models.csv: a.gguf: the URL names another file, c.gguf"]) + + def test_hash_feature_and_duplicates(self): + self.assertEqual(models.check_rows(CSV.replace(SHA, "AB" * 32, 1)), + [f"models.csv: a.gguf: {'AB' * 32!r} is no lower-case SHA-256"]) + self.assertEqual(models.check_rows(CSV.replace(",draft,", ",Draft model,")), + ["models.csv: a.gguf: feature 'Draft model' is not one lower-case word"]) + first = CSV.splitlines()[1] + self.assertEqual(models.check_rows(CSV + first + "\n"), ["models.csv: a.gguf is listed twice"]) + def test_the_repository_passes(self): self.assertEqual(models.check(natives.read(REPO, ".github/models.csv"), natives.read(REPO, ".github/workflows/publish.yml")), []) diff --git a/.github/fetch-models.sh b/.github/fetch-models.sh new file mode 100755 index 000000000..ea892a1ac --- /dev/null +++ b/.github/fetch-models.sh @@ -0,0 +1,103 @@ +#!/bin/bash + +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# Downloads models of the CI set (.github/models.csv) into models/ and checks each against the +# SHA-256 of its row -- the one place CI gets a model, and the same command works locally. +# +# Usage: fetch-models.sh [feature ...] +# no argument every model of the list +# feature ... only the rows whose feature column names one of them (draft, tts, vision, ...); +# a feature no row has is an error, so a typo cannot fetch nothing and pass +# +# Environment: +# HF_TOKEN optional Hugging Face token (CI: the HF_TOKEN secret, absent on fork pull +# requests). With it the downloads count against that account's rate limit instead +# of the shared one of the runner's IP address, which answered 429 before. +# MODELS_DIR target directory (default: models, relative to the working directory) +# +# A file that is already there with the right SHA-256 is kept, so a local models/ directory is +# reused; one with another hash is downloaded again. Downloads land in .part and are moved +# into place only after the hash matched. Runs under Linux, macOS (shasum) and Git Bash on Windows. + +set -euo pipefail + +MODELS_CSV="$(dirname "$0")/models.csv" +DEST="${MODELS_DIR:-models}" +[ -f "$MODELS_CSV" ] || { echo "ERROR: model manifest not found: $MODELS_CSV" >&2; exit 1; } + +sha256_of() { + if command -v sha256sum > /dev/null 2>&1; then + sha256sum "$1" | cut -d' ' -f1 + else + shasum -a 256 "$1" | cut -d' ' -f1 + fi +} + +# Read the whole manifest first: curl must not inherit a loop's stdin. +NAMES=() FEATURES=() HASHES=() URLS=() +while IFS=, read -r name feature hash url rest; do + name="${name%$'\r'}" + case "$name" in '' | \#*) continue ;; esac + url="${url%$'\r'}" + if [ -z "$url" ] || [ -n "${rest:-}" ]; then + echo "ERROR: $MODELS_CSV: row '$name' is not filename,feature,sha256,url" >&2 + exit 1 + fi + NAMES+=("$name") FEATURES+=("$feature") HASHES+=("$hash") URLS+=("$url") +done < "$MODELS_CSV" +[ "${#NAMES[@]}" -gt 0 ] || { echo "ERROR: no models parsed from $MODELS_CSV" >&2; exit 1; } + +for wanted in "$@"; do + found=no + for feature in "${FEATURES[@]}"; do [ "$feature" = "$wanted" ] && found=yes; done + [ "$found" = yes ] || { echo "ERROR: no model of $MODELS_CSV has the feature '$wanted'" >&2; exit 1; } +done + +CURL_AUTH=() +if [ -n "${HF_TOKEN:-}" ]; then + # The header goes through a file, so the token is not on curl's command line. curl sends a + # custom Authorization header only to the host it was given, not to the CDN it is redirected to. + HEADER_FILE="$(mktemp)" + trap 'rm -f "$HEADER_FILE"' EXIT + printf 'Authorization: Bearer %s\n' "$HF_TOKEN" > "$HEADER_FILE" + CURL_AUTH=(-H "@$HEADER_FILE") + echo "Hugging Face: with a token (HF_TOKEN)" +else + echo "Hugging Face: anonymous (no HF_TOKEN; the runner's shared per-IP rate limit applies)" +fi + +mkdir -p "$DEST" +fetched=0 kept=0 +for i in "${!NAMES[@]}"; do + name="${NAMES[$i]}" feature="${FEATURES[$i]}" hash="${HASHES[$i]}" url="${URLS[$i]}" + if [ "$#" -gt 0 ]; then + selected=no + for wanted in "$@"; do [ "$feature" = "$wanted" ] && selected=yes; done + [ "$selected" = yes ] || continue + fi + file="$DEST/$name" + if [ -f "$file" ]; then + if [ "$(sha256_of "$file")" = "$hash" ]; then + echo "- $name ($feature): present, SHA-256 matches" + kept=$((kept + 1)) + continue + fi + echo "- $name ($feature): present with another SHA-256, downloading again" + fi + start=$SECONDS + curl -L --proto =https --proto-redir =https --fail -sS --retry 5 --retry-all-errors \ + --connect-timeout 30 ${CURL_AUTH[@]+"${CURL_AUTH[@]}"} -o "$file.part" "$url" + actual="$(sha256_of "$file.part")" + if [ "$actual" != "$hash" ]; then + rm -f "$file.part" + echo "ERROR: $name: SHA-256 $actual, the manifest says $hash ($url)" >&2 + exit 1 + fi + mv -f "$file.part" "$file" + echo "- $name ($feature): downloaded, $(wc -c < "$file" | tr -d ' ') bytes in $((SECONDS - start)) s, SHA-256 matches" + fetched=$((fetched + 1)) +done +echo "models: $fetched downloaded, $kept already present (${*:-all features})" diff --git a/.github/models.csv b/.github/models.csv index aa77f3836..3a24f4bb7 100644 --- a/.github/models.csv +++ b/.github/models.csv @@ -2,21 +2,27 @@ # # SPDX-License-Identifier: MIT OR Apache-2.0 # -# Single source of truth for the CI model set: one "filename,url" row per model. +# Single source of truth for the CI model set: one "filename,feature,sha256,url" row per model. +# filename the file under models/ (no spaces or commas) +# feature what the model is in the test set for, one word; a job asks for the features it needs +# (.github/actions/fetch-models) and gets only their files +# sha256 the file's SHA-256; fetch-models.sh refuses a download that does not match +# url pinned to a Hugging Face commit (resolve//), so the file cannot change under +# the test set; to move a model, change commit and sha256 together # Consumed by: -# - .github/workflows/publish.yml download-models job (manifest-driven download loop; -# the cache key is gguf-models-, so EDITING THIS FILE automatically -# creates a fresh cache entry - no manual cache deletion needed) -# - .github/validate-models.sh / .bat (the required-model list) -# Filenames must not contain spaces or commas. Lines starting with # are ignored. -codellama-7b.Q2_K.gguf,https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/resolve/main/codellama-7b.Q2_K.gguf -jina-reranker-v1-tiny-en-Q4_0.gguf,https://huggingface.co/gpustack/jina-reranker-v1-tiny-en-GGUF/resolve/main/jina-reranker-v1-tiny-en-Q4_0.gguf -AMD-Llama-135m-code.Q2_K.gguf,https://huggingface.co/QuantFactory/AMD-Llama-135m-code-GGUF/resolve/main/AMD-Llama-135m-code.Q2_K.gguf -Qwen3-0.6B-Q4_K_M.gguf,https://huggingface.co/unsloth/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q4_K_M.gguf -Qwen2.5-1.5B-Instruct-Q4_K_M.gguf,https://huggingface.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF/resolve/main/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf -nomic-embed-text-v1.5.f16.gguf,https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf -SmolVLM-500M-Instruct-Q8_0.gguf,https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF/resolve/main/SmolVLM-500M-Instruct-Q8_0.gguf -mmproj-SmolVLM-500M-Instruct-Q8_0.gguf,https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF/resolve/main/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf -Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf,https://huggingface.co/ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF/resolve/main/Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf -mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf,https://huggingface.co/ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF/resolve/main/mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf -stories260K.gguf,https://huggingface.co/ggml-org/tiny-llamas/resolve/main/stories260K.gguf +# - .github/fetch-models.sh (the composite action fetch-models; also the local download), the +# only place that downloads the set -- from Hugging Face, with the HF_TOKEN secret when present +# - TestConstantsTest (the test defaults are exactly these files), buildcheck/models.py (the row +# format and publish.yml's *_MODEL_NAME env), the smoke scripts' HTTPS round (stories260K) +# Lines starting with # are ignored. +codellama-7b.Q2_K.gguf,text,d3c95001df2912a4214af265918c91515aaae02279492adde5656404e05e9ff5,https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/resolve/98596f7f6c318118824bcbee4b0e20010ec510ec/codellama-7b.Q2_K.gguf +jina-reranker-v1-tiny-en-Q4_0.gguf,rerank,ed226ea4efdb7cc8409961e22969231c90aa1194344486ed9e54b97a97e86b3a,https://huggingface.co/gpustack/jina-reranker-v1-tiny-en-GGUF/resolve/34fdafe5a08b64246bcbfdbf0b8a23f818baf8e3/jina-reranker-v1-tiny-en-Q4_0.gguf +AMD-Llama-135m-code.Q2_K.gguf,draft,b7a897c113a93bc238e392613fe73d764f4e53fe24e03722e99dc59b0ed0f2c6,https://huggingface.co/QuantFactory/AMD-Llama-135m-code-GGUF/resolve/1e4516c4243d1b1a8669737aa0aa2e54122f3591/AMD-Llama-135m-code.Q2_K.gguf +Qwen3-0.6B-Q4_K_M.gguf,reasoning,ac2d97712095a558e31573f62f466a3f9d93990898b0ec79d7c974c1780d524a,https://huggingface.co/unsloth/Qwen3-0.6B-GGUF/resolve/50968a4468ef4233ed78cd7c3de230dd1d61a56b/Qwen3-0.6B-Q4_K_M.gguf +Qwen2.5-1.5B-Instruct-Q4_K_M.gguf,tools,1adf0b11065d8ad2e8123ea110d1ec956dab4ab038eab665614adba04b6c3370,https://huggingface.co/bartowski/Qwen2.5-1.5B-Instruct-GGUF/resolve/9eadc66189c7641e1ddd226b8267a9119b2ce2d4/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf +nomic-embed-text-v1.5.f16.gguf,embedding,f7af6f66802f4df86eda10fe9bbcfc75c39562bed48ef6ace719a251cf1c2fdb,https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/0188c9bf409793f810680a5a431e7b899c46104c/nomic-embed-text-v1.5.f16.gguf +SmolVLM-500M-Instruct-Q8_0.gguf,vision,9d4612de6a42214499e301494a3ecc2be0abdd9de44e663bda63f1152fad1bf4,https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF/resolve/72e986006ef53e37cdd3f6d4241c90b0f01df376/SmolVLM-500M-Instruct-Q8_0.gguf +mmproj-SmolVLM-500M-Instruct-Q8_0.gguf,vision,d1eb8b6b23979205fdf63703ed10f788131a3f812c7b1f72e0119d5d81295150,https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF/resolve/72e986006ef53e37cdd3f6d4241c90b0f01df376/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf +Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf,tts,8d18c94acb2addd042f97da63c98be144eafa76d0d9495177eab65130cf85129,https://huggingface.co/ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF/resolve/ca27d74bc954b73dadab5b71ca265d87fc861a7c/Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf +mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf,tts,6fd65188839bcd6ecc91b277ad471e22a0edfada4699a0fe82f1165c18cfcce2,https://huggingface.co/ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF/resolve/ca27d74bc954b73dadab5b71ca265d87fc861a7c/mmproj-Qwen3-TTS-12Hz-1.7B-Base-Q8_0.gguf +stories260K.gguf,train,047bf46455a544931cff6fef14d7910154c56afbc23ab1c5e56a72e69912c04b,https://huggingface.co/ggml-org/tiny-llamas/resolve/99dd1a73db5a37100bd4ae633f4cfce6560e1567/stories260K.gguf diff --git a/.github/smoke-native-macos.sh b/.github/smoke-native-macos.sh index d4e4127bf..9b20e4d23 100755 --- a/.github/smoke-native-macos.sh +++ b/.github/smoke-native-macos.sh @@ -111,7 +111,7 @@ java -cp "$SMOKE_CLASSES:$CP" NativeLoadSmoke \ # (--ssl-key-file / --ssl-cert-file) serving a model it downloaded from an https:// URL, verified # against the macOS certificate store (Security.framework). -ngl 0: this is about TLS, not Metal. echo "== HTTPS server + https:// model download ==" -HTTPS_MODEL_URL="$(grep -E "^${HTTPS_MODEL_NAME//./\\.}," "$SCRIPT_DIR/models.csv" | cut -d, -f2- || true)" +HTTPS_MODEL_URL="$(awk -F, -v name="$HTTPS_MODEL_NAME" '$1 == name { print $4 }' "$SCRIPT_DIR/models.csv" | tr -d '\r')" [ -n "$HTTPS_MODEL_URL" ] || fail "$HTTPS_MODEL_NAME has no row in $SCRIPT_DIR/models.csv" openssl req -x509 -newkey rsa:2048 -nodes -keyout "$WORK/tls-key.pem" -out "$WORK/tls-cert.pem" -days 2 \ -subj "/CN=127.0.0.1" > /dev/null 2>&1 || fail "could not create the self-signed TLS certificate" diff --git a/.github/smoke-test-natives.ps1 b/.github/smoke-test-natives.ps1 index b7a34e3fb..600812343 100644 --- a/.github/smoke-test-natives.ps1 +++ b/.github/smoke-test-natives.ps1 @@ -171,7 +171,7 @@ Write-Host "smoke set: $SetDir ($($jars.Count) jars; GPU module jars: $($setModu $modelsCsv = Join-Path $PSScriptRoot 'models.csv' $httpsModelUrl = (Get-Content $modelsCsv | Where-Object { $_ -like "$HttpsModelName,*" } | Select-Object -First 1) if (-not $httpsModelUrl) { Write-Error "$HttpsModelName has no row in $modelsCsv (round 2 downloads it over HTTPS)" } -$httpsModelUrl = $httpsModelUrl.Substring($HttpsModelName.Length + 1) +$httpsModelUrl = ($httpsModelUrl -split ',')[3] # filename,feature,sha256,url # ---- Round 1: plain HTTP, the cached model ------------------------------------------------------- $proc = Start-Process java -PassThru -NoNewWindow ` diff --git a/.github/smoke-test-natives.sh b/.github/smoke-test-natives.sh index 40fe3394d..ce2f4ba19 100755 --- a/.github/smoke-test-natives.sh +++ b/.github/smoke-test-natives.sh @@ -60,7 +60,7 @@ CP="$(smoke_classpath "$SET_DIR")" || exit 1 echo "smoke set: $SET_DIR ($(find "$SET_DIR" -maxdepth 1 -name 'llama-*.jar' -exec basename {} \; | sort | tr '\n' ' '))" echo "GPU module jars in the set: $(smoke_set_modules "$SET_DIR" | tr '\n' ' ')" -HTTPS_MODEL_URL="$(grep -E "^${HTTPS_MODEL_NAME//./\\.}," "$SCRIPT_DIR/models.csv" | cut -d, -f2- || true)" +HTTPS_MODEL_URL="$(awk -F, -v name="$HTTPS_MODEL_NAME" '$1 == name { print $4 }' "$SCRIPT_DIR/models.csv" | tr -d '\r')" [ -n "$HTTPS_MODEL_URL" ] || fail "$HTTPS_MODEL_NAME has no row in $SCRIPT_DIR/models.csv (round 2 downloads it over HTTPS)" SERVER_PID="" diff --git a/.github/validate-models.sh b/.github/validate-models.sh deleted file mode 100755 index 2971b7eec..000000000 --- a/.github/validate-models.sh +++ /dev/null @@ -1,73 +0,0 @@ -#!/bin/bash - -# SPDX-FileCopyrightText: 2026 Bernard Ladenthin -# -# SPDX-License-Identifier: MIT OR Apache-2.0 - -# Validate that all required model files exist and are valid GGUF files -# GGUF files start with magic bytes: 0x47 0x47 0x55 0x46 ("GGUF") - -set -e - -# Every CI Java test job (Linux + all macOS + all Windows) restores the shared model -# cache before validating, and runs the embedding / vision / TTS integration tests -# with their properties set — so all of these are REQUIRED, not optional. A missing -# model is a hard failure here (it would otherwise let an integration test silently -# self-skip). The required set comes from the single source of truth -# .github/models.csv (filename,url per line; # comments ignored) so this gate can -# never drift from what download-models actually fetches. -MODELS_CSV="$(dirname "$0")/models.csv" -[ -f "${MODELS_CSV}" ] || { echo "ERROR: model manifest not found: ${MODELS_CSV}"; exit 1; } -MODELS=() -while IFS=, read -r name _url; do - name="${name%$'\r'}" # a CRLF checkout on Windows - case "$name" in ''|\#*) continue ;; esac - MODELS+=("models/$name") -done < "${MODELS_CSV}" -[ "${#MODELS[@]}" -gt 0 ] || { echo "ERROR: no models parsed from ${MODELS_CSV}"; exit 1; } - -# Optional GGUFs validated only when present. The vision test image is committed to -# src/test/resources/images/test-image.jpg and is not validated here — its presence -# is asserted directly by MultimodalIntegrationTest. The audio-input model -# (AudioInputIntegrationTest) has no committed clip and no CI download, so that test -# self-skips and its model is intentionally not listed here. -OPTIONAL_MODELS=() - -validate_gguf() { - local model="$1" - local required="$2" - if [[ ! -f "$model" ]]; then - if [[ "$required" == "required" ]]; then - echo "ERROR: Model not found: $model" - exit 1 - else - echo "- $model (optional, skipped: not present)" - return - fi - fi - # wc and head, not stat/xxd: the same script runs under Linux, macOS (BSD tools) and Git Bash - # on Windows, which has no xxd and whose stat does not read BSD's -f. - local size - size=$(wc -c < "$model" | tr -d ' ') - if [[ $size -lt 4 ]]; then - echo "ERROR: Model file too small (likely corrupted): $model (size: $size bytes)" - exit 1 - fi - local magic - magic=$(head -c 4 "$model") - if [[ "$magic" != "GGUF" ]]; then - echo "ERROR: Invalid GGUF magic bytes in $model (got: $(head -c 4 "$model" | od -An -tx1 | tr -d ' '), expected: 47475546 'GGUF')" - exit 1 - fi - echo "✓ $model ($(numfmt --to=iec-i --suffix=B $size 2>/dev/null || echo $size bytes))" -} - -echo "Validating model files..." -for model in "${MODELS[@]}"; do - validate_gguf "$model" required -done -for model in "${OPTIONAL_MODELS[@]}"; do - validate_gguf "$model" optional -done - -echo "All models validated successfully!" diff --git a/.github/verify-test-counts.sh b/.github/verify-test-counts.sh index ef096a1bc..9f6820aeb 100755 --- a/.github/verify-test-counts.sh +++ b/.github/verify-test-counts.sh @@ -88,7 +88,7 @@ if [ "${#empty_suites[@]}" -gt 0 ]; then printf ' %s\n' "${empty_suites[@]}" >&2 echo " A class-level @BeforeAll assumption that fails looks exactly like this. It is NOT a" >&2 echo " skip — the class reports no tests at all, so it is invisible to a skip check. If a" >&2 - echo " model is missing, validate-models should have failed the job before this point." >&2 + echo " model is missing, fetch-models should have failed the job before this point." >&2 status=1 fi diff --git a/.github/workflows/java-tests.yml b/.github/workflows/java-tests.yml index 10b15c55c..0bb98b8c9 100644 --- a/.github/workflows/java-tests.yml +++ b/.github/workflows/java-tests.yml @@ -9,8 +9,9 @@ # # The only differences between the callers are the inputs below; everything OS-specific (memory # report, core dumps vs. Windows Error Reporting minidumps) is decided here by runner.os. The -# JDK is .java-version, the models come from the shared cache, so nothing is passed through -# `with:` that the caller's `env:` would have to repeat (a reusable workflow sees none of it). +# JDK is .java-version and the models are every row of .github/models.csv (fetch-models), so +# nothing is passed through `with:` that the caller's `env:` would have to repeat (a reusable +# workflow sees none of it); the one secret it needs, HF_TOKEN, each caller passes explicitly. name: Java tests on: workflow_call: @@ -32,6 +33,10 @@ on: description: Name of the artifact uploaded on failure (crash logs, dumps, surefire reports). type: string required: true + secrets: + HF_TOKEN: + description: Hugging Face token for the model downloads (fetch-models); absent on fork pull requests. + required: false permissions: contents: read jobs: @@ -48,7 +53,9 @@ jobs: with: name: ${{ inputs.natives-artifact }} path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: 'temurin' diff --git a/.github/workflows/post-release.yml b/.github/workflows/post-release.yml index cac91b2cc..10873e2eb 100644 --- a/.github/workflows/post-release.yml +++ b/.github/workflows/post-release.yml @@ -123,19 +123,25 @@ jobs: java -version .github/post-release-consumer.sh "${TAG#v}" release/.github/natives.csv consumer "${{ matrix.target }}" ${{ matrix.extra }} - name: Download the 1 MB test model - shell: bash - run: | - mkdir -p models - curl -sSfL --retry 3 -o models/stories260K.gguf "$(grep '^stories260K\.gguf,' .github/models.csv | cut -d, -f2-)" + uses: ./.github/actions/fetch-models + with: + features: train + hf-token: ${{ secrets.HF_TOKEN }} - name: The server smoke on the resolved jars, Java 8 (check 12) if: runner.os == 'Linux' + env: + HF_TOKEN: ${{ secrets.HF_TOKEN }} # the HTTPS round of the smoke downloads through llama.cpp run: .github/smoke-test-natives.sh consumer models/stories260K.gguf - name: The server smoke on the resolved jars, Java 8 (check 12) if: runner.os == 'Windows' shell: pwsh + env: + HF_TOKEN: ${{ secrets.HF_TOKEN }} # the HTTPS round of the smoke downloads through llama.cpp run: .github/smoke-test-natives.ps1 -SetDir consumer -Model models/stories260K.gguf - name: The macOS smoke on the resolved jars, Java 8 (check 12) if: runner.os == 'macOS' + env: + HF_TOKEN: ${{ secrets.HF_TOKEN }} # the HTTPS round of the smoke downloads through llama.cpp run: .github/smoke-native-macos.sh consumer - name: The embedded WebUI of the released library (check 15) if: matrix.target == 'linux-x86-64' diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 9c009385d..be39b2b19 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -47,8 +47,9 @@ env: # of the merged tree to be signed. SIGN_WINDOWS: ${{ github.event_name == 'workflow_dispatch' && inputs.sign_windows }} SCCACHE_WEBDAV_ENDPOINT: https://cache.depot.dev - # The CI model set is .github/models.csv (URLs, and the model cache key is its hash). The Java - # tests default to exactly that set (TestConstants; TestConstantsTest asserts both directions), + # The CI model set is .github/models.csv (one row per file: feature, SHA-256, a Hugging Face URL + # pinned to a commit); every job downloads the features it needs through .github/actions/fetch-models, + # with the HF_TOKEN secret. The Java tests default to exactly that set (TestConstants; TestConstantsTest asserts both directions), # so the test jobs pass no model properties. The names below are only for the consumers outside # the llama module's tests -- the smoke scripts, the Android emulator jobs and the langchain4j / # agent integration jobs; check-natives.py fails when one is not a filename of models.csv. @@ -174,71 +175,6 @@ jobs: passphrase: ${{ secrets.GPG_PASSPHRASE }} key-id: ${{ secrets.GPG_KEY_ID }} - # --------------------------------------------------------------------------- - # Download + cache the GGUF test models ONCE, upfront, for the whole pipeline. - # Every Java test job (`test-java-*`) and the langchain4j integration job `needs:` this - # job and then only RESTORES the shared cache (key gguf-models-) — so the ~5 GB model - # set is fetched from HuggingFace at most once per cache lifetime instead of racing to - # download in each job. GGUF is platform-independent, so this single ubuntu job's cache - # is reused by the macOS and Windows jobs too. On a warm cache this job is a no-op - # restore; on a cold cache it downloads all models, validates them, and saves the cache - # at job end (immutable key, so downstream jobs restore-hit). This is the single source - # of the download logic — do not re-add per-job downloads. - # --------------------------------------------------------------------------- - - download-models: - name: Download + cache GGUF models (once) - needs: startgate - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - - name: Cache GGUF models (GitHub Actions cache; avoids re-downloading from HuggingFace) - # This job is the ONLY writer of the model cache (every consumer job uses the - # restore-only action). enableCrossOsArchive makes the one ubuntu-built entry - # restorable on macOS AND Windows — without it, cache entries are versioned - # per-OS, and the unreachable Windows-side entry was once found re-saved EMPTY - # (343 B) after an eviction, silently starving the Windows jobs of models. - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6 - with: - path: models/ - # GGUF is platform-independent, so ubuntu + macOS + Windows share one entry. - # The key is derived from the model manifest, so EDITING models.csv - # automatically creates a fresh complete entry — no manual cache deletion. - key: gguf-models-${{ hashFiles('.github/models.csv') }} - enableCrossOsArchive: true - - name: Download missing models (manifest-driven, .github/models.csv) - # The manifest is the single source of truth for the model set; this loop is - # the ONLY place in CI that downloads models. Files already restored from the - # cache are skipped, so a warm cache downloads nothing. - run: | - while IFS=, read -r name url; do - case "$name" in ''|\#*) continue ;; esac - test -f "models/$name" || curl -L --proto =https --proto-redir =https --fail --retry 5 --retry-all-errors "$url" --create-dirs -o "models/$name" - done < .github/models.csv - - name: List files in models directory - run: ls -l models/ - - name: Validate model files - run: bash .github/validate-models.sh - - # Prove the freshly written cache entry is restorable AND complete on every OS the - # pipeline uses BEFORE any model-consuming job starts: the same restore-only action - # the consumers use (fail-on-cache-miss makes an unrestorable entry fail here, not - # deep inside a test job), followed by the full validate gate. Every model-backed - # job `needs:` this instead of download-models directly. - verify-model-cache: - name: "Verify model cache (${{ matrix.os }})" - needs: download-models - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest, macos-15, windows-2025-vs2026] - runs-on: ${{ matrix.os }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - - uses: ./.github/actions/restore-models - with: - fail-on-cache-miss: true - # --------------------------------------------------------------------------- # Cross-compile jobs (Docker / dockcross) — produce release artifacts, no testing # --------------------------------------------------------------------------- @@ -337,18 +273,16 @@ jobs: run: mvn -B --no-transfer-progress -f llama-kotlin/pom.xml verify # --------------------------------------------------------------------------- - # Model-backed integration for the langchain4j adapters. Reuses the SAME shared GGUF - # cache (populated once by download-models) and the SAME Linux-x86_64 native artifact the - # core Java jobs already use — no extra model download and no duplicated download logic - # (restore-only cache, no curl steps). It exercises the chat / embedding / scoring - # adapters against the already-cached chat (Qwen3-0.6B), nomic-embedding and jina-reranker - # models. The model-backed tests self-skip when a model is absent, so a cold cache degrades - # to a skip, never a failure. + # Model-backed integration for the langchain4j adapters. Reuses the SAME Linux-x86_64 native + # artifact the core Java jobs already use and fetches only the four models it needs + # (fetch-models, which fails on a missing or changed file). It exercises the chat / embedding / + # scoring / tool adapters against the chat (Qwen3-0.6B), nomic-embedding, jina-reranker and + # Qwen2.5-1.5B tool models. # --------------------------------------------------------------------------- test-java-llama-langchain4j-integration: name: Integration Test llama-langchain4j (model-backed) - needs: [crosscompile-linux-x86_64, verify-model-cache] + needs: [crosscompile-linux-x86_64] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 @@ -359,14 +293,17 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + features: reasoning embedding rerank tools + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: 'temurin' java-version-file: .java-version - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) uses: ./.github/actions/build-core - - name: Run llama-langchain4j model-backed integration tests (reused cached models) + - name: Run llama-langchain4j model-backed integration tests run: > mvn -B --no-transfer-progress -f llama-langchain4j/pom.xml test -Dnet.ladenthin.llama.model.path=models/${REASONING_MODEL_NAME} @@ -397,7 +334,7 @@ jobs: # --------------------------------------------------------------------------- # llama-ffm: the C API of libjllama (jllama_api.h) through the Java 22+ Foreign Function and - # Memory API, no JNI. Reuses the Linux x86_64 natives artifact and the cached models like the + # Memory API, no JNI. Reuses the Linux x86_64 natives artifact like the # langchain4j integration job (lib.path, no second loading path); Maven runs on the JDK of # .java-version (25), which compiles, tests and documents the module for its Java 22 target. # Gates both publish jobs: the module ships on Maven Central. @@ -405,7 +342,7 @@ jobs: test-java-llama-ffm: name: Build and Test llama-ffm (Java 22+ FFM over the C API, model-backed) - needs: [crosscompile-linux-x86_64, verify-model-cache] + needs: [crosscompile-linux-x86_64] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 @@ -416,7 +353,10 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + features: draft tts + hf-token: ${{ secrets.HF_TOKEN }} # Build with the shared JDK; llama-ffm retains its Java 22 bytecode target. - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: @@ -424,7 +364,7 @@ jobs: java-version-file: .java-version - name: Install parent + core net.ladenthin:llama (classes; the test JVM loads the downloaded native library via lib.path) uses: ./.github/actions/build-core - - name: Build, test and document llama-ffm (the model-backed tests run on the cached draft model and the cached Qwen3-TTS pair) + - name: Build, test and document llama-ffm (the model-backed tests run on the draft model and the Qwen3-TTS pair) run: > mvn -B --no-transfer-progress -f llama-ffm/pom.xml verify -Dgpg.skip=true -Dnet.ladenthin.llama.ffm.model=models/${DRAFT_MODEL_NAME} @@ -473,7 +413,7 @@ jobs: # OpenAiCompatServer over a loopback socket with a scripted backend (no native lib, # no GGUF) and pin the streamed tool_calls / role=tool / multi-round shape — seconds, # on every PR. It also collects the agent jar and its dependencies for the smoke below. - # - model-backed: the same loop against the cached Qwen2.5-1.5B tool model through the + # - model-backed: the same loop against the Qwen2.5-1.5B tool model through the # downloaded Linux native library (chat, streaming, tool call + result, read/write/read # loop). A gate since its assertions stopped pinning wording (content checks are limited # to facts no instruct model gets wrong and to tool results) and it ran green throughout. @@ -529,7 +469,7 @@ jobs: test-java-llama-atmosphere-agent-integration: name: Integration Test llama-atmosphere-agent (model-backed) - needs: [crosscompile-linux-x86_64, verify-model-cache] + needs: [crosscompile-linux-x86_64] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 @@ -540,7 +480,10 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + features: tools + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: 'temurin' @@ -551,7 +494,7 @@ jobs: modules: llama,llama-platform - name: Resolve the reactor version run: echo "VERSION=$(mvn -q -DforceStdout help:evaluate -Dexpression=project.version | tail -n1)" >> "$GITHUB_ENV" - - name: Run the Atmosphere tool-loop integration test (cached Qwen2.5-1.5B tool model, CPU) + - name: Run the Atmosphere tool-loop integration test (Qwen2.5-1.5B tool model, CPU) run: > mvn -B --no-transfer-progress -f llama-atmosphere-agent/pom.xml "-Dllama.version=${VERSION}" -Dllama.natives=none test -Dtest=AtmosphereToolLoopIntegrationTest -Dsurefire.failIfNoSpecifiedTests=false @@ -1304,7 +1247,7 @@ jobs: # On-emulator runtime validation of the Android AAR: boots a KVM-accelerated # x86_64 emulator (GitHub Linux runners have KVM; arm64 images cannot run here, # which is exactly why the AAR carries the jni/x86_64 ABI), publishes the CPU AAR - # to mavenLocal, adb-pushes the already-cached tiny draft model, and runs the + # to mavenLocal, adb-pushes the tiny draft model, and runs the # consumer fixture's connectedDebugAndroidTest — System.loadLibrary from the APK, # pure-Java GgufInspector on-device, and real native inference on Android/bionic. # RELEASE GATE (in both publish needs graphs) since PR #298: the job ran @@ -1316,7 +1259,7 @@ jobs: test-android-emulator: name: Android emulator on-device test (x86_64) - needs: [crosscompile-android-aarch64, crosscompile-android-x86_64, verify-model-cache] + needs: [crosscompile-android-aarch64, crosscompile-android-x86_64] runs-on: ubuntu-latest steps: - name: Free up disk space @@ -1350,15 +1293,17 @@ jobs: with: goal: package - uses: ./.github/actions/publish-cpu-aar-local - - uses: ./.github/actions/restore-models - - name: Free disk space for the emulator (only the draft model is needed on-device) + - uses: ./.github/actions/fetch-models + with: + features: draft + hf-token: ${{ secrets.HF_TOKEN }} + - name: Free disk space for the emulator # Same guard as test-android-llmservice: the AVD userdata partition needs ~7.4 GB; drop the - # rest of the restored ~10 GB GGUF cache (this job adb-pushes only the tiny draft model) plus - # the few preinstalled toolchains the free-disk-space step leaves, so the emulator can create - # userdata and boot. + # few preinstalled toolchains the free-disk-space step leaves, so the emulator can create + # userdata and boot. (Only the tiny draft model is fetched; until 2026-10 the whole GGUF set + # was restored here and deleted again.) run: | echo "Disk before cleanup:"; df -h / | tail -1 - find models -type f ! -name "${DRAFT_MODEL_NAME}" -delete 2>/dev/null || true sudo rm -rf /usr/local/share/powershell /opt/hostedtoolcache/CodeQL 2>/dev/null || true echo "Disk after cleanup:"; df -h / | tail -1 - name: Pre-install the Android emulator package (retried) @@ -1527,7 +1472,7 @@ jobs: test-android-llmservice: name: LLM Service app UI test on emulator (non-gating) - needs: [crosscompile-android-aarch64, crosscompile-android-x86_64, verify-model-cache] + needs: [crosscompile-android-aarch64, crosscompile-android-x86_64] runs-on: ubuntu-latest steps: - name: Free up disk space @@ -1559,15 +1504,16 @@ jobs: with: modules: llama,llama-kotlin - uses: ./.github/actions/publish-cpu-aar-local - - uses: ./.github/actions/restore-models - - name: Free disk space for the emulator (only the draft model is needed on-device) - # The AVD userdata partition needs ~7.4 GB; the full ~10 GB GGUF cache restore leaves too - # little free, so the emulator FATALs ("Not enough space to create userdata partition") and - # the boot poll loops until timeout. This job only adb-pushes the tiny draft model, so drop - # the rest of the restored cache plus what the free-disk-space step leaves (PowerShell, CodeQL). + - uses: ./.github/actions/fetch-models + with: + features: draft + hf-token: ${{ secrets.HF_TOKEN }} + - name: Free disk space for the emulator + # The AVD userdata partition needs ~7.4 GB; without this the emulator FATALs ("Not enough + # space to create userdata partition") and the boot poll loops until timeout. Drop what the + # free-disk-space step leaves (PowerShell, CodeQL); only the tiny draft model is fetched. run: | echo "Disk before cleanup:"; df -h / | tail -1 - find models -type f ! -name "${DRAFT_MODEL_NAME}" -delete 2>/dev/null || true sudo rm -rf /usr/local/share/powershell /opt/hostedtoolcache/CodeQL 2>/dev/null || true echo "Disk after cleanup:"; df -h / | tail -1 - name: Pre-install the Android emulator package (retried) @@ -2856,7 +2802,7 @@ jobs: test-java-linux-x86_64: name: Java Tests Ubuntu Latest x86_64 - needs: [crosscompile-linux-x86_64, verify-model-cache] + needs: [crosscompile-linux-x86_64] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 @@ -2866,7 +2812,9 @@ jobs: with: name: natives-cpu-linux-x86-64 path: ${{ github.workspace }}/llama/src/main/natives/net/ladenthin/llama/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: 'temurin' @@ -2966,8 +2914,10 @@ jobs: # the release gate and every reader of this file see the same jobs as before. test-java-macos-arm64-metal: name: Java Tests macOS 26 arm64 (Metal) - needs: [build-macos-arm64-metal, verify-model-cache] + needs: [build-macos-arm64-metal] uses: ./.github/workflows/java-tests.yml + secrets: + HF_TOKEN: ${{ secrets.HF_TOKEN }} with: runs-on: macos-26 natives-artifact: macos-26-metal @@ -2976,8 +2926,10 @@ jobs: test-java-macos-arm64-no-metal: name: Java Tests macOS 15 arm64 (no Metal) - needs: [build-macos-arm64-no-metal, verify-model-cache] + needs: [build-macos-arm64-no-metal] uses: ./.github/workflows/java-tests.yml + secrets: + HF_TOKEN: ${{ secrets.HF_TOKEN }} with: runs-on: macos-15 natives-artifact: macos-15-no-metal @@ -2985,8 +2937,10 @@ jobs: test-java-macos-arm64-metal-15: name: Java Tests macOS 15 arm64 (Metal) - needs: [build-macos-arm64-metal-15, verify-model-cache] + needs: [build-macos-arm64-metal-15] uses: ./.github/workflows/java-tests.yml + secrets: + HF_TOKEN: ${{ secrets.HF_TOKEN }} with: runs-on: macos-15 natives-artifact: natives-metal-macos-aarch64 @@ -2998,8 +2952,10 @@ jobs: # the CPU module -- a load alone proves little (CLAUDE.md "CPU variants"), this runs inference. test-java-windows-x86_64: name: Java Tests Windows 2025 x86_64 (default / CPU variants, clang) - needs: [build-windows-x86_64, verify-model-cache] + needs: [build-windows-x86_64] uses: ./.github/workflows/java-tests.yml + secrets: + HF_TOKEN: ${{ secrets.HF_TOKEN }} with: runs-on: windows-2025-vs2026 natives-artifact: natives-cpu-windows-x86-64 @@ -3011,8 +2967,10 @@ jobs: # end-to-end -- the MSVC one is not shipped, it is the second compiler's verdict on the code. test-java-windows-x86_64-msvc: name: Java Tests Windows 2025 x86_64 (MSVC, test only) - needs: [build-windows-x86_64-msvc, verify-model-cache] + needs: [build-windows-x86_64-msvc] uses: ./.github/workflows/java-tests.yml + secrets: + HF_TOKEN: ${{ secrets.HF_TOKEN }} with: runs-on: windows-2025-vs2026 natives-artifact: msvc-windows-x86_64 @@ -3414,7 +3372,7 @@ jobs: # pipeline. fail-fast is off: one platform's failure must not hide another's. smoke-natives: name: Smoke test natives jars (${{ matrix.target }}) - needs: [package, verify-model-cache] + needs: [package] strategy: fail-fast: false matrix: @@ -3436,7 +3394,10 @@ jobs: with: name: llama-smoke-${{ matrix.target }} path: smoke/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + features: draft + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: ${{ matrix.jdk || 'temurin' }} @@ -3453,10 +3414,18 @@ jobs: smoke - name: Run the server smoke test if: runner.os != 'Windows' + env: + # Round 2 downloads stories260K over HTTPS through llama.cpp, which sends HF_TOKEN as a + # bearer token (its -hft default): without it the runner IP's shared rate limit answered 429. + HF_TOKEN: ${{ secrets.HF_TOKEN }} run: .github/smoke-test-natives.sh smoke "models/${DRAFT_MODEL_NAME}" - name: Run the server smoke test if: runner.os == 'Windows' shell: pwsh + env: + # Round 2 downloads stories260K over HTTPS through llama.cpp, which sends HF_TOKEN as a + # bearer token (its -hft default): without it the runner IP's shared rate limit answered 429. + HF_TOKEN: ${{ secrets.HF_TOKEN }} run: .github/smoke-test-natives.ps1 -SetDir smoke -Model "models/$env:DRAFT_MODEL_NAME" # RPC over two JVMs from the same jars: RpcServer in one, the default NativeServer with --rpc # in the other. Requires a chat completion, the model buffer on the RPC endpoint in the load @@ -3477,6 +3446,10 @@ jobs: java-version: '8' - name: Run the same published core and natives on Java 8 if: matrix.target == 'linux-x86-64' + env: + # Round 2 downloads stories260K over HTTPS through llama.cpp, which sends HF_TOKEN as a + # bearer token (its -hft default): without it the runner IP's shared rate limit answered 429. + HF_TOKEN: ${{ secrets.HF_TOKEN }} run: .github/smoke-test-natives.sh smoke "models/${DRAFT_MODEL_NAME}" - name: Upload server logs if: failure() @@ -3502,7 +3475,7 @@ jobs: # answer, a read_file tool round, --web and --acp work through the shipped jars. smoke-agent-linux: name: Smoke test the agent on the natives jars (Linux) - needs: [test-java-llama-atmosphere-agent, package, verify-model-cache] + needs: [test-java-llama-atmosphere-agent, package] runs-on: ubuntu-latest steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 @@ -3516,7 +3489,10 @@ jobs: with: name: llama-smoke-linux-x86-64 path: smoke/ - - uses: ./.github/actions/restore-models + - uses: ./.github/actions/fetch-models + with: + features: tools + hf-token: ${{ secrets.HF_TOKEN }} - uses: actions/setup-java@b843ebf9b913146b38da2eaa7aeddf76074ae1ad # v6 with: distribution: 'temurin' @@ -3564,7 +3540,7 @@ jobs: # macOS-specific in two ways. Its smoke set has one natives jar, metal-macos-aarch64 (macOS has # no GPU module jar: Metal is compiled into the library), and it asserts native loadability # rather than a CLI exit code -- `codesign --strict` is what actually detects a dylib assembled - # from two builds. No cache restore (~1 min): a full model-backed macOS server smoke would be + # from two builds. No model fetch (~1 min): a full model-backed macOS server smoke would be # strictly more, but this catches the failure class that shipped and is cheap enough to always # run. Its last step downloads the 1 MB stories260K.gguf over HTTPS and serves it behind a # self-signed certificate -- the proof that the dylib's SSL is the statically linked BoringSSL @@ -3588,6 +3564,10 @@ jobs: distribution: 'temurin' java-version-file: .java-version - name: Run the packaged-native smoke test + env: + # Round 2 downloads stories260K over HTTPS through llama.cpp, which sends HF_TOKEN as a + # bearer token (its -hft default): without it the runner IP's shared rate limit answered 429. + HF_TOKEN: ${{ secrets.HF_TOKEN }} run: .github/smoke-native-macos.sh smoke - name: Upload server logs if: failure() diff --git a/CHANGELOG.md b/CHANGELOG.md index 8942fecc7..fc5709e62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,15 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] ### Changed +- **CI downloads its test models from Hugging Face, per job and per feature, instead of through the + GitHub Actions cache.** `.github/models.csv` has four columns now, `filename,feature,sha256,url`, + with every URL pinned to a Hugging Face commit; `.github/fetch-models.sh` (the composite action + `fetch-models`, replacing `restore-models`, `validate-models.sh` and the `download-models` / + `verify-model-cache` jobs) downloads the rows of the features a job names with the `HF_TOKEN` + secret and refuses a file whose SHA-256 differs. The 6.1 GB cache entry, restored whole by about 20 + jobs per Publish run, had gone over the account's cache egress once; the smoke jobs need 57 MB now, + and the natives smoke's HTTPS round sends the token too (it had got a 429 anonymously). Locally, + `bash .github/fetch-models.sh [feature ...]` fills `models/`. - **llama.cpp b11901** (from b11887, 14 upstream commits, tagged). Nothing new to call from Java; what changes is robustness. Input nested deeper than upstream's new limits is refused instead of overflowing the native stack, which ended the JVM (#31153): a JSON schema (`response_format`, tool diff --git a/CLAUDE.md b/CLAUDE.md index 6010608ee..180c5bba9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -452,8 +452,9 @@ bernardladenthin/actions, so a file can move between them without a second heade third-party ones annotated in `REUSE.toml` (the generated dockcross wrappers, the JNI headers). **Composite actions (`.github/actions/`).** The shared ones are in bernardladenthin/actions (see -above and "Signed publication"); the local ones are this repo's own: `restore-models` (restore the shared GGUF -cache + `validate-models.sh`; the one place a job gets its models — 16 call sites) and +above and "Signed publication"); the local ones are this repo's own: `fetch-models` (`.github/fetch-models.sh`: the models of +the features a job names, from Hugging Face with `HF_TOKEN`, SHA-256 checked; the one place a job gets +its models — 9 call sites, plus the reusable `java-tests.yml` and `post-release.yml`) and `install-sccache-windows` (10 call sites; the caller keeps `if:` and `continue-on-error`), `build-core` (parent + `llama`, optionally more modules, built without tests/checks/javadoc/signing, `install` or `package` -- the one copy of that skip-flag list, 9 call sites) and @@ -1140,9 +1141,9 @@ unlaunched. 1. **`smoke-natives`** (matrix, `fail-fast: false`; rows `linux-x86-64` on `ubuntu-latest`, `linux-aarch64` on `ubuntu-24.04-arm`, `windows-x86-64` on `windows-2025-vs2026`, - `windows-aarch64` on `windows-11-arm`; `needs: [package, verify-model-cache]`). Each row runs + `windows-aarch64` on `windows-11-arm`; `needs: [package]`). Each row runs `java -cp '/*' net.ladenthin.llama.server.ServerLauncher` (`smoke-test-natives.sh`, in - PowerShell `smoke-test-natives.ps1`) on a GPU-less runner with the cached draft model: poll + PowerShell `smoke-test-natives.ps1`) on a GPU-less runner with the draft model: poll `/health` to 200, assert a `/v1/chat/completions` choice, and require the loader line `[jllama] native backend '' loaded ... with GPU module(s) ...` naming **every** module jar of the platform (read from `natives.csv`, `smoke-lib.sh` / `Get-SetModules`) -- so every GPU module @@ -1645,7 +1646,7 @@ suffixes only, and the merge step's "nothing else" rule applies to module direct **Tests.** `src/test/cpp/test_c_api.cpp` drives the C API compiled into `jllama_test` (`jllama_engine.cpp` + `jllama_api.cpp` are in its source list, with the same two definitions) over a tiny random llama-architecture GGUF it writes itself -- hermetic, so it runs in every `C++ Tests` -job without `restore-models`; the output of a random model is noise, the contract is what is +job without `fetch-models`; the output of a random model is noise, the contract is what is asserted. `test_c_abi.c` (the modular jobs' `c_abi`) links the shipped `libjllama` and resolves `jllama_api_version` / `jllama_build_info` through its exports, holding the build info's number to the `llama.cpp=` line of the directory's `jllama-build.txt` (`llama_build_info()` itself is llama-common, @@ -1833,10 +1834,13 @@ verification needs an IP SAN for an IP-literal host, which a bare `CN=127.0.0.1` `smoke-natives-macos`: a second server round behind a self-signed certificate, serving a model it downloaded itself from the `https://` URL of `stories260K.gguf` (1 MB, `models.csv`) into a `LLAMA_CACHE` of the run -- `/health` and a chat completion over HTTPS, plain HTTP on the TLS port -refused, the `.gguf` present in the cache. That download is the only place besides `download-models` +refused, the `.gguf` present in the cache. That download is the only place besides `fetch-models` that reaches Hugging Face, on purpose: it is the end-to-end HTTPS client test with a real -certificate chain and the OS store, which no local check can give. The macOS smoke thereby stopped -being model-free; it is still cache-free and ~1 min. +certificate chain and the OS store, which no local check can give. The step passes the `HF_TOKEN` +secret, which llama.cpp sends as a bearer token (`-hft` reads it from the environment, also for a +`--model-url` download): anonymous, the runner IP's shared rate limit answered 429 once (run +40554119477). The macOS smoke thereby stopped being model-free; it still fetches no CI model and +takes ~1 min. **Cost.** BoringSSL is ~400 C/C++ files compiled with `OPENSSL_NO_ASM` (upstream's setting, no assembler needed), cached by sccache like everything else; the shipped library grows by its @@ -2143,7 +2147,8 @@ mvn test -Dtest=LlamaModelTest#testGenerateAnswer ``` **Optional models.** Every model a test uses defaults to `models/` from `.github/models.csv` -(`TestConstants.DEFAULT_*`), so downloading a model from that list into `models/` is all it takes — +(`TestConstants.DEFAULT_*`), so downloading a model from that list into `models/` is all it takes +(`bash .github/fetch-models.sh vision`, or with no argument the whole set) — the tests find it (module-relative or at the reactor root) and self-skip while it is missing. The properties below only **override** a default. The full property → consumer → default table for every `net.ladenthin.llama.*` property the library understands (runtime + test) is the user-facing @@ -2170,7 +2175,7 @@ below covers the model bindings: | `net.ladenthin.llama.audio.input` | `AudioInputIntegrationTest` | committed `src/test/resources/audios/sample.wav`; any `.wav`/`.mp3` | ```bash -# CI's models, e.g. the vision pair (URLs in .github/models.csv) -- no property needed afterwards: +# CI's models, e.g. the vision pair (bash .github/fetch-models.sh vision) -- no property needed afterwards: mvn test -Dtest=MultimodalIntegrationTest # Another model than the default: mvn test -Dtest=TtsIntegrationTest \ @@ -2715,8 +2720,9 @@ the wiring (a future edit that drops the wrapper from a constant fails that test silently re-muting the suite). `llama-langchain4j` had the identical defect and carries the same resolver as `TestModelPaths` (test classes are not shared between modules). -`validate-models.sh` treats all of these as **required** (a missing model hard-fails the job before -tests run, so a download regression can never silently downgrade to a skip). **The tests default to +`fetch-models.sh` treats every model a job asks for as **required** (a missing or changed model +hard-fails the job before tests run, so a download regression can never silently downgrade to a +skip). **The tests default to exactly the CI model set**: every model-gated test reads its path through a `TestConstants` `DEFAULT_*` constant (`models/`), a `-Dnet.ladenthin.llama.*` property only overrides it, and `TestConstantsTest.theModelDefaultsAreExactlyTheCiModelSet` asserts that the `.gguf` constants and the @@ -2728,27 +2734,46 @@ has a runnable guard. **One** class still self-skips everywhere: `AudioInputInte prompt clip is committed (`src/test/resources/audios/sample.wav`), but the audio model + mmproj have no CI download. Half of `SystemOneIntegrationTest` does too (the `/v1/systemone` answers need a decision model, which is not in the CI set either); its rejection case runs everywhere with the draft model. -The model set has a **single source of truth: `.github/models.csv`** (one `filename,url` row per -model; `#` comments). Everything derives from it: the **`download-models`** job (ubuntu, -`needs: startgate`) is the only place models are fetched from HuggingFace (one manifest-driven -`curl` loop; files already restored from cache are skipped) and the **only writer** of the shared -GGUF cache (path `models/`, key **`gguf-models-`** — so *editing the manifest -automatically creates a fresh complete cache entry*; no manual cache deletion on a model-set -change). The writer sets **`enableCrossOsArchive: true`**, making the one ubuntu-built entry the -same entry macOS and Windows restore. Every consumer uses the composite action -**`.github/actions/restore-models`** — restore-only (`actions/cache/restore`; a consumer can never -re-save an empty/partial entry) plus `validate-models.sh`, which reads its required list from the -same manifest. A **`verify-model-cache` matrix job** (ubuntu / macOS / Windows, `needs: -download-models`) runs the same action with `fail-on-cache-miss: true` and so proves the entry is -restorable **and complete** on every OS **before any model-consuming job starts** — all `test-java-*` -jobs, the langchain4j integration job, the Android emulator job and the smoke jobs `need: -verify-model-cache`. (This design hardened after run 28805360584: without the cross-OS flag, cache -entries are versioned per-OS and the unreachable Windows-side entry had been re-saved **empty** (343 B) -after an eviction; and the then-separate `validate-models.bat`'s quoted `MODELS` list broke cmd's -for-tokenization so its `exit /b 1` never fired — the empty cache sailed through the "gate" and the -Windows jobs silently self-skipped every model-backed test until the server smoke's hard check caught -it. The `.bat` is gone: Windows runs the one bash script under Git Bash, which is why it uses -`head -c`/`wc -c` rather than `xxd`/`stat`.) The remaining `*_MODEL_NAME` env vars of publish.yml +The model set has a **single source of truth: `.github/models.csv`**, one `filename,feature,sha256,url` +row per model (`#` comments): the **feature** says what the model is in the set for (`text`, `draft`, +`reasoning`, `tools`, `rerank`, `embedding`, `vision`, `tts`, `train`), the **URL** is pinned to a +Hugging Face commit (`resolve//`, never `main`) and the **SHA-256** is the file's. A job +gets its models from the composite action **`.github/actions/fetch-models`**, naming the features it +needs (`features: draft tts`; none = every row); it runs **`.github/fetch-models.sh`**, which downloads +those rows from Hugging Face with the **`HF_TOKEN`** secret (a fine-grained read-only token; on a +fork pull request it is absent and the download is anonymous), refuses a file whose SHA-256 differs, +and keeps a file that is already there with the right hash (so the same command fills a local +`models/`). What each job asks for: + +| Job | Features | +|---|---| +| `test-java-linux-x86_64`, the five `java-tests.yml` callers | all | +| `test-java-llama-langchain4j-integration` | `reasoning embedding rerank tools` | +| `test-java-llama-ffm` | `draft tts` | +| `test-java-llama-atmosphere-agent-integration`, `smoke-agent-linux` | `tools` | +| `smoke-natives` (4 rows), `test-android-emulator`, `test-android-llmservice` | `draft` | +| `post-release.yml` `consumer` | `train` | + +`buildcheck/models.py` (`check-natives.py`) holds the rows to that format (four fields, one +lower-case word as feature, a lower-case SHA-256, a pinned URL that names the row's file, no file +twice) and every `features:` of publish.yml to a feature of the list. **Moving a model** means a new +commit in its URL and the new SHA-256 in the same row (`/api/models//revision/main` gives the +commit, `/api/models//paths-info/` the `lfs.oid`, which is the SHA-256). + +**Until 2026-10 the set went through the GitHub Actions cache**, and why that ended is worth keeping: +a `download-models` job fetched every row into one entry keyed `gguf-models-` +(6.1 GB, `enableCrossOsArchive`), a three-OS `verify-model-cache` job proved it restorable, and +`restore-models` brought **the whole entry** to every job that needed any model -- one cache entry +cannot be restored in part -- about 20 times and 100 GB per Publish run. One restore failed with +"Egress is over the account limit" (run 40580317285), the repository's caches stood at 11.1 GB of the +10 GB GitHub keeps, and the one cold-cache download in between had to come from Hugging Face +anonymously, from a runner IP whose shared rate limit answered 429. A cache entry per feature would +have needed one fixed `actions/cache` step per entry (a composite action cannot loop over `uses:`) +and kept the egress on GitHub; Hugging Face with a token serves each file on its own and counts the +requests against the account. (The cache design had hardened after run 28805360584, when an entry +re-saved **empty** after an eviction and a `validate-models.bat` whose `exit /b 1` never fired let the +Windows jobs self-skip every model-backed test; the SHA-256 check is the stricter form of that gate.) +The remaining `*_MODEL_NAME` env vars of publish.yml serve only the consumers outside the llama module's tests (smoke scripts, Android emulator jobs, langchain4j/agent integration); `check-natives.py` fails when one is not a filename of `models.csv`. Locally the model tests self-skip when a GGUF is absent (`Assume.assumeTrue`), so a partial local @@ -2817,7 +2842,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" | `src/test/cpp/test_wire_contracts.cpp` | 6 | **The same contract for the two quieter surfaces.** `RequestField` against `server_schema::make_llama_cmpl_schema(...)` (5 tests) and `TrainingField` against `jllama_train::config_keys()` (1 test). Both receivers *silently ignore* an unknown key — the schema skips it, `train_engine.cpp` reads with `j.value(key, default)` and falls back — so a dead field produces no error anywhere and every string-mapping test keeps passing. `OAI_LAYER`-declared keys (consumed by `oaicompat_*_params_parse` before the schema) are exempt from the schema check, and are checked **both** ways: still unknown to the schema (the inverted check), and read by at least one upstream reader-shaped site (the configure-time sweep — this is what caught `chat_template`, a key a public builder wrote and nothing read). See [`docs/history/parameter-wire-surface.md`](docs/history/parameter-wire-surface.md). | | `src/test/cpp/test_rpc.cpp` | 28 | **The runnable guard for `patches/0015` and for `rpc_support.hpp`.** The device-selection rules (pure, literal inputs: `--rpc` endpoints accumulate across repeated options, an explicit `--device`/`-dev` is never overridden, stale RPC devices are replaced by the local GPUs chosen the way llama.cpp's default does — duplicates by device id dropped, iGPUs only without a discrete GPU, `none` on a CPU-only host). Then the **real** ggml-rpc client and server over loopback, no model: a `mul_mat` graph computed on the RPC backend equals the local CPU result; `ggml_backend_rpc_stop_server()` ends the server, frees the port and disconnects a client that is still connected; an unreachable or malformed endpoint is `nullptr`/`std::invalid_argument` instead of an abort; a registered server that went away is re-checked on the next registration; its device reports 0/0 memory instead of aborting; and a stale server is kept out of a later load that did not ask for it. The loopback server picks a free port from a range so parallel jobs do not collide, and serves the **CPU** by name — a served GPU that lacks an op aborts the server (see the RPC section). Also: server devices chosen by name (case-insensitive, deduplicated), an unknown name listing the available devices, and a registered RPC device refused; the mmproj device pinned the way clip would choose it minus the stale server; and `exclude_stale_devices()`, the params-level guard for TTS and the trainer. | | `src/test/cpp/test_rpc_comm.cpp` | 6 | **`patches/0015`'s COMM_INIT and stop contract over real TCP, protocol v8.** Each scenario runs the C API's RPC server in a fresh subprocess with a deadline, so a regression fails instead of hanging the runner: a loopback server opens no wildcard peer listener and a network bind stays an explicit opt-in; a stop with no peer closes both listeners and lets the port be bound again; a stop interrupts a stalled capability handshake of rank 0 and of rank 1; and a successful peer setup still works and stops. No model, GPU or external network. | -| `src/test/cpp/test_c_api.cpp` | 55 | **The C API (`jllama_api.h`, layer B) without a JVM.** Writes a tiny random two-layer llama GGUF with the GPT-2 byte tokenizer (the public `gguf` API, like `test_kolibri1.cpp`), loads it through `jllama_model_load` and drives the shipped contract: version and build info, null-argument refusal with a message, a missing model failing to load, the load-progress callback (reported, and a `false` return aborts the load), a chat completion that is an OpenAI `chat.completion` object, an invalid request answered as the caller's fault (`JLLAMA_STATUS_INVALID_REQUEST`), streaming until `stop` with the task released afterwards ("Task not found" on the next receive), a released task gone, and the log callback seeing the server's lines with a null set flushing synchronously. Hermetic, so it runs in every `C++ Tests` job without `restore-models`; the output of a random model is noise, the contract is what is asserted. Two things the fixture does on purpose: the output rows of every non-ASCII and special token are scaled to almost nothing, because a byte above 0x7F is an incomplete UTF-8 piece the server holds back rather than streams (a random model that emits only those sends no partial chunk at all), and the task id is checked `>= 0` -- the server numbers its tasks from zero. Then every other `LlamaModel` operation in turn (the meta, raw and OAI completions, raw streaming, the streamed OAI completion (`text_completion` chunks with one id, probabilities, `usage` on the last), the template, tokenize/detokenize, metrics, slots, LoRA, configure, the refusals of a model loaded without `--embedding`/`--reranking`/FIM tokens, the grammar conversion, a Q8_0 quantisation that loads again, several prompts in one request and several completions of one (`"n"` > 1) refused on the blocking, raw and streamed paths while a token array stays one prompt, and every OpenAI answer and chunk carrying a `chatcmpl-` id and the model name, one id per stream), then the other engines: the embedded server in router mode and attached to the model (which still answers after the server stopped), the RPC server over loopback, text to speech refusing missing models, and a fine-tuning run over the random F32 model that writes a GGUF -- which is why `tts_engine.cpp`, `train_engine.cpp` and `native_server_engine.cpp` are in `jllama_test`'s source list. Five model-free cases (`CApiStatic`) run before any load. `CApiSlotFiles` loads the random model with `--slot-save-path`: a saved slot file lands under that directory by its name and restores, and a name with a directory part, `..`, an absolute path or nothing is `JLLAMA_STATUS_INVALID_REQUEST` with nothing written (red before F251: the name was the path, relative to the working directory); a model without the directory refuses save and restore with `JLLAMA_STATUS_ERROR`. Four `CApiRpc` cases pin `jllama_rpc_server_stop()` around the bind and the next serve: a stop issued before ggml binds the socket -- from the serving thread through the test seam `jllama::rpc::set_serve_hook_for_testing()` (`rpc_support.hpp`, not exported), or from another thread while the serve is held in that seam, where the stop must not return before the serve has -- ends the serve instead of being lost (patches/0015 clears ggml's stop flag at the bind; red before F241), a stop while nothing serves does not stop the next serve, and a stop does not reach a serve started while it waits for the one it is for (the stop seam `set_stop_hook_for_testing()` starts the next serve once the stop is recorded; red before F271: the stop waited 30 s for the newcomer and ended it too). Two `CApiFim` cases load the same random model with FIM tokens (`write_tiny_llama_gguf(prefix, true)`): a raw completion with `input_prefix`/`input_suffix`, blocking and streamed, evaluates exactly the prompt `jllama_infill` builds from the same fields, and one of the two fields alone is refused naming the other (red before F246: the completion path completed the empty prompt and dropped both). Two more `CApi` cases pin `messages` on the raw paths: blocking and streamed, the request evaluates exactly the tokens of a completion of the prompt `jllama_apply_template` renders from them (an empty `prompt` beside them counts as none), and a non-empty `prompt` or the infill fields beside them are refused (red before F247: the field was ignored). Two more: the chat fields the raw paths cannot apply beside `messages` (`response_format`, `tools`, `tool_choice`, `parallel_tool_calls`, `top_logprobs`, `thinking_budget_tokens`) are refused by name on both raw paths, and `CApiTemplateStops` loads the random model with an end-of-generation byte of its own (`write_tiny_llama_gguf(prefix, false, 4)`) and a template upstream adds the stop string `` to, which a grammar forces: the raw paths stop there as the chat path does, also beside a `stop` of the request (red before F248: dropped, `stop_type` `eos` and the string in `content`). And `CApiReasoningBudget` loads it with a template that opens a `` block in its generation prompt: a raw completion with `messages` and `reasoning_budget_tokens` 1 completes the forced `` and then the chat path's answer, blocking and streamed, while an output-format grammar beside it still runs (the generation prompt the budget is primed with is left out there); a template without thinking tags leaves the request unchanged (red before F249: the tags were dropped and the budget did nothing). A third case loads the random model with ``, `` and `` as control tokens (`write_tiny_llama_gguf(prefix, false, 4, special)`): the forced end tag reads as text in the raw content, blocking and streamed, and a request's own `preserved_tokens` stays in force beside the template's (red before F250: the session's preserved tokens were not taken over, and the special end tag was an empty piece). A fourth, without `messages`, names the tags in the request with a logit bias that makes the model write the start tag: a budget of zero forces the end tag after each one, and a `reasoning_budget_message` before it (red before F252: without the message the budget forced nothing). Four cases pin the raw multimodal prompt: `CApiVision` loads the random model with a random llava-style vision projector the shared fixture writes too (`write_tiny_mmproj_gguf()`, `--mmproj`) and finds the server's `media_marker` in the model meta; a raw completion whose `prompt` is upstream's `{"prompt_string": "......", "multimodal_data": [""]}` evaluates the text tokens plus the image's four patch embeddings on the blocking, streamed and OpenAI paths (red before F258: refused as if the model had no projector), while markers that do not match the images, data that is not base64, no data and an image at the very end (an abort of the process in `update_slots`) are refused, and the same object on the model without a projector is the caller's fault. `CApiVocabOnly` loads the random model with `--vocab-only`: the meta, tokenize and detokenize calls answer, and a parameterized case per other C function on a model (22 instances: every completion, chat and stream path, the template, embeddings, rerank, infill, metrics, slots, LoRA, configure, systemone, the attached server) is `JLLAMA_STATUS_ERROR` with "`jllama_` is not available in vocab-only mode" (red before F266: a `SIGSEGV` in `llama_vocab::token_bos()` or a wait that never ended). Two pin the `capabilities` object of the model meta (b11901, from upstream's server meta): the random model chats and neither generates nor streams audio, with the vision projector loaded as well (`CApiVision`), while a vocab-only load claims none; and `CApiTts` refuses `jllama_tts_supports_speaker_reference()` with a null handle or out-parameter. | +| `src/test/cpp/test_c_api.cpp` | 55 | **The C API (`jllama_api.h`, layer B) without a JVM.** Writes a tiny random two-layer llama GGUF with the GPT-2 byte tokenizer (the public `gguf` API, like `test_kolibri1.cpp`), loads it through `jllama_model_load` and drives the shipped contract: version and build info, null-argument refusal with a message, a missing model failing to load, the load-progress callback (reported, and a `false` return aborts the load), a chat completion that is an OpenAI `chat.completion` object, an invalid request answered as the caller's fault (`JLLAMA_STATUS_INVALID_REQUEST`), streaming until `stop` with the task released afterwards ("Task not found" on the next receive), a released task gone, and the log callback seeing the server's lines with a null set flushing synchronously. Hermetic, so it runs in every `C++ Tests` job without `fetch-models`; the output of a random model is noise, the contract is what is asserted. Two things the fixture does on purpose: the output rows of every non-ASCII and special token are scaled to almost nothing, because a byte above 0x7F is an incomplete UTF-8 piece the server holds back rather than streams (a random model that emits only those sends no partial chunk at all), and the task id is checked `>= 0` -- the server numbers its tasks from zero. Then every other `LlamaModel` operation in turn (the meta, raw and OAI completions, raw streaming, the streamed OAI completion (`text_completion` chunks with one id, probabilities, `usage` on the last), the template, tokenize/detokenize, metrics, slots, LoRA, configure, the refusals of a model loaded without `--embedding`/`--reranking`/FIM tokens, the grammar conversion, a Q8_0 quantisation that loads again, several prompts in one request and several completions of one (`"n"` > 1) refused on the blocking, raw and streamed paths while a token array stays one prompt, and every OpenAI answer and chunk carrying a `chatcmpl-` id and the model name, one id per stream), then the other engines: the embedded server in router mode and attached to the model (which still answers after the server stopped), the RPC server over loopback, text to speech refusing missing models, and a fine-tuning run over the random F32 model that writes a GGUF -- which is why `tts_engine.cpp`, `train_engine.cpp` and `native_server_engine.cpp` are in `jllama_test`'s source list. Five model-free cases (`CApiStatic`) run before any load. `CApiSlotFiles` loads the random model with `--slot-save-path`: a saved slot file lands under that directory by its name and restores, and a name with a directory part, `..`, an absolute path or nothing is `JLLAMA_STATUS_INVALID_REQUEST` with nothing written (red before F251: the name was the path, relative to the working directory); a model without the directory refuses save and restore with `JLLAMA_STATUS_ERROR`. Four `CApiRpc` cases pin `jllama_rpc_server_stop()` around the bind and the next serve: a stop issued before ggml binds the socket -- from the serving thread through the test seam `jllama::rpc::set_serve_hook_for_testing()` (`rpc_support.hpp`, not exported), or from another thread while the serve is held in that seam, where the stop must not return before the serve has -- ends the serve instead of being lost (patches/0015 clears ggml's stop flag at the bind; red before F241), a stop while nothing serves does not stop the next serve, and a stop does not reach a serve started while it waits for the one it is for (the stop seam `set_stop_hook_for_testing()` starts the next serve once the stop is recorded; red before F271: the stop waited 30 s for the newcomer and ended it too). Two `CApiFim` cases load the same random model with FIM tokens (`write_tiny_llama_gguf(prefix, true)`): a raw completion with `input_prefix`/`input_suffix`, blocking and streamed, evaluates exactly the prompt `jllama_infill` builds from the same fields, and one of the two fields alone is refused naming the other (red before F246: the completion path completed the empty prompt and dropped both). Two more `CApi` cases pin `messages` on the raw paths: blocking and streamed, the request evaluates exactly the tokens of a completion of the prompt `jllama_apply_template` renders from them (an empty `prompt` beside them counts as none), and a non-empty `prompt` or the infill fields beside them are refused (red before F247: the field was ignored). Two more: the chat fields the raw paths cannot apply beside `messages` (`response_format`, `tools`, `tool_choice`, `parallel_tool_calls`, `top_logprobs`, `thinking_budget_tokens`) are refused by name on both raw paths, and `CApiTemplateStops` loads the random model with an end-of-generation byte of its own (`write_tiny_llama_gguf(prefix, false, 4)`) and a template upstream adds the stop string `` to, which a grammar forces: the raw paths stop there as the chat path does, also beside a `stop` of the request (red before F248: dropped, `stop_type` `eos` and the string in `content`). And `CApiReasoningBudget` loads it with a template that opens a `` block in its generation prompt: a raw completion with `messages` and `reasoning_budget_tokens` 1 completes the forced `` and then the chat path's answer, blocking and streamed, while an output-format grammar beside it still runs (the generation prompt the budget is primed with is left out there); a template without thinking tags leaves the request unchanged (red before F249: the tags were dropped and the budget did nothing). A third case loads the random model with ``, `` and `` as control tokens (`write_tiny_llama_gguf(prefix, false, 4, special)`): the forced end tag reads as text in the raw content, blocking and streamed, and a request's own `preserved_tokens` stays in force beside the template's (red before F250: the session's preserved tokens were not taken over, and the special end tag was an empty piece). A fourth, without `messages`, names the tags in the request with a logit bias that makes the model write the start tag: a budget of zero forces the end tag after each one, and a `reasoning_budget_message` before it (red before F252: without the message the budget forced nothing). Four cases pin the raw multimodal prompt: `CApiVision` loads the random model with a random llava-style vision projector the shared fixture writes too (`write_tiny_mmproj_gguf()`, `--mmproj`) and finds the server's `media_marker` in the model meta; a raw completion whose `prompt` is upstream's `{"prompt_string": "......", "multimodal_data": [""]}` evaluates the text tokens plus the image's four patch embeddings on the blocking, streamed and OpenAI paths (red before F258: refused as if the model had no projector), while markers that do not match the images, data that is not base64, no data and an image at the very end (an abort of the process in `update_slots`) are refused, and the same object on the model without a projector is the caller's fault. `CApiVocabOnly` loads the random model with `--vocab-only`: the meta, tokenize and detokenize calls answer, and a parameterized case per other C function on a model (22 instances: every completion, chat and stream path, the template, embeddings, rerank, infill, metrics, slots, LoRA, configure, systemone, the attached server) is `JLLAMA_STATUS_ERROR` with "`jllama_` is not available in vocab-only mode" (red before F266: a `SIGSEGV` in `llama_vocab::token_bos()` or a wait that never ended). Two pin the `capabilities` object of the model meta (b11901, from upstream's server meta): the random model chats and neither generates nor streams audio, with the vision projector loaded as well (`CApiVision`), while a vocab-only load claims none; and `CApiTts` refuses `jllama_tts_supports_speaker_reference()` with a null handle or out-parameter. | | `src/test/cpp/test_native_server_shutdown.cpp` | 3 | **The runnable guard for the shutdown-handler guard in `patches/0006`/`0007`.** Runs the real `llama_server()` in router mode over an empty `--models-dir` on an ephemeral loopback port (no model, no worker) and stops it the way `native_server.cpp` does. Pins a clean stop (exit code 0), that `llama_server_request_shutdown()` **after** the server returned is a no-op (the deterministic form of the CI `SIGSEGV` — it crashed every run before the fix), and that requests hammered from another thread while the server tears down are safe (5 rounds). Compiled on non-Android only, like `server.cpp` itself. | | `src/test/cpp/test_gallocr_leafs.cpp` | 1 | **The runnable guard for `patches/0020`.** Two graphs with the same two nodes in the same order (`n1 = add(x, x)`, `n2 = mul(w, n1)`) and only the leafs moved: one expands `n1` before `n2` and meets the input first, the other expands only `n2`, whose first source is the weight -- the shape of the llama.cpp case, where every node and node source still matches its slot, so the old check passed it. Reserved with the first, the second must get an allocation for the input inside the allocator's buffer and compute the right product on it. No model, every platform. **Verified both ways:** with the fix removed from the source tree the test dies with SIGSEGV (rc 139); a first version that swapped the operands of one `add` passed without the fix, because a swapped *node source* was already caught -- a guard has to move only the leafs. | | `src/test/cpp/test_lora_cache.cpp` | 5 | **The runnable guard for `patches/0021`.** Writes a random llama model (the shared `tiny_llama_gguf.hpp`, also used by `test_c_api.cpp`) and a random LoRA adapter on K, V and the FFN output of every layer, loads them with one slot and `--cache-ram 0`, and answers the same prompt greedily. After `jllama_set_lora_adapters` (scale 1 -> 0 and back) and after a request with its own adapters, the warm slot must answer exactly like an emptied one. One case guards the others against passing vacuously: the adapter changes the answer, and a repeated request reuses the cache (`timings.cache_n > 0`; not `tokens_cached`, which counts every token the slot holds and is positive after any request). After each adapter change the next request must take nothing from the cache (`timings.cache_n == 0`), so those three cases are red without the patch whether or not the random model happens to answer differently. **Verified both ways:** without the patch 3 of 4 red (the warm slot answered from the old scales), with it 4 of 4 green in three runs; no model, every platform. A second fixture (`LoraCacheFreshSlot`: two slots, `--slot-save-path`) saves slot 0 (by its file name inside that directory) after a request and restores it into the never-used slot 1, whose request must take all of the prompt but at most two tokens from it and answer like slot 0 -- red before the `init()` hunk (`Expected: (prompt_n) <= (2), actual: 77 vs 2`), green with it (F240). | @@ -3293,8 +3318,8 @@ the "Java 22+ over the C API" item of the delivery roadmap (TODO.md item 10), de Java 8, this optional module targets Java 22, and test release settings remain unchanged. CI needs one JDK installation. The `toolchain-jdk22` profile remains a local fallback for Maven running below JDK 22, selecting a registered JDK 22+ from `~/.m2/toolchains.xml`. -- **CI**: `test-java-llama-ffm` (`needs: [crosscompile-linux-x86_64, verify-model-cache]`) downloads - the Linux x86-64 natives, restores the models, builds the core, runs `mvn -f llama-ffm/pom.xml +- **CI**: `test-java-llama-ffm` (`needs: [crosscompile-linux-x86_64]`) downloads + the Linux x86-64 natives, fetches the `draft` and `tts` models, builds the core, runs `mvn -f llama-ffm/pom.xml verify` with `lib.path` and the draft model, and fails when `FfmLlamaModelIntegrationTest`, `FfmTextToSpeechIntegrationTest` or `JllamaNativeTest` skipped itself (a self-skip would hide a broken binding). It gates both publish jobs; the release gate check @@ -3446,14 +3471,13 @@ Wiring: `mvn -P release deploy` (no separate module deploy step — the parent's inherited `release` profile signs and publishes every reactor module together at the same version). A separate **`test-java-llama-langchain4j-integration`** job runs the model-backed tests - (chat/streaming/embedding/scoring adapters) by **reusing** the shared GGUF cache - (`gguf-models-v1`, restore-only — no extra download) and the `natives-cpu-linux-x86-64` native - artifact: it `needs: [crosscompile-linux-x86_64, download-models]` (so the cache is already - populated and it runs in parallel), installs parent+core (classes), points the test JVM at the - downloaded library with `-Dnet.ladenthin.llama.lib.path`, and passes the already-cached chat (`REASONING_MODEL_NAME`), nomic-embedding and - jina-reranker model paths via the module's + (chat/streaming/embedding/scoring/tool adapters) with the four models it fetches + (`features: reasoning embedding rerank tools`) and the `natives-cpu-linux-x86-64` native + artifact: it `needs: [crosscompile-linux-x86_64]`, installs parent+core (classes), points the test JVM at the + downloaded library with `-Dnet.ladenthin.llama.lib.path`, and passes the chat (`REASONING_MODEL_NAME`), nomic-embedding, + jina-reranker and tool model paths via the module's `-Dnet.ladenthin.llama.langchain4j.{embedding,rerank}.model` / `net.ladenthin.llama.model.path` - properties. It is validation-only (not a release gate); a cold cache degrades to a self-skip. + properties. **Mapped** (since 5.0.6): blocking tool calling (`ToolSpecification` ↔ jllama `ToolDefinition` via the module's own `JsonSchemaElementSerializer` — langchain4j's serializer lives in its diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9a0fb2405..c8cb02a5e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -41,7 +41,7 @@ Follow the conventions documented in [CLAUDE.md](CLAUDE.md): native helper file Every new feature or behaviour change MUST include automated tests; bug fixes SHOULD include a regression test. -- Java tests: JUnit Jupiter, under `llama/src/test/java/net/ladenthin/llama/` and `llama/src/test/java/examples/`. Model-backed tests read their GGUF from `models/` (the list is `.github/models.csv`, which CI downloads) and skip themselves without it. +- Java tests: JUnit Jupiter, under `llama/src/test/java/net/ladenthin/llama/` and `llama/src/test/java/examples/`. Model-backed tests read their GGUF from `models/` (the list is `.github/models.csv`; `bash .github/fetch-models.sh [feature ...]` downloads it, as CI does) and skip themselves without it. - C++ unit tests: GoogleTest, under `llama/src/test/cpp/` (one file per helper header and per carried llama.cpp patch; CLAUDE.md lists them), built with `cmake -B build -DBUILD_TESTING=ON` and run with `ctest`. No JVM and no downloaded model required: the tests that need a model write a tiny random GGUF themselves. - Coverage is reported via JaCoCo and pushed to Coveralls and Codecov on every CI run; new code must not regress coverage on critical paths. diff --git a/README.md b/README.md index b97b75510..0a15e3fb0 100644 --- a/README.md +++ b/README.md @@ -565,7 +565,7 @@ Every `net.ladenthin.llama.*` system property recognised by the library, deep-sc The other modules read their own test properties (`net.ladenthin.llama.ffm.model`, `net.ladenthin.llama.langchain4j.*`), documented in [`llama-ffm/README.md`](llama-ffm/README.md) and [`llama-langchain4j/README.md`](llama-langchain4j/README.md). -The test-model defaults are exactly CI's model set (`.github/models.csv`, URLs included), so a model downloaded into `models/` is found without any property; the properties only point a test at another file. `MultimodalIntegrationTest` self-skips when any of the three `vision.*` paths is missing, so a partial setup (just the vision model + the committed image, no mmproj) lets the test class load without erroring. `AudioInputIntegrationTest` self-skips the same way over the three `audio.*` properties. `TtsIntegrationTest` likewise self-skips unless both `tts.model` and `tts.mmproj` paths exist, and `LlamaTrainerIntegrationTest` unless `train.model` (default `models/stories260K.gguf`, an **F32** model) does. +The test-model defaults are exactly CI's model set (`.github/models.csv`, URLs included; `bash .github/fetch-models.sh [feature ...]` downloads it and checks every file's SHA-256), so a model downloaded into `models/` is found without any property; the properties only point a test at another file. `MultimodalIntegrationTest` self-skips when any of the three `vision.*` paths is missing, so a partial setup (just the vision model + the committed image, no mmproj) lets the test class load without erroring. `AudioInputIntegrationTest` self-skips the same way over the three `audio.*` properties. `TtsIntegrationTest` likewise self-skips unless both `tts.model` and `tts.mmproj` paths exist, and `LlamaTrainerIntegrationTest` unless `train.model` (default `models/stories260K.gguf`, an **F32** model) does. ## Documentation diff --git a/TODO.md b/TODO.md index 267fc1c20..c46b59e11 100644 --- a/TODO.md +++ b/TODO.md @@ -663,32 +663,17 @@ measurements and is not evidence that GPU testing is currently blocked. `test_patches.RenderTest.test_doc_is_current_compares_the_committed_page_with_the_render` fails with a bare `False is not true` (cause not found; `docs/patches.md` is LF in the work tree). Fix them, or document that the suite is Linux-only. -- **Round 2 of the natives smoke depends on Hugging Face answering.** `smoke-test-natives.sh` (and - its `.ps1` and macOS counterparts) downloads `stories260K.gguf` from huggingface.co through the - server's own HTTPS client, on purpose -- that download is the proof that the statically linked - BoringSSL verifies against the OS store. In run 40554119477 (PR #583, the re-run) the - `linux-aarch64` row passed round 1 and then failed: `failed to download model: ... failed with - status code: 429`, "server process exited before becoming healthy". A 429 is Hugging Face's rate - limit, not a defect, and it already shows that the TLS handshake and the certificate check - worked. Make the round robust against it without losing what it proves: retry round 2 after a - pause when the server log names an HTTP status of 429 or 5xx, and fail only when the retries run - out or the error is a TLS or connection error. -- **The GGUF cache is downloaded about 20 times per Publish run, and GitHub refused one download.** - In run 40580317285 (`main` at `8bab7232`) the non-gating `LLM Service app UI test on emulator` - failed at `actions/cache/restore` with "Failed to restore: Egress is over the account limit" - (an Azure storage request id), while the 14 other restores of the run succeeded. The entry - `gguf-models-` holds all eleven models, 6.1 GB, and `restore-models` brings - the whole set to every job that needs any model: `verify-model-cache` (3 rows), the four - `smoke-natives` rows, `smoke-agent-linux`, the six Java test jobs, the ffm, langchain4j and agent - integration jobs, `build-macos-arm64-metal-15` and the two Android emulator jobs -- the latter two - delete every model but the draft model right after the restore. That is roughly 100 GB of cache - egress per run. **Do not delete the entry to fix this**: it is intact, its key follows - `models.csv` by itself, and a miss only makes `download-models` fetch the 6.1 GB from Hugging - Face again (the 429 above). The lever is a second, small entry (the draft model, plus - `stories260K.gguf` if round 2 moves to it) for the jobs that need nothing else. Also note: the - repository's caches total 11.1 GB, above the 10 GB GitHub keeps per repository, so it evicts the - least recently used entries (Gradle and Maven caches); the model entry, read by every run, is - not the one that goes. +- **First runs with the models from Hugging Face (`fetch-models`, 2026-10).** The GitHub Actions + model cache is gone: every job downloads the features it needs from Hugging Face with the + `HF_TOKEN` secret, SHA-256 checked (CLAUDE.md, "single source of truth: `.github/models.csv`"); the + natives smoke's HTTPS round passes the same token to llama.cpp. It replaced a 6.1 GB entry that + ~20 jobs restored per Publish run (one restore failed with "Egress is over the account limit", run + 40580317285) and an anonymous HTTPS round that got a 429 (run 40554119477). Check over the first + Publish runs: no 429 and no other download failure, and how long a full-set job spends in + `fetch-models` (6.3 GB). Should Hugging Face still answer 429 with the token, retry the smoke's + round 2 after a pause when the server log names 429 or 5xx (failing only on a TLS or connection + error or when the retries run out); should a model disappear from Hugging Face, mirror the set to + a read-only bucket keyed by the SHA-256 column. Then delete this item. - **REUSE attribution review:** check the remaining Java/documentation attribution to the original fork author with `git blame`, file by file. Attribution is not removed without an explicit owner decision. The generated dockcross wrappers already carry dockcross's own copyright attribution. diff --git a/android-llmservice/CLAUDE.md b/android-llmservice/CLAUDE.md index aa28d5493..4b3e9f6b4 100644 --- a/android-llmservice/CLAUDE.md +++ b/android-llmservice/CLAUDE.md @@ -90,7 +90,7 @@ manages the final app-signing key. CI wires it from the optional `ANDROID_UPLOAD + password/alias secrets. **CI wiring** — **two split jobs** in `.github/workflows/publish.yml` (both `needs:` the two Android -native jobs; the test job additionally `needs: verify-model-cache`), so the installable artifacts are +native jobs; the test job fetches the draft model itself), so the installable artifacts are **always** produced even when the on-device UI test is flaky/slow: - **`build-android-llmservice`** — installs the core + `llama-kotlin` to mavenLocal (`llama-kotlin`'s core dep is provided-scope, so the desktop JAR is never pulled into the APK), stages the CI-built diff --git a/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java b/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java index 84eb9b440..566914fdf 100644 --- a/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java +++ b/llama/src/test/java/net/ladenthin/llama/TestConstantsTest.java @@ -82,7 +82,7 @@ public void propertyLookupResolvesAndHonoursTheDefault() { /** * The models the tests default to and the models CI downloads are one set. * - *

{@code .github/models.csv} is what download-models fetches and validate-models requires; + *

{@code .github/models.csv} is what fetch-models downloads and checks in CI; * every model-gated test defaults to a {@code models/} constant here. A model in the list * that no constant names is downloaded for nothing, and a constant naming a model the list lacks * makes its test self-skip in CI -- green, and silently. Both directions are asserted, which is diff --git a/models/README.md b/models/README.md index 1ea5349c9..34ff08d8a 100644 --- a/models/README.md +++ b/models/README.md @@ -4,28 +4,29 @@ The model-backed tests look for their GGUF files here, as `models/`. Nothi automatically on a local checkout: a test whose model is missing skips itself, so the suite stays green with any subset of the models present. -**The list is [`.github/models.csv`](../.github/models.csv)** — one `filename,url` row per model. -It is the single source for both sides: +**The list is [`.github/models.csv`](../.github/models.csv)** — one +`filename,feature,sha256,url` row per model, each URL pinned to a Hugging Face commit. It is the +single source for both sides: -- **CI**: the `download-models` job of `publish.yml` fetches every row into a shared cache, which - every test job restores; a missing model fails the job there instead of turning into a skip. +- **CI**: every job that needs models runs `.github/fetch-models.sh` (the composite action + `.github/actions/fetch-models`) for the features it needs; a missing or changed model fails the + job there instead of turning into a skip. - **The tests**: each default path in `TestConstants` names a file of that list (`TestConstantsTest` checks that the two are the same set). A `-Dnet.ladenthin.llama.*` property only overrides a default; see the [System Properties Reference](../README.md#system-properties-reference). -To run the tests of one model locally, download its row: +To run the tests of one model locally, fetch its feature (the second column) with the same script, +from the repository root: ```bash -# e.g. the small draft model most tests use -grep '^AMD-Llama-135m-code' .github/models.csv | while IFS=, read -r file url; do - curl -L --fail -o "models/$file" "$url" -done -# or all of them (several GB) -grep -v '^#' .github/models.csv | while IFS=, read -r file url; do - [ -n "$file" ] && curl -L --fail -C - -o "models/$file" "$url" -done +bash .github/fetch-models.sh draft # the small draft model most tests use +bash .github/fetch-models.sh vision tts # several features +bash .github/fetch-models.sh # all of them (about 6.3 GB) ``` +A file that is already here with the right SHA-256 is kept. With `HF_TOKEN` set in the environment +the downloads use that Hugging Face account's rate limit. + The tests find this directory both from the repository root and from the `llama/` module (they also accept `llama/models/`). `*.gguf` files here are git-ignored.