diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e988df3..917e59c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,19 @@ The release procedure (prompt template and step-by-step instructions) lives in [ ## [Unreleased] +### Added +- **`examples/windows-fatjar/`: build your own fat jar, then run it.** Four files: + - `pom.xml` names srcmorph-cli and the llama classifiers of the machine, with `llama-platform` + excluded; + - `build.bat` is one `mvn package` on a JDK 8; + - `srcmorph.yaml` is a commented configuration that starts with `Plan`; + - `run.bat` runs it. + + `docs/windows-intel-core-ultra.md` walks through them for an Intel Core Ultra (CPU and Vulkan + classifiers, checking what loaded, threads). Tested on Windows: built on JDK 8 (32 MiB instead of + the release fat jar's 111 MiB), `Plan` on JDK 8, and a real index with the Vulkan module through + `run.bat`. `ExamplesConfigBindingTest` binds its configuration and checks the pom's main class. + ### Changed - **`bernardladenthin/actions` pinned at v1.1.0.** v1.1.0 adds `java/audit-publication`, an audit of a published release from the outside (not used here yet); nothing changes for the actions in use. @@ -56,6 +69,16 @@ The release procedure (prompt template and step-by-step instructions) lives in [ directory unless the entry was edited. ### Fixed +- **Stale prompt-template documentation.** The plugin README described a template as taking two `%s` + placeholders, and its examples, like the README Quickstart, ended in `File: %s` / `Source: %s`. + Since the core extraction (`b141d20`) a template is the system prompt, sent verbatim, and the file + name and source go to the model as a separate user message. A `%s` therefore reached the model + literally. The text now says so, and the examples carry no placeholders. +- **The plugin README's GPU section described the llama natives before 6.0.0.** It said a GPU natives + jar holds `jllama.dll`, that `lib.path` points at "the GPU `jllama.dll`", and that a GPU library + whose runtime is missing fails over to the CPU one. Since 6.0.0 a GPU jar holds one ggml backend + module that is put next to the CPU library, and `lib.path` names a natives directory. The section + now describes that, adds `cublasLt64_13.dll` to the CUDA runtime list, and links the Windows guide. - **Two CI defects that the unresolvable llama 6.0.0 had hidden.** Until 6.0.0 reached Central, every run stopped at the first Maven step. The first run that resolved it (srcmorph#243) went further and found two more: diff --git a/README.md b/README.md index 5be638ff..4476baf0 100644 --- a/README.md +++ b/README.md @@ -120,8 +120,8 @@ Maven Central regardless; see `CLAUDE.md`.) file-body - + + @@ -171,6 +171,11 @@ model required) — see [`examples/`](examples/) for the full set plus paired `r launcher scripts, and [`srcmorph-cli/README.md`](srcmorph-cli/README.md) for the config-file reference. +**To build a fat jar with only your machine's natives** (for example the CPU and Vulkan classifiers +on Windows), see [`docs/windows-intel-core-ultra.md`](docs/windows-intel-core-ultra.md). It covers a +pom, `build.bat` and `run.bat` in [`examples/windows-fatjar/`](examples/windows-fatjar/), and a +commented configuration. + ### 3. Library dependency ```xml diff --git a/docs/windows-intel-core-ultra.md b/docs/windows-intel-core-ultra.md new file mode 100644 index 00000000..9c8d9789 --- /dev/null +++ b/docs/windows-intel-core-ultra.md @@ -0,0 +1,115 @@ + + +# srcmorph on Windows: build your own fat jar, then run it + +Two steps, four files, all in [`examples/windows-fatjar/`](../examples/windows-fatjar/): + +| File | What it does | +|---|---| +| `pom.xml` | names srcmorph and **the llama natives this machine needs** (the classifiers) | +| `build.bat` | one `mvn package` on a JDK 8 → `target\srcmorph.jar` | +| `srcmorph.yaml` | what to index, with which prompt and which model | +| `run.bat` | `java -jar target\srcmorph.jar srcmorph.yaml` | + +Needed on the machine: a JDK 8 or newer, Maven, and a GGUF model. Nothing else: no C++ toolchain and +no GPU SDK. The Vulkan module uses the GPU driver that is already installed. + +Why build the jar yourself: the fat jar on the GitHub release carries the CPU natives of every +desktop platform (~111 MiB with Vulkan). Your own carries only the ones you name (32 MiB for the two +below), and you choose the GPU module. + +> [!NOTE] +> This is for **srcmorph 1.3.0** (on `net.ladenthin:llama` 6.0.0). Until 1.3.0 is released the pom +> uses `1.3.0-SNAPSHOT` from Central's snapshot repository; then set the version to `1.3.0` and delete +> the `` block. + +## Step 1 — pick the classifiers, build the jar + +The pom declares one **library** jar for the OS/arch and, optionally, **GPU module** jars of the same +OS/arch. For a Windows x86-64 machine with an Intel Core Ultra (Arc iGPU), as shipped: + +| Classifier | Size | Why | +|---|--:|---| +| `cpu-windows-x86-64` | 16.6 MiB | `jllama.dll`, llama/ggml and all CPU variants; ggml picks the best one at start (`alderlake` on a Core Ultra: AVX2 + AVX-VNNI, no AVX-512) | +| `vulkan-windows-x86-64` | 12.0 MiB | the iGPU (or any GPU) through the driver's `vulkan-1.dll`; without a working driver ggml skips it and runs on the CPU | + +Other machines swap the classifiers: for example `cuda13-windows-x86-64` for an NVIDIA GPU (it needs +the CUDA 13 runtime on `PATH`), or `cpu-linux-x86-64` on Linux. The full table is in the java-llama.cpp +README, [Choosing the natives +jars](https://github.com/bernardladenthin/java-llama.cpp#choosing-the-natives-jars). +`openvino-*` is opt-in and can be slower than the CPU. Leave it out unless you want to experiment +with the NPU. + +Set `JAVA_HOME` in `build.bat` to your JDK, then: + +```bat +build.bat +``` + +The result is `target\srcmorph.jar`. + +## Step 2 — the configuration, then run + +Edit `srcmorph.yaml`. Its comments explain every key. The ones to set: + +- **`baseDirectory`, `outputDirectory`, `subtrees`, `fileExtensions`**: which files to index, and + where the `.ai.md` files go. +- **`modelDirectory`, `modelPath`**: the GGUF. A good start on a laptop is + `granite-4.0-h-tiny-Q4_K_M.gguf`, the fastest on a CPU in [the benchmark](ai-index-benchmark/COMPARISON.md). +- **`promptDefinitions`**: the instruction to the model (the system prompt, **no placeholders**). + srcmorph sends each file's name and source as the user message itself. +- **`fieldGenerations`**: which prompt and model each file gets. One `fallback` rule covers every file. + +What a run does: for every matching file, srcmorph sends the prompt and the source to the model and +writes the answer next to the source's path under `outputDirectory`, as `.ai.md`. + +Run it in two passes: + +1. With `command: Plan` (the shipped value), `run.bat` loads no model and writes nothing. It prints + which file gets which model, and which files are larger than the model's window. +2. Then set `command: GenerateFileIndex`, or `All` to add package and project summaries, and run + `run.bat` again. + +A second run regenerates only three kinds of files: a changed file, a file whose settings changed, +and a file whose summary is empty. An empty summary usually means `maxOutputTokens` was too small for +the model. + +## Check what loaded + +The start of the run names the CPU variant, the backends and the devices: + +``` +load_backend: loaded Vulkan backend from \ggml-vulkan.dll +load_backend: loaded CPU backend from \ggml-cpu-alderlake.dll +load_backends_next_to_this_library: 3 backend(s) registered [RPC, Vulkan, CPU], 2 device(s): Vulkan0 (Vulkan), CPU (CPU) +[jllama] native backend 'cpu' loaded from with GPU module(s) vulkan; 18 file(s) in place for ggml to load +``` + +- **`with no GPU module`**: the Vulkan jar is not in the build. +- **Two Vulkan devices** (an iGPU and a discrete GPU): choose one with `devices: Vulkan1` in the + `aiDefinition`. +- **`gpuLayers: 0`**, or `-Dnet.ladenthin.llama.backend=cpu` before `-jar` in `run.bat`: runs on the + CPU only. That is the comparison to measure. + +## Two settings worth measuring + +- **`threads`**: 4 is the default and enough for these models. A Core Ultra mixes P-, E- and slow + LP-E-cores, and ggml waits for the slowest thread, so more threads can be slower. Never set more + than the machine has cores: an oversubscribed run is orders of magnitude slower, not a little. +- **`gpuLayers: 99` against `0`**: the iGPU shares memory and a 15 W budget with the CPU, so measure + before deciding. `command: Calibrate` prints each model's prefill and decode speed. + +## Tested + +On Windows, `cmd.exe`, srcmorph 1.3.0-SNAPSHOT: +- `build.bat` ran on a JDK 8 (1.8.0_504) and produced a 32 MiB jar holding only the Windows x86-64 + natives. +- `Plan` ran on the same JDK 8. +- `run.bat` indexed 6 Java files with Qwen3-0.6B on the Vulkan module, in 17 s, on a JDK 21. The test + machine has no JDK 8 installed. +- The test machine is a Ryzen with an AMD iGPU and an RTX 3070, which is why the log above was + written down for a Core Ultra rather than copied from it. diff --git a/examples/windows-fatjar/build.bat b/examples/windows-fatjar/build.bat new file mode 100644 index 00000000..3645b7a8 --- /dev/null +++ b/examples/windows-fatjar/build.bat @@ -0,0 +1,11 @@ +REM SPDX-FileCopyrightText: 2026 Bernard Ladenthin +REM +REM SPDX-License-Identifier: MIT OR Apache-2.0 + +@echo off +rem Builds target\srcmorph.jar from pom.xml (the CLI plus the natives the pom names). Adapt JAVA_HOME. +cd /d "%~dp0" +set "JAVA_HOME=C:\Program Files\Java\openjdk1.8.0_491" +set "PATH=%JAVA_HOME%\bin;%PATH%" + +mvn -B package diff --git a/examples/windows-fatjar/pom.xml b/examples/windows-fatjar/pom.xml new file mode 100644 index 00000000..8f5ee7f2 --- /dev/null +++ b/examples/windows-fatjar/pom.xml @@ -0,0 +1,95 @@ + + + + + 4.0.0 + local + srcmorph-local + 1 + pom + + + 1.3.0-SNAPSHOT + 6.0.0 + UTF-8 + + + + + + central-snapshots + https://central.sonatype.com/repository/maven-snapshots/ + false + true + + + + + + net.ladenthin + srcmorph-cli + ${srcmorph.version} + + + + net.ladenthin + llama-platform + + + + + + net.ladenthin + llama + ${llama.version} + cpu-windows-x86-64 + + + + net.ladenthin + llama + ${llama.version} + vulkan-windows-x86-64 + + + + + + + org.apache.maven.plugins + maven-assembly-plugin + 3.7.1 + + + jar-with-dependencies + + srcmorph + false + + + net.ladenthin.srcmorph.cli.Main + + + + + + package + + single + + + + + + + diff --git a/examples/windows-fatjar/run.bat b/examples/windows-fatjar/run.bat new file mode 100644 index 00000000..c926df0d --- /dev/null +++ b/examples/windows-fatjar/run.bat @@ -0,0 +1,11 @@ +REM SPDX-FileCopyrightText: 2026 Bernard Ladenthin +REM +REM SPDX-License-Identifier: MIT OR Apache-2.0 + +@echo off +rem Runs srcmorph with srcmorph.yaml from this directory. Adapt JAVA_HOME. +cd /d "%~dp0" +set "JAVA_HOME=C:\Program Files\Java\openjdk1.8.0_491" +set "PATH=%JAVA_HOME%\bin;%PATH%" + +java -jar target\srcmorph.jar srcmorph.yaml diff --git a/examples/windows-fatjar/srcmorph.yaml b/examples/windows-fatjar/srcmorph.yaml new file mode 100644 index 00000000..fef557dc --- /dev/null +++ b/examples/windows-fatjar/srcmorph.yaml @@ -0,0 +1,46 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# What run.bat does with it: every file under baseDirectory/subtrees with one of the fileExtensions +# is matched to a rule in fieldGenerations. The rule names a prompt (promptDefinitions) and a model +# (aiDefinitions). srcmorph sends the prompt as the system message and the file's name and source as +# the user message, and writes the model's answer to outputDirectory as .ai.md. +# +# command: +# Plan prints which file gets which model and prompt; loads nothing, writes nothing +# GenerateFileIndex one .ai.md per source file +# All GenerateFileIndex, then one summary per package and one for the project +# Start with Plan, then switch to GenerateFileIndex. A second run only redoes changed files. +command: Plan + +srcMorph: + # Use forward slashes in Windows paths here: in a double-quoted YAML string, "C:\x" is an escape. + baseDirectory: "C:/path/to/project" + outputDirectory: "C:/path/to/project/target/srcmorph-ai" + subtrees: + - "src/main/java" + fileExtensions: + - ".java" + + generationProvider: llamacpp-jni # "mock" runs without a model, for a dry test of the setup + modelDirectory: "C:/models" # a relative modelPath below is relative to this directory + + promptDefinitions: + - key: java + template: "Summarize this Java file as structured Markdown: purpose, public API, collaborators." + + aiDefinitions: + - key: local + modelPath: granite-4.0-h-tiny-Q4_K_M.gguf + contextSize: 16384 # tokens; a file larger than this is handled by onOversize + maxOutputTokens: 1024 + temperature: 0.15 + threads: 4 # the default; more only after measuring, never more than cores + gpuLayers: 99 # 99 = everything on the GPU (Vulkan), 0 = CPU only + + fieldGenerations: + - promptKey: java + aiDefinitionKey: local + fallback: true # the rule for every file no other rule matches + onOversize: sample # default "fail" stops the run on a file larger than the window diff --git a/srcmorph-cli/README.md b/srcmorph-cli/README.md index e93daee0..1aef9098 100644 --- a/srcmorph-cli/README.md +++ b/srcmorph-cli/README.md @@ -26,7 +26,9 @@ is one classifier variant per llama GPU backend module — plus that one backend's ggml module. ggml loads the module when the matching vendor runtime is installed and skips it otherwise, so a GPU variant never runs worse than the plain jar; it only runs *faster* on the matching OS/arch with that GPU runtime present. Pick the plain CPU jar unless you specifically want a -GPU build. +GPU build. To build your own fat jar with exactly the classifiers of your machine instead, see +[`../docs/windows-intel-core-ultra.md`](../docs/windows-intel-core-ultra.md) and +[`../examples/windows-fatjar/`](../examples/windows-fatjar/). Both `.json`/`.js` (parsed via a Jackson `ObjectMapper`) and `.yaml`/`.yml` (parsed via `YAMLMapper`) are supported — pick whichever you diff --git a/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java b/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java index 2a722a03..352bfeb5 100644 --- a/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java +++ b/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java @@ -4,6 +4,7 @@ package net.ladenthin.srcmorph.cli; import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.greaterThan; import static org.hamcrest.Matchers.is; @@ -15,9 +16,13 @@ import java.nio.file.Path; import java.util.ArrayList; import java.util.List; +import net.ladenthin.srcmorph.cli.configuration.CCommand; import net.ladenthin.srcmorph.cli.configuration.CConfiguration; +import net.ladenthin.srcmorph.config.AiFieldGenerationConfig; +import net.ladenthin.srcmorph.config.AiOversizeStrategy; import net.ladenthin.srcmorph.config.SrcMorphConfiguration; import org.junit.jupiter.api.DynamicTest; +import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestFactory; /** @@ -77,6 +82,33 @@ private static List listExampleConfigFiles() throws IOException { return found; } + /** + * {@code examples/windows-fatjar/} is the build-it-yourself example (docs/windows-intel-core-ultra.md): + * its {@code srcmorph.yaml} is meant for a real model, so it is not one of the mock fixtures above, + * but it must still bind under the strict mapper, start with the safe {@code Plan}, and route every + * file through a fallback rule that handles an oversized file. Its {@code pom.xml} names the CLI's + * main class by string, so a renamed {@link Main} would otherwise only fail on the user's machine. + * + * @throws IOException if an example file cannot be read + */ + @Test + public void windowsFatJarExample_bindsAndNamesTheMainClass() throws IOException { + final Path directory = EXAMPLES_DIRECTORY.resolve("windows-fatjar"); + final CConfiguration configuration = Main.fromYaml(Main.readString(directory.resolve("srcmorph.yaml"))); + + assertThat(configuration.command, is(CCommand.Plan)); + final SrcMorphConfiguration srcMorph = configuration.srcMorph; + assertThat(srcMorph.getGenerationProvider(), is(equalTo("llamacpp-jni"))); + assertThat(srcMorph.getFieldGenerations().size(), is(1)); + final AiFieldGenerationConfig rule = srcMorph.getFieldGenerations().get(0); + assertThat(rule.isFallback(), is(true)); + assertThat(rule.getOnOversize(), is(equalTo(AiOversizeStrategy.SAMPLE.configValue()))); + + assertThat( + Main.readString(directory.resolve("pom.xml")), + containsString("" + Main.class.getName() + "")); + } + private static void assertBindsSuccessfully(final Path exampleConfig) throws IOException { final String fileName = exampleConfig.getFileName().toString(); final String content = Main.readString(exampleConfig); diff --git a/srcmorph-maven-plugin/README.md b/srcmorph-maven-plugin/README.md index d6e3c988..3eebc55a 100644 --- a/srcmorph-maven-plugin/README.md +++ b/srcmorph-maven-plugin/README.md @@ -137,9 +137,10 @@ The plugin is configured from three building blocks, declared on the plugin insi ``: 1. **``** — define each GGUF model once (path + sampling parameters), each with a ``. -2. **``** — define each prompt template once, each with a ``. A template takes - two `%s` placeholders: the file/package name and the source (for packages, the child summaries; for - the optional project overview, the per-package leads). +2. **``** — define each prompt template once, each with a ``. A template is + the **system prompt** and takes **no placeholders**: the file/package name and the source (for + packages, the child summaries; for the optional project overview, the per-package leads) are sent + as a separate user message (`AiPromptSupport`). A `%s` in a template reaches the model literally. 3. **``** — per goal, the routing **rules**. Each rule maps a `` to an `` (model id, which carries the full parameter set) and selects files with a composable **``** tree: ``/``/`` over the leaves ``, `` @@ -221,39 +222,20 @@ The plugin is configured from three building blocks, declared on the plugin insi file-body-java - + file-body-fallback - + package-body - + project-body - + @@ -570,10 +552,13 @@ RTX 3070 a CUDA build measured **~4.5× the CPU decode speed**; Vulkan also work a one-time shader-compilation cost on the first run. OpenCL is intentionally not offered (llama.cpp's OpenCL backend does not support NVIDIA GPUs). -How the native is found: `net.ladenthin.llama.loader.LlamaLoader` tries `net.ladenthin.llama.lib.path`, -then `java.library.path`, then **every natives jar on the classpath**, GPU backends before the CPU one: -a GPU library whose runtime is missing fails its load and the loader falls through to the CPU natives. -So there are two ways to enable a GPU when running the *published* plugin in your own build. +How the native is found (`net.ladenthin:llama` 6.0.0): `net.ladenthin.llama.loader.LlamaLoader` tries +`net.ladenthin.llama.lib.path`, then `java.library.path`, then the natives jars on the classpath. The +library itself (`jllama.dll` with ggml and the CPU modules) comes from the CPU natives jar; a GPU +natives jar holds **one ggml backend module** (`ggml-cuda.dll`, `ggml-vulkan.dll`, …), which the loader +puts next to the library. ggml opens a module only when its vendor runtime is installed; otherwise the +work stays on the CPU modules. So there are two ways to enable a GPU when running the *published* +plugin in your own build. **Recommended — add the GPU natives jar to the plugin's own classpath.** Declare the matching `net.ladenthin:llama` natives jar as a dependency of the plugin in your POM. It is added next to the @@ -600,9 +585,9 @@ runs on the CPU: mvn srcmorph:generate -Dsrcmorph.generationProvider=llamacpp-jni # + the GPU runtime on PATH (see below) ``` -**Alternative — runtime library override (no POM change).** Point `net.ladenthin.llama.lib.path` at a -folder holding the GPU `jllama.dll` (extracted once from the natives jar); it is tried before the -natives jars: +**Alternative — a natives directory (no POM change).** Unpack the CPU natives jar's directory and +copy the GPU module jar's files into it, then point `net.ladenthin.llama.lib.path` at that directory; +it is tried before the natives jars (the binding's README, "Natives outside Maven"): ``` mvn srcmorph:generate -Dnet.ladenthin.llama.lib.path=C:\path\to\gpu-native -Dsrcmorph.generationProvider=llamacpp-jni @@ -619,7 +604,10 @@ mvn srcmorph:generate -Dnet.ladenthin.llama.lib.path=C:\path\to\gpu-native -Dsrc In both cases: - **CUDA** needs a matching CUDA 13 toolkit + driver, and the toolkit's `bin\x64` (with `cudart64_13.dll`, - `cublas64_13.dll`) on `PATH` — the natives jar bundles only `jllama.dll`, not the CUDA runtime. + `cublas64_13.dll`, `cublasLt64_13.dll`) on `PATH` — the natives jar bundles only the backend module + `ggml-cuda.dll`, not the CUDA runtime. +- **Vulkan** needs only the GPU driver (it brings `vulkan-1.dll`). A CLI fat jar built with exactly + the CPU and Vulkan natives of a Windows machine is in [docs/windows-intel-core-ultra.md](../docs/windows-intel-core-ultra.md). - **GPU layer offload** — set `` inside your `` (see "Per-model `` parameters" above): `-1` (default) does **not** pin a layer count, so llama.cpp **auto-fits** as many layers as fit the card's free VRAM — the robust "runs on any card" setting (it