From 69050ddf31789c3ab3056902f683e829cf57e21b Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Mon, 26 Oct 2026 13:44:44 +0100 Subject: [PATCH 1/3] docs: srcmorph on Windows (Intel Core Ultra), and two stale plugin README sections docs/windows-intel-core-ultra.md is the counterpart of java-llama.cpp's guide of the same name, for srcmorph 1.3.0 on llama 6.0.0. It covers both routes, the CLI fat jar and the Maven plugin, with the Vulkan natives for the Arc iGPU: - which artifacts to take (sizes measured); - one configuration for both routes; - the plan that fails an oversized file on purpose, the incremental index, Calibrate, threads on a P/E/LP-E CPU, what the log must show, the NPU. Run on Windows (cmd.exe, a local JDK 21, 1.3.0-SNAPSHOT fat jars built from main): - CLI on the CPU: ggml-cpu-haswell on this Ryzen, 6 files, 50 s; - CLI with the Vulkan jar: both GPUs enumerated, 34 s; - the plugin with the Vulkan natives jar as plugin dependency: BUILD SUCCESS, 17 s; - three re-runs: 6, then 4, then 1 written; only the empty summaries were rewritten; - the guide's own YAML through the CLI's strict mapper. Found on the way, fixed in the plugin README (and the README Quickstart): - Prompt templates were documented with two %s placeholders. Since b141d20 a template is the system prompt, sent verbatim, and the file name and source go separately, so a %s reached the model literally. - The GPU section described the natives before llama 6.0.0 (a GPU jar with jllama.dll, lib.path pointing at it). Since 6.0.0 a GPU jar holds one ggml module next to the CPU library, and lib.path names a natives directory; checked against the 6.0.0 jars. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AJGh28Gu8pLWYbTCqG1LZf --- CHANGELOG.md | 18 ++ README.md | 8 +- docs/windows-intel-core-ultra.md | 323 +++++++++++++++++++++++++++++++ srcmorph-cli/README.md | 3 +- srcmorph-maven-plugin/README.md | 56 +++--- 5 files changed, 371 insertions(+), 37 deletions(-) create mode 100644 docs/windows-intel-core-ultra.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e988df..c6c7796 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,14 @@ The release procedure (prompt template and step-by-step instructions) lives in [ ## [Unreleased] +### Added +- **`docs/windows-intel-core-ultra.md`: srcmorph on Windows.** The guide covers both routes, the CLI + fat jar and the Maven plugin, with the Vulkan natives for an Intel iGPU. It says which artifacts to + take, gives one configuration for both routes, and explains the plan, the incremental index, threads + and how to verify what loaded. Every command and snippet was run on Windows (`cmd.exe`, 1.3.0-SNAPSHOT + on llama 6.0.0): the CLI on the CPU and with Vulkan, the plugin with Vulkan, and the guide's own YAML + through the CLI's strict mapper. + ### Changed - **`bernardladenthin/actions` pinned at v1.1.0.** v1.1.0 adds `java/audit-publication`, an audit of a published release from the outside (not used here yet); nothing changes for the actions in use. @@ -56,6 +64,16 @@ The release procedure (prompt template and step-by-step instructions) lives in [ directory unless the entry was edited. ### Fixed +- **Stale prompt-template documentation.** The plugin README described a template as taking two `%s` + placeholders, and its examples, like the README Quickstart, ended in `File: %s` / `Source: %s`. + Since the core extraction (`b141d20`) a template is the system prompt, sent verbatim, and the file + name and source go to the model as a separate user message. A `%s` therefore reached the model + literally. The text now says so, and the examples carry no placeholders. +- **The plugin README's GPU section described the llama natives before 6.0.0.** It said a GPU natives + jar holds `jllama.dll`, that `lib.path` points at "the GPU `jllama.dll`", and that a GPU library + whose runtime is missing fails over to the CPU one. Since 6.0.0 a GPU jar holds one ggml backend + module that is put next to the CPU library, and `lib.path` names a natives directory. The section + now describes that, adds `cublasLt64_13.dll` to the CUDA runtime list, and links the Windows guide. - **Two CI defects that the unresolvable llama 6.0.0 had hidden.** Until 6.0.0 reached Central, every run stopped at the first Maven step. The first run that resolved it (srcmorph#243) went further and found two more: diff --git a/README.md b/README.md index 5be638f..3d089f9 100644 --- a/README.md +++ b/README.md @@ -120,8 +120,8 @@ Maven Central regardless; see `CLAUDE.md`.) file-body - + + @@ -171,6 +171,10 @@ model required) — see [`examples/`](examples/) for the full set plus paired `r launcher scripts, and [`srcmorph-cli/README.md`](srcmorph-cli/README.md) for the config-file reference. +**On Windows**, with the iGPU through Vulkan and either route (the CLI fat jar or the plugin), see +[`docs/windows-intel-core-ultra.md`](docs/windows-intel-core-ultra.md): which artifacts to take, a +configuration that runs as it is, and how to check what loaded. + ### 3. Library dependency ```xml diff --git a/docs/windows-intel-core-ultra.md b/docs/windows-intel-core-ultra.md new file mode 100644 index 0000000..1698d8a --- /dev/null +++ b/docs/windows-intel-core-ultra.md @@ -0,0 +1,323 @@ + + +# Windows on an Intel Core Ultra: which artifacts, and how to run srcmorph + +For a Windows machine with **one or more JDKs in `C:\Program Files\Java`**, and Maven if you want the +plugin route. Nothing else is needed: no C++ toolchain, no GPU SDK, and no checkout of this +repository. The hardware notes are for an **Intel Core Ultra 7 165U** (Meteor Lake-U). The steps +apply to any Windows x86-64 machine with an Intel iGPU; only the CPU paragraph changes between CPU +generations. + +srcmorph runs the GGUF model in-process through `net.ladenthin:llama` (java-llama.cpp), so its +natives are what decide CPU against GPU. The binding's own Windows guide covers the same natives from +the java-llama.cpp side: [java-llama.cpp, docs/windows-intel-core-ultra.md](https://github.com/bernardladenthin/java-llama.cpp/blob/main/docs/windows-intel-core-ultra.md). + +> [!NOTE] +> **Version.** This guide is for **srcmorph 1.3.0**, the first release on `net.ladenthin:llama` +> 6.0.0, where the natives are separate jars and a GPU backend is an additive module. Until 1.3.0 is +> released, the same artifacts are built from `main` as **1.3.0-SNAPSHOT**: +> - the fat jars are on the rolling [`snapshot` pre-release](https://github.com/bernardladenthin/srcmorph/releases/tag/snapshot); +> - the plugin is on Central's snapshot repository (see [Route B](#route-b--the-maven-plugin-in-your-project)). +> +> srcmorph 1.2.0 runs on llama 5.1.0. Its steps are the same, but its log lines differ from the ones +> quoted below. + +Two routes, and both were run on Windows for this guide (1.3.0-SNAPSHOT, a local JDK 21, `cmd.exe`): + +| Route | Needs | Use it when | +|---|---|---| +| **A — the CLI fat jar** | a JDK | you want to index a directory, with no Maven project involved | +| **B — the Maven plugin** | a JDK and Maven | the index belongs to a Maven build (`mvn srcmorph:generate`) | + +## Which artifacts + +**Route A** downloads one file from the [GitHub release](https://github.com/bernardladenthin/srcmorph/releases), +never from Maven Central. Both jars run on any OS; the Vulkan one also uses the iGPU on Windows: + +| Take | Size | Why | +|---|--:|---| +| `srcmorph-cli--jar-with-dependencies-vulkan-windows-x86-64.jar` | ~111 MiB | the CLI, the CPU natives of every desktop platform, and the Vulkan module for the Arc iGPU through the installed driver's `vulkan-1.dll`. Nothing else to install | +| `srcmorph-cli--jar-with-dependencies.jar` | ~99 MiB | the same without the Vulkan module: CPU only | + +Take the Vulkan jar. On a machine without a working Vulkan driver, ggml skips the module and runs on +the CPU, so it is never slower than the plain jar. Every `.jar` has a `.asc` signature next to it in +the release. + +**Route B** gets the plugin from Maven Central and declares **one extra natives jar** as a dependency +of the plugin: `net.ladenthin:llama:6.0.0:vulkan-windows-x86-64` (12 MiB). The CPU natives already +come with the plugin through `net.ladenthin:llama-platform`. + +**Skip** these: +- the `cuda13-*` and `rocm-*` variants: there is no NVIDIA or AMD GPU here; +- `sycl-windows-x86-64`: it needs the oneAPI 2026 runtime DLLs on every machine; +- `opencl-windows-x86-64`: ggml's OpenCL backend is tuned for Qualcomm Adreno; +- `openvino-windows-x86-64`: it is opt-in and can be slower than the CPU. See [the NPU](#the-npu). + +What the hardware implies: + +- **CPU:** AVX2 and AVX-VNNI, no AVX-512, no AMX (Core Ultra series 1 ships AVX-512 disabled). ggml + therefore loads **`ggml-cpu-alderlake.dll`**. +- **iGPU:** Arc Graphics (Xe-LPG, 4 Xe-cores) shares LPDDR5 with the CPU. Vulkan helps prompt + processing, which is most of srcmorph's work since every file is a long prompt. Token generation + is memory-bound on both, at a 15 W package budget. Measure both with `Calibrate` (below) before you + believe either. +- **Cores:** 2 P-cores, 8 E-cores and 2 LP-E-cores. Do not use all 14 threads; see [Thread + count](#thread-count). + +## The JDK + +| | JDK floor | +|---|---| +| the CLI, the plugin, the core and the natives | **Java 8+** (all of it is class-file major 52) | + +Any installed JDK from 8 up works, so take the newest. To pick one for a single window without +touching the machine-wide setting: + +```bat +set "JAVA_HOME=C:\Program Files\Java\openjdk21.0.11" +set "PATH=%JAVA_HOME%\bin;%PATH%" +java -version +``` + +For Route B, `mvn -v` reports the JDK Maven runs on; the plugin runs inside Maven's JVM. + +## A model + +Put a GGUF anywhere, for example `C:\models\`. The model choice is measured in +[the benchmark](ai-index-benchmark/COMPARISON.md) and summarised in the plugin README's +[Recommended Models](../srcmorph-maven-plugin/README.md#recommended-models). For a 15 W laptop: + +- **`granite-4.0-h-tiny-Q4_K_M.gguf`**: the fastest on a CPU (~4×), Apache-2.0. Start here. +- **`Seed-Coder-8B-Instruct-Q4_K_M.gguf`**: a small dense coder, MIT. +- **`Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf`**: more faithful, but a ~17 GB file. Only with + 32 GB RAM or more. + +## The configuration: one file for both routes + +The CLI reads JSON or YAML, and the plugin reads the same keys as XML elements; the CLI README says +so key by key. Save this as `C:\srcmorph\srcmorph.yaml`. Relative paths are relative to the directory +the CLI is started in: + +```yaml +command: Plan +srcMorph: + baseDirectory: "C:/path/to/project" + outputDirectory: "C:/path/to/project/target/srcmorph-ai" + subtrees: + - "src/main/java" + fileExtensions: + - ".java" + generationProvider: llamacpp-jni + modelDirectory: "C:/models" + promptDefinitions: + - key: file-body + template: "Summarize this Java file as structured Markdown: purpose, public API, collaborators." + aiDefinitions: + - key: local + modelPath: granite-4.0-h-tiny-Q4_K_M.gguf + contextSize: 16384 + maxOutputTokens: 1024 + temperature: 0.15 + threads: 8 + threadsBatch: 10 + gpuLayers: 99 + fieldGenerations: + - promptKey: file-body + aiDefinitionKey: local + fallback: true + onOversize: sample +``` + +- **`template`** is the system prompt and takes **no placeholders**. srcmorph sends the file name and + the source as a separate user message. +- **`gpuLayers: 99`** offloads everything to the iGPU when the Vulkan module loaded. `0` keeps the + model on the CPU, which is the comparison to measure. +- **`modelDirectory`** makes every relative `modelPath` relative to it. An absolute `modelPath` is + used as it is. +- **`onOversize`** matters on day one. By default a file larger than the model's context window + **fails the run on purpose**: srcmorph does not trim silently. `sample` summarises a representative + slice instead; `mapReduce` and `deterministic` are the other strategies. The plan in the next step + lists every file that would hit this. + +In a double-quoted YAML string, use forward slashes or doubled backslashes for Windows paths, as +above; a single `\` starts an escape sequence there (`"C:\models"` fails to parse). + +## Route A — the CLI + +### Step 1 — the plan: no model loaded, nothing written + +`command: Plan` is the default and is always safe. It loads no model and writes nothing; it prints +which model and prompt every file would get, and which files are larger than the window: + +```bat +cd /d C:\srcmorph +java -jar srcmorph-cli-1.3.0-jar-with-dependencies-vulkan-windows-x86-64.jar srcmorph.yaml +``` + +With the default `onOversize`, an oversized file is reported like this, and the run stops there: + +``` +### (!) Over window - source exceeds the model's context window, would be trimmed (1) +- src/main/java/.../Main.java -> model `local` (source 15071 chars > window 7203 chars) onOversize=fail -> FAILS BUILD +``` + +That line was measured with a deliberately small 2048-token window. Raise `contextSize`, or keep +`onOversize: sample`, and run the plan again until no line says `FAILS BUILD`. + +### Step 2 — the index + +Set `command: GenerateFileIndex` for the per-file summaries, or `command: All` to also aggregate the +packages and the project. Then run the same command again: + +```bat +java -jar srcmorph-cli-1.3.0-jar-with-dependencies-vulkan-windows-x86-64.jar srcmorph.yaml +``` + +The `.ai.md` files land under `outputDirectory`, mirroring the source tree. + +**Re-runs are incremental.** A file is regenerated only in three cases: its source changed, its +settings changed, or its summary is **empty**. `force: true` regenerates everything. On the test +machine three consecutive runs over 6 files gave 6 written, then 4 written and 2 unchanged, then 1 +written and 5 unchanged. The rewrites were the empty summaries: a small reasoning model (Qwen3-0.6B) +had spent its 256 output tokens thinking. If the same files are rewritten on every run, open them; +an empty body means `maxOutputTokens` is too small for that model. + +### Step 3 — measure, then pick the settings + +`command: Calibrate` loads each model your rules route to, measures its prefill and decode throughput, +and prints a paste-ready `` block per model. Run it once with `gpuLayers: 99` and once +with `gpuLayers: 0`, with a few `threads` values each, and keep the fastest combination. On this class +of machine neither direction is obvious; see [Thread count](#thread-count). + +## Route B — the Maven plugin in your project + +The same configuration as XML in the project's `pom.xml`, with the Vulkan natives jar as a plugin +dependency: + +```xml + + + + net.ladenthin + srcmorph-maven-plugin + 1.3.0 + + + + net.ladenthin + llama + 6.0.0 + vulkan-windows-x86-64 + + + + ${project.build.directory}/srcmorph-ai + llamacpp-jni + C:\models + + + local + granite-4.0-h-tiny-Q4_K_M.gguf + 16384 + 1024 + 0.15 + 8 + 10 + 99 + + + + + file-body + + + + + + true + sample + file-body + local + + + + + + +``` + +In XML a Windows path needs no escaping. Then run: + +```bat +cd /d C:\path\to\project +mvn srcmorph:generate -Dsrcmorph.planOnly=true +mvn srcmorph:generate +``` + +The first command prints the plan, like the CLI's `Plan`. The second writes the index. The plugin +also has `aggregate-packages`, `aggregate-project` and `calibrate` goals; see the +[plugin README](../srcmorph-maven-plugin/README.md) for binding them to phases. + +**Before 1.3.0 is released**, version `1.3.0-SNAPSHOT` resolves only with Central's snapshot +repository added to the project: + +```xml + + + central-snapshots + https://central.sonatype.com/repository/maven-snapshots/ + false + true + + +``` + +## Thread count + +The biggest single knob on this CPU. On Windows ggml runs its own thread pool and synchronises on a +barrier per operation. **The barrier waits for the slowest thread**, and the two LP-E-cores are much +slower than the P- and E-cores, so all 14 threads is very likely not the optimum. + +Measure `threads` 6, 8 and 10 with `Calibrate` and keep the best. `threadsBatch` sets the +prompt-processing thread count separately and tolerates a higher value, because prompt processing has +enough work per barrier to hide the synchronisation cost. srcmorph sends long prompts, so +`threadsBatch` matters more here than in a chat. + +## Verifying what loaded + +The log names the CPU variant, the backends and the devices ggml ended up with. From the Vulkan run on +the test machine (an AMD Ryzen, hence `haswell`; a Core Ultra shows `alderlake`): + +``` +load_backend: loaded Vulkan backend from \ggml-vulkan.dll +load_backend: loaded CPU backend from \ggml-cpu-haswell.dll +load_backends_next_to_this_library: 3 backend(s) registered from [RPC, Vulkan, CPU], 3 device(s): Vulkan0 (Vulkan), Vulkan1 (Vulkan), CPU (CPU) +[jllama] native backend 'cpu' loaded from with GPU module(s) vulkan; 18 file(s) in place for ggml to load +``` + +- **Vulkan module missing from the last line** (`with no GPU module`): you are running the plain jar, + or the plugin dependency is missing. +- **More than one Vulkan device** (`Vulkan0`, `Vulkan1`, …, as on a laptop with an iGPU and a + discrete GPU): pick one by name with `devices: Vulkan1` in the `aiDefinition`. Device indices depend + on which modules loaded, so always name the device, never rely on an index. + +A Core Ultra 7 165U has a single GPU, so `Vulkan0` is the Arc iGPU. + +`-Dnet.ladenthin.llama.backend=cpu` before `-jar` (or on the `mvn` command line) puts no GPU module in +place. That gives the clean CPU baseline for a comparison without switching jars. + +## The NPU + +`openvino-windows-x86-64` is the only path to the Intel **NPU**, and it is **opt-in** in the binding: +its jar does nothing until `-Dnet.ladenthin.llama.backend` names it. The reason is measured: OpenVINO's +CPU plugin registers as a device that llama.cpp then prefers, and on a host without an Intel +accelerator that dropped token generation to 0.2 tokens per second +([java-llama.cpp, gpu-module-jars.md](https://github.com/bernardladenthin/java-llama.cpp/blob/main/docs/benchmarks/gpu-module-jars.md)). +It also needs the Intel OpenVINO runtime DLLs on `PATH`. srcmorph ships no OpenVINO fat jar. With the +plugin, add the `openvino-windows-x86-64` natives jar as a second plugin dependency, run with +`-Dnet.ladenthin.llama.backend=vulkan,openvino`, and read the device list before trusting a number. +NPU use is **not measured** here. diff --git a/srcmorph-cli/README.md b/srcmorph-cli/README.md index e93daee..bec0581 100644 --- a/srcmorph-cli/README.md +++ b/srcmorph-cli/README.md @@ -26,7 +26,8 @@ is one classifier variant per llama GPU backend module — plus that one backend's ggml module. ggml loads the module when the matching vendor runtime is installed and skips it otherwise, so a GPU variant never runs worse than the plain jar; it only runs *faster* on the matching OS/arch with that GPU runtime present. Pick the plain CPU jar unless you specifically want a -GPU build. +GPU build. A worked Windows example with the Vulkan jar (configuration, plan, index, what the log must +show) is in [`../docs/windows-intel-core-ultra.md`](../docs/windows-intel-core-ultra.md). Both `.json`/`.js` (parsed via a Jackson `ObjectMapper`) and `.yaml`/`.yml` (parsed via `YAMLMapper`) are supported — pick whichever you diff --git a/srcmorph-maven-plugin/README.md b/srcmorph-maven-plugin/README.md index d6e3c98..8502e38 100644 --- a/srcmorph-maven-plugin/README.md +++ b/srcmorph-maven-plugin/README.md @@ -137,9 +137,10 @@ The plugin is configured from three building blocks, declared on the plugin insi ``: 1. **``** — define each GGUF model once (path + sampling parameters), each with a ``. -2. **``** — define each prompt template once, each with a ``. A template takes - two `%s` placeholders: the file/package name and the source (for packages, the child summaries; for - the optional project overview, the per-package leads). +2. **``** — define each prompt template once, each with a ``. A template is + the **system prompt** and takes **no placeholders**: the file/package name and the source (for + packages, the child summaries; for the optional project overview, the per-package leads) are sent + as a separate user message (`AiPromptSupport`). A `%s` in a template reaches the model literally. 3. **``** — per goal, the routing **rules**. Each rule maps a `` to an `` (model id, which carries the full parameter set) and selects files with a composable **``** tree: ``/``/`` over the leaves ``, `` @@ -221,39 +222,20 @@ The plugin is configured from three building blocks, declared on the plugin insi file-body-java - + file-body-fallback - + package-body - + project-body - + @@ -570,10 +552,13 @@ RTX 3070 a CUDA build measured **~4.5× the CPU decode speed**; Vulkan also work a one-time shader-compilation cost on the first run. OpenCL is intentionally not offered (llama.cpp's OpenCL backend does not support NVIDIA GPUs). -How the native is found: `net.ladenthin.llama.loader.LlamaLoader` tries `net.ladenthin.llama.lib.path`, -then `java.library.path`, then **every natives jar on the classpath**, GPU backends before the CPU one: -a GPU library whose runtime is missing fails its load and the loader falls through to the CPU natives. -So there are two ways to enable a GPU when running the *published* plugin in your own build. +How the native is found (`net.ladenthin:llama` 6.0.0): `net.ladenthin.llama.loader.LlamaLoader` tries +`net.ladenthin.llama.lib.path`, then `java.library.path`, then the natives jars on the classpath. The +library itself (`jllama.dll` with ggml and the CPU modules) comes from the CPU natives jar; a GPU +natives jar holds **one ggml backend module** (`ggml-cuda.dll`, `ggml-vulkan.dll`, …), which the loader +puts next to the library. ggml opens a module only when its vendor runtime is installed; otherwise the +work stays on the CPU modules. So there are two ways to enable a GPU when running the *published* +plugin in your own build. **Recommended — add the GPU natives jar to the plugin's own classpath.** Declare the matching `net.ladenthin:llama` natives jar as a dependency of the plugin in your POM. It is added next to the @@ -600,9 +585,9 @@ runs on the CPU: mvn srcmorph:generate -Dsrcmorph.generationProvider=llamacpp-jni # + the GPU runtime on PATH (see below) ``` -**Alternative — runtime library override (no POM change).** Point `net.ladenthin.llama.lib.path` at a -folder holding the GPU `jllama.dll` (extracted once from the natives jar); it is tried before the -natives jars: +**Alternative — a natives directory (no POM change).** Unpack the CPU natives jar's directory and +copy the GPU module jar's files into it, then point `net.ladenthin.llama.lib.path` at that directory; +it is tried before the natives jars (the binding's README, "Natives outside Maven"): ``` mvn srcmorph:generate -Dnet.ladenthin.llama.lib.path=C:\path\to\gpu-native -Dsrcmorph.generationProvider=llamacpp-jni @@ -619,7 +604,10 @@ mvn srcmorph:generate -Dnet.ladenthin.llama.lib.path=C:\path\to\gpu-native -Dsrc In both cases: - **CUDA** needs a matching CUDA 13 toolkit + driver, and the toolkit's `bin\x64` (with `cudart64_13.dll`, - `cublas64_13.dll`) on `PATH` — the natives jar bundles only `jllama.dll`, not the CUDA runtime. + `cublas64_13.dll`, `cublasLt64_13.dll`) on `PATH` — the natives jar bundles only the backend module + `ggml-cuda.dll`, not the CUDA runtime. +- **Vulkan** needs only the GPU driver (it brings `vulkan-1.dll`). A worked Windows setup, both the + plugin and the CLI with the Vulkan natives, is in [docs/windows-intel-core-ultra.md](../docs/windows-intel-core-ultra.md). - **GPU layer offload** — set `` inside your `` (see "Per-model `` parameters" above): `-1` (default) does **not** pin a layer count, so llama.cpp **auto-fits** as many layers as fit the card's free VRAM — the robust "runs on any card" setting (it From 015c46b09cb58a5d8592a6efcff0758dd7bac482 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Mon, 26 Oct 2026 14:21:24 +0100 Subject: [PATCH 2/3] examples/windows-fatjar: build your own fat jar with the machine's classifiers The Windows guide was too much for a local run. What is needed is two things: build a fat jar with exactly the natives of the machine, and run srcmorph with a configuration. Four files in examples/windows-fatjar/: - pom.xml: srcmorph-cli with llama-platform excluded, plus the classifiers of the machine (cpu-windows-x86-64, vulkan-windows-x86-64), packed into target/srcmorph.jar by maven-assembly-plugin with the CLI's main class. - build.bat: JAVA_HOME of a JDK 8, then one `mvn -B package`. - srcmorph.yaml: a commented configuration starting with the safe Plan. - run.bat: `java -jar target\srcmorph.jar srcmorph.yaml`. docs/windows-intel-core-ultra.md is now that walk-through: pick the classifiers, build, configure, run, check what loaded, and the two settings worth measuring. Tested: - build.bat's Maven call on JDK 8 (1.8.0_504, Docker) gave a 32 MiB jar, against the release fat jar's 111 MiB, holding only the Windows x86-64 natives and one SLF4J provider. - Plan ran on the same JDK 8. - run.bat, called by full path from another directory, indexed 6 files with Qwen3-0.6B on the Vulkan module in 17 s (JDK 21: no JDK 8 on the Windows test machine). All 6 summaries are non-empty. ExamplesConfigBindingTest gains a test for the example: srcmorph.yaml binds under the strict mapper, starts with Plan, has a fallback rule with onOversize=sample, and pom.xml names Main as the main class. A rename would otherwise fail only on the user's machine. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AJGh28Gu8pLWYbTCqG1LZf --- CHANGELOG.md | 17 +- README.md | 7 +- docs/windows-intel-core-ultra.md | 365 ++++-------------- examples/windows-fatjar/build.bat | 11 + examples/windows-fatjar/pom.xml | 95 +++++ examples/windows-fatjar/run.bat | 11 + examples/windows-fatjar/srcmorph.yaml | 46 +++ srcmorph-cli/README.md | 5 +- .../cli/ExamplesConfigBindingTest.java | 32 ++ srcmorph-maven-plugin/README.md | 4 +- 10 files changed, 293 insertions(+), 300 deletions(-) create mode 100644 examples/windows-fatjar/build.bat create mode 100644 examples/windows-fatjar/pom.xml create mode 100644 examples/windows-fatjar/run.bat create mode 100644 examples/windows-fatjar/srcmorph.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index c6c7796..917e59c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,12 +12,17 @@ The release procedure (prompt template and step-by-step instructions) lives in [ ## [Unreleased] ### Added -- **`docs/windows-intel-core-ultra.md`: srcmorph on Windows.** The guide covers both routes, the CLI - fat jar and the Maven plugin, with the Vulkan natives for an Intel iGPU. It says which artifacts to - take, gives one configuration for both routes, and explains the plan, the incremental index, threads - and how to verify what loaded. Every command and snippet was run on Windows (`cmd.exe`, 1.3.0-SNAPSHOT - on llama 6.0.0): the CLI on the CPU and with Vulkan, the plugin with Vulkan, and the guide's own YAML - through the CLI's strict mapper. +- **`examples/windows-fatjar/`: build your own fat jar, then run it.** Four files: + - `pom.xml` names srcmorph-cli and the llama classifiers of the machine, with `llama-platform` + excluded; + - `build.bat` is one `mvn package` on a JDK 8; + - `srcmorph.yaml` is a commented configuration that starts with `Plan`; + - `run.bat` runs it. + + `docs/windows-intel-core-ultra.md` walks through them for an Intel Core Ultra (CPU and Vulkan + classifiers, checking what loaded, threads). Tested on Windows: built on JDK 8 (32 MiB instead of + the release fat jar's 111 MiB), `Plan` on JDK 8, and a real index with the Vulkan module through + `run.bat`. `ExamplesConfigBindingTest` binds its configuration and checks the pom's main class. ### Changed - **`bernardladenthin/actions` pinned at v1.1.0.** v1.1.0 adds `java/audit-publication`, an audit of a diff --git a/README.md b/README.md index 3d089f9..4476baf 100644 --- a/README.md +++ b/README.md @@ -171,9 +171,10 @@ model required) — see [`examples/`](examples/) for the full set plus paired `r launcher scripts, and [`srcmorph-cli/README.md`](srcmorph-cli/README.md) for the config-file reference. -**On Windows**, with the iGPU through Vulkan and either route (the CLI fat jar or the plugin), see -[`docs/windows-intel-core-ultra.md`](docs/windows-intel-core-ultra.md): which artifacts to take, a -configuration that runs as it is, and how to check what loaded. +**To build a fat jar with only your machine's natives** (for example the CPU and Vulkan classifiers +on Windows), see [`docs/windows-intel-core-ultra.md`](docs/windows-intel-core-ultra.md). It covers a +pom, `build.bat` and `run.bat` in [`examples/windows-fatjar/`](examples/windows-fatjar/), and a +commented configuration. ### 3. Library dependency diff --git a/docs/windows-intel-core-ultra.md b/docs/windows-intel-core-ultra.md index 1698d8a..a8d72a4 100644 --- a/docs/windows-intel-core-ultra.md +++ b/docs/windows-intel-core-ultra.md @@ -4,320 +4,111 @@ SPDX-FileCopyrightText: 2026 Bernard Ladenthin SPDX-License-Identifier: MIT --> -# Windows on an Intel Core Ultra: which artifacts, and how to run srcmorph +# srcmorph on Windows: build your own fat jar, then run it -For a Windows machine with **one or more JDKs in `C:\Program Files\Java`**, and Maven if you want the -plugin route. Nothing else is needed: no C++ toolchain, no GPU SDK, and no checkout of this -repository. The hardware notes are for an **Intel Core Ultra 7 165U** (Meteor Lake-U). The steps -apply to any Windows x86-64 machine with an Intel iGPU; only the CPU paragraph changes between CPU -generations. +Two steps, four files, all in [`examples/windows-fatjar/`](../examples/windows-fatjar/): -srcmorph runs the GGUF model in-process through `net.ladenthin:llama` (java-llama.cpp), so its -natives are what decide CPU against GPU. The binding's own Windows guide covers the same natives from -the java-llama.cpp side: [java-llama.cpp, docs/windows-intel-core-ultra.md](https://github.com/bernardladenthin/java-llama.cpp/blob/main/docs/windows-intel-core-ultra.md). - -> [!NOTE] -> **Version.** This guide is for **srcmorph 1.3.0**, the first release on `net.ladenthin:llama` -> 6.0.0, where the natives are separate jars and a GPU backend is an additive module. Until 1.3.0 is -> released, the same artifacts are built from `main` as **1.3.0-SNAPSHOT**: -> - the fat jars are on the rolling [`snapshot` pre-release](https://github.com/bernardladenthin/srcmorph/releases/tag/snapshot); -> - the plugin is on Central's snapshot repository (see [Route B](#route-b--the-maven-plugin-in-your-project)). -> -> srcmorph 1.2.0 runs on llama 5.1.0. Its steps are the same, but its log lines differ from the ones -> quoted below. - -Two routes, and both were run on Windows for this guide (1.3.0-SNAPSHOT, a local JDK 21, `cmd.exe`): - -| Route | Needs | Use it when | -|---|---|---| -| **A — the CLI fat jar** | a JDK | you want to index a directory, with no Maven project involved | -| **B — the Maven plugin** | a JDK and Maven | the index belongs to a Maven build (`mvn srcmorph:generate`) | - -## Which artifacts - -**Route A** downloads one file from the [GitHub release](https://github.com/bernardladenthin/srcmorph/releases), -never from Maven Central. Both jars run on any OS; the Vulkan one also uses the iGPU on Windows: - -| Take | Size | Why | -|---|--:|---| -| `srcmorph-cli--jar-with-dependencies-vulkan-windows-x86-64.jar` | ~111 MiB | the CLI, the CPU natives of every desktop platform, and the Vulkan module for the Arc iGPU through the installed driver's `vulkan-1.dll`. Nothing else to install | -| `srcmorph-cli--jar-with-dependencies.jar` | ~99 MiB | the same without the Vulkan module: CPU only | - -Take the Vulkan jar. On a machine without a working Vulkan driver, ggml skips the module and runs on -the CPU, so it is never slower than the plain jar. Every `.jar` has a `.asc` signature next to it in -the release. - -**Route B** gets the plugin from Maven Central and declares **one extra natives jar** as a dependency -of the plugin: `net.ladenthin:llama:6.0.0:vulkan-windows-x86-64` (12 MiB). The CPU natives already -come with the plugin through `net.ladenthin:llama-platform`. - -**Skip** these: -- the `cuda13-*` and `rocm-*` variants: there is no NVIDIA or AMD GPU here; -- `sycl-windows-x86-64`: it needs the oneAPI 2026 runtime DLLs on every machine; -- `opencl-windows-x86-64`: ggml's OpenCL backend is tuned for Qualcomm Adreno; -- `openvino-windows-x86-64`: it is opt-in and can be slower than the CPU. See [the NPU](#the-npu). - -What the hardware implies: - -- **CPU:** AVX2 and AVX-VNNI, no AVX-512, no AMX (Core Ultra series 1 ships AVX-512 disabled). ggml - therefore loads **`ggml-cpu-alderlake.dll`**. -- **iGPU:** Arc Graphics (Xe-LPG, 4 Xe-cores) shares LPDDR5 with the CPU. Vulkan helps prompt - processing, which is most of srcmorph's work since every file is a long prompt. Token generation - is memory-bound on both, at a 15 W package budget. Measure both with `Calibrate` (below) before you - believe either. -- **Cores:** 2 P-cores, 8 E-cores and 2 LP-E-cores. Do not use all 14 threads; see [Thread - count](#thread-count). - -## The JDK - -| | JDK floor | +| File | What it does | |---|---| -| the CLI, the plugin, the core and the natives | **Java 8+** (all of it is class-file major 52) | - -Any installed JDK from 8 up works, so take the newest. To pick one for a single window without -touching the machine-wide setting: - -```bat -set "JAVA_HOME=C:\Program Files\Java\openjdk21.0.11" -set "PATH=%JAVA_HOME%\bin;%PATH%" -java -version -``` - -For Route B, `mvn -v` reports the JDK Maven runs on; the plugin runs inside Maven's JVM. +| `pom.xml` | names srcmorph and **the llama natives this machine needs** (the classifiers) | +| `build.bat` | one `mvn package` on a JDK 8 → `target\srcmorph.jar` | +| `srcmorph.yaml` | what to index, with which prompt and which model | +| `run.bat` | `java -jar target\srcmorph.jar srcmorph.yaml` | -## A model - -Put a GGUF anywhere, for example `C:\models\`. The model choice is measured in -[the benchmark](ai-index-benchmark/COMPARISON.md) and summarised in the plugin README's -[Recommended Models](../srcmorph-maven-plugin/README.md#recommended-models). For a 15 W laptop: - -- **`granite-4.0-h-tiny-Q4_K_M.gguf`**: the fastest on a CPU (~4×), Apache-2.0. Start here. -- **`Seed-Coder-8B-Instruct-Q4_K_M.gguf`**: a small dense coder, MIT. -- **`Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf`**: more faithful, but a ~17 GB file. Only with - 32 GB RAM or more. - -## The configuration: one file for both routes - -The CLI reads JSON or YAML, and the plugin reads the same keys as XML elements; the CLI README says -so key by key. Save this as `C:\srcmorph\srcmorph.yaml`. Relative paths are relative to the directory -the CLI is started in: - -```yaml -command: Plan -srcMorph: - baseDirectory: "C:/path/to/project" - outputDirectory: "C:/path/to/project/target/srcmorph-ai" - subtrees: - - "src/main/java" - fileExtensions: - - ".java" - generationProvider: llamacpp-jni - modelDirectory: "C:/models" - promptDefinitions: - - key: file-body - template: "Summarize this Java file as structured Markdown: purpose, public API, collaborators." - aiDefinitions: - - key: local - modelPath: granite-4.0-h-tiny-Q4_K_M.gguf - contextSize: 16384 - maxOutputTokens: 1024 - temperature: 0.15 - threads: 8 - threadsBatch: 10 - gpuLayers: 99 - fieldGenerations: - - promptKey: file-body - aiDefinitionKey: local - fallback: true - onOversize: sample -``` +Needed on the machine: a JDK 8 or newer, Maven, and a GGUF model. Nothing else: no C++ toolchain and +no GPU SDK. The Vulkan module uses the GPU driver that is already installed. -- **`template`** is the system prompt and takes **no placeholders**. srcmorph sends the file name and - the source as a separate user message. -- **`gpuLayers: 99`** offloads everything to the iGPU when the Vulkan module loaded. `0` keeps the - model on the CPU, which is the comparison to measure. -- **`modelDirectory`** makes every relative `modelPath` relative to it. An absolute `modelPath` is - used as it is. -- **`onOversize`** matters on day one. By default a file larger than the model's context window - **fails the run on purpose**: srcmorph does not trim silently. `sample` summarises a representative - slice instead; `mapReduce` and `deterministic` are the other strategies. The plan in the next step - lists every file that would hit this. +Why build the jar yourself: the fat jar on the GitHub release carries the CPU natives of every +desktop platform (~111 MiB with Vulkan). Your own carries only the ones you name (32 MiB for the two +below), and you choose the GPU module. -In a double-quoted YAML string, use forward slashes or doubled backslashes for Windows paths, as -above; a single `\` starts an escape sequence there (`"C:\models"` fails to parse). - -## Route A — the CLI +> [!NOTE] +> This is for **srcmorph 1.3.0** (on `net.ladenthin:llama` 6.0.0). Until 1.3.0 is released the pom +> uses `1.3.0-SNAPSHOT` from Central's snapshot repository; then set the version to `1.3.0` and delete +> the `` block. -### Step 1 — the plan: no model loaded, nothing written +## Step 1 — pick the classifiers, build the jar -`command: Plan` is the default and is always safe. It loads no model and writes nothing; it prints -which model and prompt every file would get, and which files are larger than the window: +The pom declares one **library** jar for the OS/arch and, optionally, **GPU module** jars of the same +OS/arch. For a Windows x86-64 machine with an Intel Core Ultra (Arc iGPU), as shipped: -```bat -cd /d C:\srcmorph -java -jar srcmorph-cli-1.3.0-jar-with-dependencies-vulkan-windows-x86-64.jar srcmorph.yaml -``` - -With the default `onOversize`, an oversized file is reported like this, and the run stops there: - -``` -### (!) Over window - source exceeds the model's context window, would be trimmed (1) -- src/main/java/.../Main.java -> model `local` (source 15071 chars > window 7203 chars) onOversize=fail -> FAILS BUILD -``` - -That line was measured with a deliberately small 2048-token window. Raise `contextSize`, or keep -`onOversize: sample`, and run the plan again until no line says `FAILS BUILD`. +| Classifier | Size | Why | +|---|--:|---| +| `cpu-windows-x86-64` | 16.6 MiB | `jllama.dll`, llama/ggml and all CPU variants; ggml picks the best one at start (`alderlake` on a Core Ultra: AVX2 + AVX-VNNI, no AVX-512) | +| `vulkan-windows-x86-64` | 12.0 MiB | the iGPU (or any GPU) through the driver's `vulkan-1.dll`; without a working driver ggml skips it and runs on the CPU | -### Step 2 — the index +Other machines swap the classifiers: for example `cuda13-windows-x86-64` for an NVIDIA GPU (it needs +the CUDA 13 runtime on `PATH`), or `cpu-linux-x86-64` on Linux. The full table is in the java-llama.cpp +README, [Choosing the natives +jars](https://github.com/bernardladenthin/java-llama.cpp#choosing-the-natives-jars). +`openvino-*` is opt-in and can be slower than the CPU. Leave it out unless you want to experiment +with the NPU. -Set `command: GenerateFileIndex` for the per-file summaries, or `command: All` to also aggregate the -packages and the project. Then run the same command again: +Set `JAVA_HOME` in `build.bat` to your JDK, then: ```bat -java -jar srcmorph-cli-1.3.0-jar-with-dependencies-vulkan-windows-x86-64.jar srcmorph.yaml +build.bat ``` -The `.ai.md` files land under `outputDirectory`, mirroring the source tree. +The result is `target\srcmorph.jar`. -**Re-runs are incremental.** A file is regenerated only in three cases: its source changed, its -settings changed, or its summary is **empty**. `force: true` regenerates everything. On the test -machine three consecutive runs over 6 files gave 6 written, then 4 written and 2 unchanged, then 1 -written and 5 unchanged. The rewrites were the empty summaries: a small reasoning model (Qwen3-0.6B) -had spent its 256 output tokens thinking. If the same files are rewritten on every run, open them; -an empty body means `maxOutputTokens` is too small for that model. +## Step 2 — the configuration, then run -### Step 3 — measure, then pick the settings +Edit `srcmorph.yaml`. Its comments explain every key. The ones to set: -`command: Calibrate` loads each model your rules route to, measures its prefill and decode throughput, -and prints a paste-ready `` block per model. Run it once with `gpuLayers: 99` and once -with `gpuLayers: 0`, with a few `threads` values each, and keep the fastest combination. On this class -of machine neither direction is obvious; see [Thread count](#thread-count). +- **`baseDirectory`, `outputDirectory`, `subtrees`, `fileExtensions`**: which files to index, and + where the `.ai.md` files go. +- **`modelDirectory`, `modelPath`**: the GGUF. A good start on a laptop is + `granite-4.0-h-tiny-Q4_K_M.gguf`, the fastest on a CPU in [the benchmark](ai-index-benchmark/COMPARISON.md). +- **`promptDefinitions`**: the instruction to the model (the system prompt, **no placeholders**). + srcmorph sends each file's name and source as the user message itself. +- **`fieldGenerations`**: which prompt and model each file gets. One `fallback` rule covers every file. -## Route B — the Maven plugin in your project +What a run does: for every matching file, srcmorph sends the prompt and the source to the model and +writes the answer next to the source's path under `outputDirectory`, as `.ai.md`. -The same configuration as XML in the project's `pom.xml`, with the Vulkan natives jar as a plugin -dependency: +Run it in two passes: -```xml - - - - net.ladenthin - srcmorph-maven-plugin - 1.3.0 - - - - net.ladenthin - llama - 6.0.0 - vulkan-windows-x86-64 - - - - ${project.build.directory}/srcmorph-ai - llamacpp-jni - C:\models - - - local - granite-4.0-h-tiny-Q4_K_M.gguf - 16384 - 1024 - 0.15 - 8 - 10 - 99 - - - - - file-body - - - - - - true - sample - file-body - local - - - - - - -``` +1. With `command: Plan` (the shipped value), `run.bat` loads no model and writes nothing. It prints + which file gets which model, and which files are larger than the model's window. +2. Then set `command: GenerateFileIndex`, or `All` to add package and project summaries, and run + `run.bat` again. -In XML a Windows path needs no escaping. Then run: +A second run regenerates only three kinds of files: a changed file, a file whose settings changed, +and a file whose summary is empty. An empty summary usually means `maxOutputTokens` was too small for +the model. -```bat -cd /d C:\path\to\project -mvn srcmorph:generate -Dsrcmorph.planOnly=true -mvn srcmorph:generate -``` +## Check what loaded -The first command prints the plan, like the CLI's `Plan`. The second writes the index. The plugin -also has `aggregate-packages`, `aggregate-project` and `calibrate` goals; see the -[plugin README](../srcmorph-maven-plugin/README.md) for binding them to phases. - -**Before 1.3.0 is released**, version `1.3.0-SNAPSHOT` resolves only with Central's snapshot -repository added to the project: - -```xml - - - central-snapshots - https://central.sonatype.com/repository/maven-snapshots/ - false - true - - -``` - -## Thread count - -The biggest single knob on this CPU. On Windows ggml runs its own thread pool and synchronises on a -barrier per operation. **The barrier waits for the slowest thread**, and the two LP-E-cores are much -slower than the P- and E-cores, so all 14 threads is very likely not the optimum. - -Measure `threads` 6, 8 and 10 with `Calibrate` and keep the best. `threadsBatch` sets the -prompt-processing thread count separately and tolerates a higher value, because prompt processing has -enough work per barrier to hide the synchronisation cost. srcmorph sends long prompts, so -`threadsBatch` matters more here than in a chat. - -## Verifying what loaded - -The log names the CPU variant, the backends and the devices ggml ended up with. From the Vulkan run on -the test machine (an AMD Ryzen, hence `haswell`; a Core Ultra shows `alderlake`): +The start of the run names the CPU variant, the backends and the devices: ``` load_backend: loaded Vulkan backend from \ggml-vulkan.dll -load_backend: loaded CPU backend from \ggml-cpu-haswell.dll -load_backends_next_to_this_library: 3 backend(s) registered from [RPC, Vulkan, CPU], 3 device(s): Vulkan0 (Vulkan), Vulkan1 (Vulkan), CPU (CPU) +load_backend: loaded CPU backend from \ggml-cpu-alderlake.dll +load_backends_next_to_this_library: 3 backend(s) registered [RPC, Vulkan, CPU], 2 device(s): Vulkan0 (Vulkan), CPU (CPU) [jllama] native backend 'cpu' loaded from with GPU module(s) vulkan; 18 file(s) in place for ggml to load ``` -- **Vulkan module missing from the last line** (`with no GPU module`): you are running the plain jar, - or the plugin dependency is missing. -- **More than one Vulkan device** (`Vulkan0`, `Vulkan1`, …, as on a laptop with an iGPU and a - discrete GPU): pick one by name with `devices: Vulkan1` in the `aiDefinition`. Device indices depend - on which modules loaded, so always name the device, never rely on an index. - -A Core Ultra 7 165U has a single GPU, so `Vulkan0` is the Arc iGPU. - -`-Dnet.ladenthin.llama.backend=cpu` before `-jar` (or on the `mvn` command line) puts no GPU module in -place. That gives the clean CPU baseline for a comparison without switching jars. - -## The NPU - -`openvino-windows-x86-64` is the only path to the Intel **NPU**, and it is **opt-in** in the binding: -its jar does nothing until `-Dnet.ladenthin.llama.backend` names it. The reason is measured: OpenVINO's -CPU plugin registers as a device that llama.cpp then prefers, and on a host without an Intel -accelerator that dropped token generation to 0.2 tokens per second -([java-llama.cpp, gpu-module-jars.md](https://github.com/bernardladenthin/java-llama.cpp/blob/main/docs/benchmarks/gpu-module-jars.md)). -It also needs the Intel OpenVINO runtime DLLs on `PATH`. srcmorph ships no OpenVINO fat jar. With the -plugin, add the `openvino-windows-x86-64` natives jar as a second plugin dependency, run with -`-Dnet.ladenthin.llama.backend=vulkan,openvino`, and read the device list before trusting a number. -NPU use is **not measured** here. +- **`with no GPU module`**: the Vulkan jar is not in the build. +- **Two Vulkan devices** (an iGPU and a discrete GPU): choose one with `devices: Vulkan1` in the + `aiDefinition`. +- **`gpuLayers: 0`**, or `-Dnet.ladenthin.llama.backend=cpu` before `-jar` in `run.bat`: runs on the + CPU only. That is the comparison to measure. + +## Two settings worth measuring + +- **`threads`**: a Core Ultra mixes P-, E- and slow LP-E-cores, and ggml waits for the slowest thread. + Try 6, 8 and 10 rather than all cores. +- **`gpuLayers: 99` against `0`**: the iGPU shares memory and a 15 W budget with the CPU, so measure + before deciding. `command: Calibrate` prints each model's prefill and decode speed. + +## Tested + +On Windows, `cmd.exe`, srcmorph 1.3.0-SNAPSHOT: +- `build.bat` ran on a JDK 8 (1.8.0_504) and produced a 32 MiB jar holding only the Windows x86-64 + natives. +- `Plan` ran on the same JDK 8. +- `run.bat` indexed 6 Java files with Qwen3-0.6B on the Vulkan module, in 17 s, on a JDK 21. The test + machine has no JDK 8 installed. +- The test machine is a Ryzen with an AMD iGPU and an RTX 3070, which is why the log above was + written down for a Core Ultra rather than copied from it. diff --git a/examples/windows-fatjar/build.bat b/examples/windows-fatjar/build.bat new file mode 100644 index 0000000..3645b7a --- /dev/null +++ b/examples/windows-fatjar/build.bat @@ -0,0 +1,11 @@ +REM SPDX-FileCopyrightText: 2026 Bernard Ladenthin +REM +REM SPDX-License-Identifier: MIT OR Apache-2.0 + +@echo off +rem Builds target\srcmorph.jar from pom.xml (the CLI plus the natives the pom names). Adapt JAVA_HOME. +cd /d "%~dp0" +set "JAVA_HOME=C:\Program Files\Java\openjdk1.8.0_491" +set "PATH=%JAVA_HOME%\bin;%PATH%" + +mvn -B package diff --git a/examples/windows-fatjar/pom.xml b/examples/windows-fatjar/pom.xml new file mode 100644 index 0000000..8f5ee7f --- /dev/null +++ b/examples/windows-fatjar/pom.xml @@ -0,0 +1,95 @@ + + + + + 4.0.0 + local + srcmorph-local + 1 + pom + + + 1.3.0-SNAPSHOT + 6.0.0 + UTF-8 + + + + + + central-snapshots + https://central.sonatype.com/repository/maven-snapshots/ + false + true + + + + + + net.ladenthin + srcmorph-cli + ${srcmorph.version} + + + + net.ladenthin + llama-platform + + + + + + net.ladenthin + llama + ${llama.version} + cpu-windows-x86-64 + + + + net.ladenthin + llama + ${llama.version} + vulkan-windows-x86-64 + + + + + + + org.apache.maven.plugins + maven-assembly-plugin + 3.7.1 + + + jar-with-dependencies + + srcmorph + false + + + net.ladenthin.srcmorph.cli.Main + + + + + + package + + single + + + + + + + diff --git a/examples/windows-fatjar/run.bat b/examples/windows-fatjar/run.bat new file mode 100644 index 0000000..c926df0 --- /dev/null +++ b/examples/windows-fatjar/run.bat @@ -0,0 +1,11 @@ +REM SPDX-FileCopyrightText: 2026 Bernard Ladenthin +REM +REM SPDX-License-Identifier: MIT OR Apache-2.0 + +@echo off +rem Runs srcmorph with srcmorph.yaml from this directory. Adapt JAVA_HOME. +cd /d "%~dp0" +set "JAVA_HOME=C:\Program Files\Java\openjdk1.8.0_491" +set "PATH=%JAVA_HOME%\bin;%PATH%" + +java -jar target\srcmorph.jar srcmorph.yaml diff --git a/examples/windows-fatjar/srcmorph.yaml b/examples/windows-fatjar/srcmorph.yaml new file mode 100644 index 0000000..8d4ce61 --- /dev/null +++ b/examples/windows-fatjar/srcmorph.yaml @@ -0,0 +1,46 @@ +# SPDX-FileCopyrightText: 2026 Bernard Ladenthin +# +# SPDX-License-Identifier: MIT OR Apache-2.0 + +# What run.bat does with it: every file under baseDirectory/subtrees with one of the fileExtensions +# is matched to a rule in fieldGenerations. The rule names a prompt (promptDefinitions) and a model +# (aiDefinitions). srcmorph sends the prompt as the system message and the file's name and source as +# the user message, and writes the model's answer to outputDirectory as .ai.md. +# +# command: +# Plan prints which file gets which model and prompt; loads nothing, writes nothing +# GenerateFileIndex one .ai.md per source file +# All GenerateFileIndex, then one summary per package and one for the project +# Start with Plan, then switch to GenerateFileIndex. A second run only redoes changed files. +command: Plan + +srcMorph: + # Use forward slashes in Windows paths here: in a double-quoted YAML string, "C:\x" is an escape. + baseDirectory: "C:/path/to/project" + outputDirectory: "C:/path/to/project/target/srcmorph-ai" + subtrees: + - "src/main/java" + fileExtensions: + - ".java" + + generationProvider: llamacpp-jni # "mock" runs without a model, for a dry test of the setup + modelDirectory: "C:/models" # a relative modelPath below is relative to this directory + + promptDefinitions: + - key: java + template: "Summarize this Java file as structured Markdown: purpose, public API, collaborators." + + aiDefinitions: + - key: local + modelPath: granite-4.0-h-tiny-Q4_K_M.gguf + contextSize: 16384 # tokens; a file larger than this is handled by onOversize + maxOutputTokens: 1024 + temperature: 0.15 + threads: 8 # measure 6, 8, 10; all cores is rarely the fastest + gpuLayers: 99 # 99 = everything on the GPU (Vulkan), 0 = CPU only + + fieldGenerations: + - promptKey: java + aiDefinitionKey: local + fallback: true # the rule for every file no other rule matches + onOversize: sample # default "fail" stops the run on a file larger than the window diff --git a/srcmorph-cli/README.md b/srcmorph-cli/README.md index bec0581..1aef909 100644 --- a/srcmorph-cli/README.md +++ b/srcmorph-cli/README.md @@ -26,8 +26,9 @@ is one classifier variant per llama GPU backend module — plus that one backend's ggml module. ggml loads the module when the matching vendor runtime is installed and skips it otherwise, so a GPU variant never runs worse than the plain jar; it only runs *faster* on the matching OS/arch with that GPU runtime present. Pick the plain CPU jar unless you specifically want a -GPU build. A worked Windows example with the Vulkan jar (configuration, plan, index, what the log must -show) is in [`../docs/windows-intel-core-ultra.md`](../docs/windows-intel-core-ultra.md). +GPU build. To build your own fat jar with exactly the classifiers of your machine instead, see +[`../docs/windows-intel-core-ultra.md`](../docs/windows-intel-core-ultra.md) and +[`../examples/windows-fatjar/`](../examples/windows-fatjar/). Both `.json`/`.js` (parsed via a Jackson `ObjectMapper`) and `.yaml`/`.yml` (parsed via `YAMLMapper`) are supported — pick whichever you diff --git a/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java b/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java index 2a722a0..352bfeb 100644 --- a/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java +++ b/srcmorph-cli/src/test/java/net/ladenthin/srcmorph/cli/ExamplesConfigBindingTest.java @@ -4,6 +4,7 @@ package net.ladenthin.srcmorph.cli; import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.greaterThan; import static org.hamcrest.Matchers.is; @@ -15,9 +16,13 @@ import java.nio.file.Path; import java.util.ArrayList; import java.util.List; +import net.ladenthin.srcmorph.cli.configuration.CCommand; import net.ladenthin.srcmorph.cli.configuration.CConfiguration; +import net.ladenthin.srcmorph.config.AiFieldGenerationConfig; +import net.ladenthin.srcmorph.config.AiOversizeStrategy; import net.ladenthin.srcmorph.config.SrcMorphConfiguration; import org.junit.jupiter.api.DynamicTest; +import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestFactory; /** @@ -77,6 +82,33 @@ private static List listExampleConfigFiles() throws IOException { return found; } + /** + * {@code examples/windows-fatjar/} is the build-it-yourself example (docs/windows-intel-core-ultra.md): + * its {@code srcmorph.yaml} is meant for a real model, so it is not one of the mock fixtures above, + * but it must still bind under the strict mapper, start with the safe {@code Plan}, and route every + * file through a fallback rule that handles an oversized file. Its {@code pom.xml} names the CLI's + * main class by string, so a renamed {@link Main} would otherwise only fail on the user's machine. + * + * @throws IOException if an example file cannot be read + */ + @Test + public void windowsFatJarExample_bindsAndNamesTheMainClass() throws IOException { + final Path directory = EXAMPLES_DIRECTORY.resolve("windows-fatjar"); + final CConfiguration configuration = Main.fromYaml(Main.readString(directory.resolve("srcmorph.yaml"))); + + assertThat(configuration.command, is(CCommand.Plan)); + final SrcMorphConfiguration srcMorph = configuration.srcMorph; + assertThat(srcMorph.getGenerationProvider(), is(equalTo("llamacpp-jni"))); + assertThat(srcMorph.getFieldGenerations().size(), is(1)); + final AiFieldGenerationConfig rule = srcMorph.getFieldGenerations().get(0); + assertThat(rule.isFallback(), is(true)); + assertThat(rule.getOnOversize(), is(equalTo(AiOversizeStrategy.SAMPLE.configValue()))); + + assertThat( + Main.readString(directory.resolve("pom.xml")), + containsString("" + Main.class.getName() + "")); + } + private static void assertBindsSuccessfully(final Path exampleConfig) throws IOException { final String fileName = exampleConfig.getFileName().toString(); final String content = Main.readString(exampleConfig); diff --git a/srcmorph-maven-plugin/README.md b/srcmorph-maven-plugin/README.md index 8502e38..3eebc55 100644 --- a/srcmorph-maven-plugin/README.md +++ b/srcmorph-maven-plugin/README.md @@ -606,8 +606,8 @@ In both cases: - **CUDA** needs a matching CUDA 13 toolkit + driver, and the toolkit's `bin\x64` (with `cudart64_13.dll`, `cublas64_13.dll`, `cublasLt64_13.dll`) on `PATH` — the natives jar bundles only the backend module `ggml-cuda.dll`, not the CUDA runtime. -- **Vulkan** needs only the GPU driver (it brings `vulkan-1.dll`). A worked Windows setup, both the - plugin and the CLI with the Vulkan natives, is in [docs/windows-intel-core-ultra.md](../docs/windows-intel-core-ultra.md). +- **Vulkan** needs only the GPU driver (it brings `vulkan-1.dll`). A CLI fat jar built with exactly + the CPU and Vulkan natives of a Windows machine is in [docs/windows-intel-core-ultra.md](../docs/windows-intel-core-ultra.md). - **GPU layer offload** — set `` inside your `` (see "Per-model `` parameters" above): `-1` (default) does **not** pin a layer count, so llama.cpp **auto-fits** as many layers as fit the card's free VRAM — the robust "runs on any card" setting (it From 0f53d46e78dbde6148b6c2a3b4b7a919ee5f9796 Mon Sep 17 00:00:00 2001 From: Bernard Ladenthin Date: Mon, 26 Oct 2026 16:28:47 +0100 Subject: [PATCH 3/3] Windows example: threads 4, the new default (srcmorph#245) The example's model definition named threads: 8 with the advice to try 6, 8 and 10. With the default moving to min(4, cores) (srcmorph#245), the example takes 4 and the guide says why: more threads than cores is orders of magnitude slower. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AJGh28Gu8pLWYbTCqG1LZf --- docs/windows-intel-core-ultra.md | 5 +++-- examples/windows-fatjar/srcmorph.yaml | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/windows-intel-core-ultra.md b/docs/windows-intel-core-ultra.md index a8d72a4..9c8d978 100644 --- a/docs/windows-intel-core-ultra.md +++ b/docs/windows-intel-core-ultra.md @@ -97,8 +97,9 @@ load_backends_next_to_this_library: 3 backend(s) registered [RPC, Vulkan, CPU], ## Two settings worth measuring -- **`threads`**: a Core Ultra mixes P-, E- and slow LP-E-cores, and ggml waits for the slowest thread. - Try 6, 8 and 10 rather than all cores. +- **`threads`**: 4 is the default and enough for these models. A Core Ultra mixes P-, E- and slow + LP-E-cores, and ggml waits for the slowest thread, so more threads can be slower. Never set more + than the machine has cores: an oversubscribed run is orders of magnitude slower, not a little. - **`gpuLayers: 99` against `0`**: the iGPU shares memory and a 15 W budget with the CPU, so measure before deciding. `command: Calibrate` prints each model's prefill and decode speed. diff --git a/examples/windows-fatjar/srcmorph.yaml b/examples/windows-fatjar/srcmorph.yaml index 8d4ce61..fef557d 100644 --- a/examples/windows-fatjar/srcmorph.yaml +++ b/examples/windows-fatjar/srcmorph.yaml @@ -36,7 +36,7 @@ srcMorph: contextSize: 16384 # tokens; a file larger than this is handled by onOversize maxOutputTokens: 1024 temperature: 0.15 - threads: 8 # measure 6, 8, 10; all cores is rarely the fastest + threads: 4 # the default; more only after measuring, never more than cores gpuLayers: 99 # 99 = everything on the GPU (Vulkan), 0 = CPU only fieldGenerations: