Add apple_fm integration for Apple's on-device model

CHANGELOG.md

@@ -5,6 +5,22 @@ under Unreleased and move into a version section at release time.
 
 ## Unreleased
 
+### Added
+
+- `apple_fm` integration tracks Apple's on-device model through apple-fm-sdk.
+  `LanguageModelSession.respond()` and `stream_response()` record duration,
+  time to first token for streams, prompt size and generation options.
+  Failures record the SDK's error name and a matching error code.
+  `SystemLanguageModel.is_available()` records a model load, failed with
+  the reason when the model can't be used, such as `MODEL_NOT_READY`. Token
+  counts, from `SystemLanguageModel.token_count()`, are opt-in with
+  `WILDEDGE_APPLE_FM_COUNT_TOKENS=1`. Its accelerator is `npu`, a new
+  accelerator name, since the model runs on the Neural Engine. See
+  docs/apple_fm.md.
+- Error codes `MODEL_UNAVAILABLE`, `CONTEXT_LENGTH_EXCEEDED`,
+  `CONTENT_FILTERED`, `RATE_LIMITED`, `BAD_REQUEST`, `INVALID_OUTPUT` and
+  `TOOL_CALL_FAILED`, listed in docs/manual-tracking.md.
+
 ## 0.3.0 - 2026-10-24
 
 ### Added

README.md

@@ -180,6 +180,7 @@ explicit registration through `client.load(...)` or `client.register_model(...)`
 | Transformers | `transformers` | Automatically through `pipeline` / `from_pretrained` | [Example](examples/transformers_example.py) |
 | llama.cpp (`llama-cpp-python`) | `llama_cpp` | Automatically when constructing `Llama` | [Example](examples/gguf_example.py) |
 | MLX-LM | `mlx` | Automatically through `mlx_lm.load` | [Example](examples/mlx_example.py) |
+| Apple Foundation Models (`apple-fm-sdk`) | `apple_fm` | Automatically for sessions created after initialization ([docs](docs/apple_fm.md)) | [Example](examples/apple_fm_example.py) |
 | faster-whisper / MLX Whisper | `faster_whisper` / `mlx_whisper` | Automatically through supported model loaders | [Example](examples/whisper_example.py) |
 | ONNX Runtime | `onnx` | Automatically when constructing `InferenceSession` | [Example](examples/onnx_example.py) |
 | timm | `timm` | Automatically through `create_model` | [Example](examples/timm_example.py) |

docs/apple_fm.md

@@ -0,0 +1,107 @@
+# Apple Foundation Models
+
+The `apple_fm` integration tracks the on-device model behind Apple
+Intelligence, called through Apple's Python SDK
+([apple-fm-sdk](https://github.com/apple/python-apple-fm-sdk)). It needs macOS
+26 or later on Apple silicon with Apple Intelligence turned on, plus Xcode to
+install the SDK.
+
+Initialize the SDK before creating sessions:
+
+```python
+import asyncio
+
+import apple_fm_sdk as fm
+import wildedge
+
+client = wildedge.init(integrations=["apple_fm"])
+
+
+async def main():
+    session = fm.LanguageModelSession(instructions="Answer in one sentence.")
+    print(await session.respond("How does rain form?"))
+    # Each snapshot is the full text so far, not just the new part.
+    text = ""
+    async for snapshot in session.stream_response("Name three clouds."):
+        text = snapshot
+    print(text)
+
+
+asyncio.run(main())
+client.close(timeout=10)
+```
+
+## What is recorded
+
+Each `respond()` or `stream_response()` call records one inference event with
+its duration, the prompt's character and word counts, and the
+`GenerationOptions` you passed (temperature, maximum response tokens, and
+sampling top-k, threshold and seed). Streams also record time to first token.
+
+Sessions share one model per use case: `apple-foundation-model` for the
+general model, and `apple-foundation-model-content-tagging` for
+`SystemLanguageModelUseCase.CONTENT_TAGGING`, also for sessions resumed with
+`from_transcript()`. The model version is the macOS
+version, since the model ships with the OS. The accelerator is `npu`, since
+the model runs on the Neural Engine.
+
+### Token counts
+
+Token counts come from `SystemLanguageModel.token_count()` and are opt-in.
+Set `WILDEDGE_APPLE_FM_COUNT_TOKENS=1`, and each call records tokens in (the
+prompt, not the instructions or earlier turns), tokens out (the reply) and
+tokens per second.
+
+Counting costs two calls to the model service per request, about 40 ms each
+on an M2 Mac. They run in the background after the response returns, so your
+call is never slower. macOS queues them behind generation, though, so they
+can delay a request that follows right after by about 80 ms. The event is
+sent once counting finishes, usually soon after the model is idle.
+
+Structured replies (`generating=` or a schema) count tokens in only. A count
+that fails sends the event without token counts. So does the last call before
+the event loop closes, since `asyncio.run()` cancels the pending count.
+
+### Availability
+
+`SystemLanguageModel.is_available()` records a model load event. When the
+model can't be used, the load fails with the SDK's reason as its error code:
+`APPLE_INTELLIGENCE_NOT_ENABLED`, `DEVICE_NOT_ELIGIBLE`, `MODEL_NOT_READY`
+(still downloading) or `UNKNOWN`. Apps often poll `is_available()` while the
+model downloads, so an event is sent only when the answer changes, once per
+process and model.
+
+### Errors
+
+Failed calls record an error event. Its `provider_error_code` is the SDK's
+error name, and its error code says what kind of failure it was:
+
+| SDK error | `provider_error_code` | Error code |
+|---|---|---|
+| `AssetsUnavailableError` | `assets_unavailable` | `MODEL_UNAVAILABLE` |
+| `ExceededContextWindowSizeError` | `exceeded_context_window_size` | `CONTEXT_LENGTH_EXCEEDED` |
+| `GuardrailViolationError` | `guardrail_violation` | `CONTENT_FILTERED` |
+| `RefusalError` | `refusal` | `CONTENT_FILTERED` |
+| `RateLimitedError` | `rate_limited` | `RATE_LIMITED` |
+| `ConcurrentRequestsError` | `concurrent_requests` | `RATE_LIMITED` |
+| `UnsupportedGuideError` | `unsupported_guide` | `BAD_REQUEST` |
+| `UnsupportedLanguageOrLocaleError` | `unsupported_language_or_locale` | `BAD_REQUEST` |
+| `InvalidGenerationSchemaError` | `invalid_schema` | `BAD_REQUEST` |
+| `DecodingFailureError` | `decoding_failure` | `INVALID_OUTPUT` |
+| `ToolCallError` | `tool_call` | `TOOL_CALL_FAILED` |
+
+Any other failure records `UNKNOWN`.
+
+Calling the model before it's ready fails with `AssetsUnavailableError`.
+
+## Caveats
+
+- Token counts are opt-in, see [Token counts](#token-counts).
+- Prompts and responses are not uploaded.
+
+## `fm serve`
+
+On macOS 27, `fm serve` exposes the same model through a local
+OpenAI-compatible server. Calls made to it with the `openai` client are
+tracked by the `openai` integration. Name the provider, because a local host is
+otherwise recorded as `127.0.0.1` (see [Providers](providers.md#other-hosts)).

docs/compatibility.md

@@ -29,7 +29,8 @@ once the bug is fixed and the marker has to come off.
 On pull requests into `main` or a release branch, on pushes to them, nightly
 against `devel`, and on manual runs. Every run covers all the combinations
 below. Python 3.14 rows are canaries and don't fail the run. mlx runs on
-macOS.
+macOS. apple_fm runs on macOS 26, where Apple Intelligence is off, so its
+test checks that a failed call is recorded as an error event.
 
 To run the full matrix locally: `python scripts/run_compat_local.py`.
 `python scripts/compat_matrix.py table` prints the table below.
@@ -69,3 +70,4 @@ To run the full matrix locally: `python scripts/run_compat_local.py`.
 | `mlx` | `current` | `mlx==0.32.4`, `mlx-lm==0.32.2` | 3.12, 3.13 |
 | `faster_whisper` | `current` | `faster-whisper==1.2.1`, `ctranslate2==4.8.2`, `numpy==2.2.6` | 3.12, 3.13 |
 | `mlx_whisper` | `current` | `mlx-whisper==0.4.3`, `mlx==0.32.3`, `numpy==2.2.6` | 3.12, 3.13 |
+| `apple_fm` | `current` | `apple-fm-sdk==0.2.1` | 3.12, 3.13 |

docs/configuration.md

@@ -47,6 +47,15 @@ inputs or outputs.
 
 See [Track attachments](manual-tracking.md#track-attachments) for usage.
 
+## Integration settings
+
+Settings for a single integration are environment variables only. They are
+read when the integration is activated.
+
+| Env var | Default | Description |
+|---|---|---|
+| `WILDEDGE_APPLE_FM_COUNT_TOKENS` | `false` | Count tokens for Apple's on-device model, at about 80 ms of background work per call. See [Token counts](apple_fm.md#token-counts) |
+
 ### When ML libraries are imported
 
 `import wildedge` does not import any ML framework or API client. The SDK imports a library only when you use it:

docs/manual-tracking.md

@@ -233,6 +233,25 @@ except Exception as exc:
 
 `wildedge.track` (decorator and context manager) captures errors automatically when `capture_errors=True` (the default).
 
+Use the most specific `ErrorCode` that fits, or `UNKNOWN`:
+
+| Code | Meaning |
+|---|---|
+| `OOM` | Ran out of memory |
+| `CORRUPTED_MODEL` | Model file is damaged or unreadable |
+| `INFERENCE_TIMEOUT` | The call timed out |
+| `UNSUPPORTED_OP` | The runtime can't run an operation in the model |
+| `THERMAL_SHUTDOWN` | The device stopped work because it was too hot |
+| `CONNECTION_ERROR` | A remote model API couldn't be reached |
+| `MODEL_UNAVAILABLE` | The model or its assets aren't on the device yet |
+| `CONTEXT_LENGTH_EXCEEDED` | The prompt and history don't fit the model's context window |
+| `CONTENT_FILTERED` | A safety filter blocked the input or output, or the model refused |
+| `RATE_LIMITED` | Too many requests, or too many at once |
+| `BAD_REQUEST` | The request was invalid, such as an unsupported language or schema |
+| `INVALID_OUTPUT` | The model's output couldn't be parsed into the requested structure |
+| `TOOL_CALL_FAILED` | A tool the model called raised an error |
+| `UNKNOWN` | Anything else |
+
 For failed calls to a remote model API, pass `http_status` and, if the provider returned one, `provider_error_code` with `error_code="UNKNOWN"`; Wild Edge derives the error code from them. The openai and anthropic integrations do this automatically, and report timeouts and connection failures as `INFERENCE_TIMEOUT` and `CONNECTION_ERROR`.
 
 ## Track feedback

docs/providers.md

@@ -49,6 +49,9 @@ wildedge.llm_api(model="gemma-7b", provider="ollama")
 wildedge.register_model(client, source="ollama", model_id="gemma-7b")
 ```
 
+The same applies to `fm serve`, Apple's local server for its on-device model on
+macOS 27. To track that model without a server, see [Apple Foundation Models](apple_fm.md).
+
 ## Caveats
 
 - `tokens_in` / `tokens_out` are populated by every provider above.

examples/apple_fm_example.py

@@ -0,0 +1,52 @@
+# /// script
+# requires-python = ">=3.10"
+# dependencies = ["wildedge-sdk", "apple-fm-sdk"]
+#
+# [tool.uv.sources]
+# wildedge-sdk = { path = "..", editable = true }
+# ///
+"""Apple Foundation Models integration example.
+
+WildEdge patches apple_fm_sdk.LanguageModelSession at instrumentation time, so
+every respond() and stream_response() call on Apple's on-device model is
+tracked.
+
+Run with: uv run apple_fm_example.py
+Requires: macOS 26+ on Apple silicon with Apple Intelligence on, and Xcode to
+install apple-fm-sdk. Set WILDEDGE_DSN to send events.
+"""
+
+import asyncio
+
+import apple_fm_sdk as fm
+
+import wildedge
+
+client = wildedge.init(
+    app_version="1.0.0",  # uses WILDEDGE_DSN if set; otherwise no-op
+    integrations="apple_fm",
+)
+
+
+async def main() -> None:
+    model = fm.SystemLanguageModel()
+    available, reason = model.is_available()
+    if not available:
+        raise SystemExit(f"Apple's on-device model is unavailable: {reason.name}")
+
+    session = fm.LanguageModelSession(
+        instructions="Answer in one sentence.", model=model
+    )
+    print(await session.respond("Explain transformer attention."))
+
+    # Each snapshot is the full text so far, not just the new part.
+    text = ""
+    async for snapshot in session.stream_response("Write a haiku about edge AI."):
+        text = snapshot
+    print(text)
+
+
+try:
+    asyncio.run(main())
+finally:
+    client.close(timeout=10)

scripts/build_llms_txt.py

@@ -26,6 +26,7 @@
     ("docs/providers.md", "providers", "Providers"),
     ("docs/streaming.md", "streaming", "Streaming lifecycle"),
     ("docs/whisper.md", "whisper", "Whisper"),
+    ("docs/apple_fm.md", "apple_fm", "Apple Foundation Models"),
     ("docs/compatibility.md", "compatibility", "Compatibility matrix"),
 ]
 

scripts/compat_matrix.py

@@ -62,6 +62,7 @@
         "current": ["faster-whisper==1.2.1", "ctranslate2==4.8.2", "numpy==2.2.6"]
     },
     "mlx_whisper": {"current": ["mlx-whisper==0.4.3", "mlx==0.32.3", "numpy==2.2.6"]},
+    "apple_fm": {"current": ["apple-fm-sdk==0.2.1"]},
 }
 
 SUPPORTED_PYTHON = {
@@ -77,6 +78,7 @@
     "mlx": ["3.12", "3.13"],
     "faster_whisper": ["3.12", "3.13"],
     "mlx_whisper": ["3.12", "3.13"],
+    "apple_fm": ["3.12", "3.13"],
 }
 
 # Integrations without a row of their own, and the row that exercises them.
@@ -88,6 +90,8 @@
 RUNNER = {
     "mlx": "macos-14",
     "mlx_whisper": "macos-14",
+    # apple-fm-sdk builds Swift bindings at install, which needs Xcode 26.
+    "apple_fm": "macos-26",
 }
 
 # Python versions run as non-blocking canaries, and only with current pins.

tests/compat/test_apple_fm_compat.py

@@ -0,0 +1,79 @@
+"""Real apple-fm-sdk sessions.
+
+CI runners can't turn on Apple Intelligence, so a call there fails with the
+SDK's own error. Either way the availability check must produce one load
+event, and the call exactly one event, for the model it ran on.
+"""
+
+import asyncio
+import inspect
+
+import apple_fm_sdk as fm
+
+
+def test_patched_signatures_match_the_sdk():
+    """The patches read these arguments by position."""
+    model_params = list(inspect.signature(fm.SystemLanguageModel).parameters)
+    session_params = list(inspect.signature(fm.LanguageModelSession).parameters)
+    stream_params = list(
+        inspect.signature(fm.LanguageModelSession.stream_response).parameters
+    )
+    respond = inspect.signature(fm.LanguageModelSession.respond).parameters
+    assert model_params[0] == "use_case"
+    assert session_params[1] == "model"
+    assert stream_params[1:3] == ["prompt", "options"]
+    assert respond["options"].kind is inspect.Parameter.KEYWORD_ONLY
+
+
+def test_respond_records_one_event(compat_client):
+    compat_client.instrument("apple_fm")
+    model = fm.SystemLanguageModel(
+        use_case=fm.SystemLanguageModelUseCase.CONTENT_TAGGING
+    )
+    available, reason = model.is_available()
+    session = fm.LanguageModelSession(model=model)
+    options = fm.GenerationOptions(maximum_response_tokens=64)
+
+    async def respond():
+        try:
+            return await session.respond("Tag this: rain", options=options)
+        except fm.FoundationModelsError as exc:
+            return exc
+
+    result = asyncio.run(respond())
+
+    [load, event] = compat_client.events
+    assert load["event_type"] == "model_load"
+    assert load["load"]["success"] is available
+    if not available:
+        assert load["load"]["error_code"] == reason.name
+    assert event["model_id"] == "apple-foundation-model-content-tagging"
+    if available:
+        assert isinstance(result, str), result
+        assert event["event_type"] == "inference"
+        assert event["inference"]["generation_config"]["max_tokens"] == 64
+    else:
+        assert isinstance(result, fm.FoundationModelsError), (reason, result)
+        assert event["event_type"] == "error"
+        assert event["error"]["provider_error_code"] is not None
+
+
+def test_resumed_session_keeps_its_model(compat_client):
+    """from_transcript() creates the session without passing the model to __init__."""
+    compat_client.instrument("apple_fm")
+    model = fm.SystemLanguageModel(
+        use_case=fm.SystemLanguageModelUseCase.CONTENT_TAGGING
+    )
+    original = fm.LanguageModelSession(model=model)
+    session = fm.LanguageModelSession.from_transcript(original.transcript, model=model)
+
+    async def respond():
+        try:
+            return await session.respond("Tag this: snow")
+        except fm.FoundationModelsError as exc:
+            return exc
+
+    asyncio.run(respond())
+
+    [event] = [e for e in compat_client.events if e["event_type"] != "model_load"]
+    assert event["model_id"] == "apple-foundation-model-content-tagging"