diff --git a/CHANGELOG.md b/CHANGELOG.md
index 513840a9..c61a74dd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,11 @@ All notable changes to SkillOpt are documented here. This project adheres to
 ## [Unreleased]
 
 ### Added
+- **SkillOpt-Sleep paired A/B evalkit** (`python -m skillopt_sleep.evalkit`):
+  McNemar plus percentile-bootstrap CIs on a fixed task manifest, with
+  task-cluster multi-seed inference and seeded null calibration for exact-test
+  type-I error and bootstrap coverage. The nightly gate is unchanged
+  (thanks @bogdanbaciu21).
 - **SkillOpt-Sleep multi-skill fan-out and reviewed subset adoption**: each
   hinted skill is consolidated from its own pinned live baseline, staged as an
   independent proposal with per-skill gate evidence, and promoted only through
diff --git a/docs/reference/cli.md b/docs/reference/cli.md
index f0ea40a5..6aefbb20 100644
--- a/docs/reference/cli.md
+++ b/docs/reference/cli.md
@@ -123,8 +123,12 @@ skillopt-sleep <action> [options]
 python -m skillopt_sleep <action> [options]
 ```
 
-Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
-`unschedule`. Common options include:
+Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`,
+`unschedule`, and `evalkit`. `evalkit` is also available as
+`python -m skillopt_sleep.evalkit` and compares two conditions on one fixed
+task manifest (McNemar + bootstrap CI). Exactly one of its `--b` comparison
+input or `--aa` identity-check flag is required. See `docs/sleep/evalkit.md`.
+Common options for the nightly actions include:
 
 | Argument | Description |
 |---|---|
diff --git a/docs/sleep/README.md b/docs/sleep/README.md
index 2576c127..f47556ee 100644
--- a/docs/sleep/README.md
+++ b/docs/sleep/README.md
@@ -339,6 +339,21 @@ correctness signal; the validation gate still governs what ships.
 | `recall_k` | `0` | Associative recall — pull the K most-similar past tasks (from a persisted archive) into tonight's dream. |
 | `dream_factor` | `0` | Add N lightweight synthetic variants of each task. |
 
+### Paired A/B evalkit
+
+Reports and PRs that claim "B beats A" should go through the shared evalkit
+rather than quoting a single-run cell. One command pairs two conditions on one
+fixed task manifest, runs McNemar's test, and reports a bootstrap CI on the
+success-rate delta. The nightly gate is unchanged.
+
+```text
+python -m skillopt_sleep.evalkit --manifest tasks.json --a cond_a.json --b cond_b.json
+```
+
+See [`evalkit.md`](evalkit.md) for the id-set contract, task-cluster inference
+over positional seed repeats, A/A checks, and the point-estimate-only published
+RESULTS cell replay.
+
 ## Results
 
 > 📊 **More results & analysis — the gate-safety stress test, experience-replay
diff --git a/docs/sleep/evalkit.md b/docs/sleep/evalkit.md
new file mode 100644
index 00000000..20565c89
--- /dev/null
+++ b/docs/sleep/evalkit.md
@@ -0,0 +1,77 @@
+# Paired A/B evalkit
+
+Sleep contributors have a shared instrument for "condition B beats condition A":
+
+```text
+python -m skillopt_sleep.evalkit --manifest tasks.json --a cond_a.json --b cond_b.json
+```
+
+The kit pairs outcomes by task id, runs McNemar's test on binary successes, and
+reports a percentile-bootstrap confidence interval on the success-rate delta.
+It does not change the nightly gate.
+
+## Inputs
+
+- `--manifest`: JSON list of task ids, or `{"ids": [...]}` / `{"tasks": [{"id": ...}]}`.
+  Every id must be a non-empty JSON string; ids are never coerced from numbers,
+  booleans, nulls, arrays, or objects.
+- `--a` / `--b`: JSON objects mapping those same ids to `0`/`1` (or an ordered
+  list of repeated-seed `0`/`1` values). A wrapper `{"outcomes": {...}}` is also
+  accepted. Direct keys that exactly match the manifest take precedence over
+  wrapper detection, so a task literally named `outcomes` remains unambiguous.
+- `--aa`: A/A identity smoke check (reuses `--a` as both conditions).
+  Exactly one of `--b` or `--aa` is required.
+- `--allow-graded`: permit non-binary scores. McNemar is omitted; bootstrap only.
+- `--boot`, `--seed`, `--alpha`, `--json`.
+
+The id sets of the manifest, A, and B must be identical. Cross-manifest
+comparisons and duplicate JSON object keys are refused. Seed lists must be
+non-empty. Scores must be JSON numbers (never booleans or numeric strings),
+finite, and in `[0, 1]`. `alpha` must be strictly between 0 and 1, and `--boot`
+must be between 1 and 1,000,000. The total bootstrap workload is capped at
+50,000,000 paired draws (`n_tasks * n_boot`). Exact McNemar evaluation is capped
+at 1,000,000 discordant pairs and accumulates its tail from a bounded-memory
+stream. JSON parsing is strict throughout the document, including metadata, and
+rejects `NaN`/`Infinity` rather than silently accepting non-standard constants.
+
+## Multi-seed
+
+When each task maps to a same-length list of seed repeats, the lists are
+positional: A and B must use the same seed ordering. The JSON report calls each
+position `seed_index`; it does not claim to verify the underlying RNG seed id.
+The kit:
+
+1. averages per task across seeds for the headline delta and bootstrap CI
+2. resamples whole tasks, preserving the task as the independent cluster
+3. omits McNemar rather than treating repeated seeds as independent samples
+4. publishes the per-position deltas plus their mean and sample sd as a
+   descriptive diagnostic only; the sd is not a confidence interval, standard
+   error, or other inferential uncertainty estimate
+
+That is the house answer to single-seed noise (see issue #108 and the
+single-seed warning in `RESULTS.md`).
+
+## RESULTS cell replay
+
+`tests/fixtures/evalkit/results_searchqa_nano_gated.json` replays the published
+SearchQA / GPT-5.4-nano / gated / cumulative nights=5 cell (baseline 0.560,
+after 0.679, Δ +11.9 on n=1400). Per-task pairs were not published, so the
+replay uses a documented maximum-concordance reconstruction: the first
+`round(n * rate)` tasks succeed in each condition. The harness recovers the
+published point delta only. It does not claim to recover the original microdata,
+and p-values or confidence intervals from the reconstructed pairs must not be
+cited as evidence for the published experiment.
+
+## A/A checks
+
+```text
+python -m skillopt_sleep.evalkit --manifest tests/fixtures/evalkit/aa_manifest.json \
+  --a tests/fixtures/evalkit/aa_outcomes.json --aa
+```
+
+The command above is an identity smoke check: identical conditions must report
+delta 0, McNemar p_exact = 1, and a CI that includes 0. The test suite separately
+runs seeded null simulations with genuine discordant pairs, bounds the empirical
+McNemar type-I-error rate on both sides of its nominal level, and checks paired
+bootstrap coverage. These are seeded null calibration checks. Comparing one
+array with itself is not presented as a statistical calibration.
diff --git a/mkdocs.yml b/mkdocs.yml
index be81237d..b5934ad6 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -50,6 +50,7 @@ nav:
     - Deep Learning Analogy: guide/dl-analogy.md
   - SkillOpt-Sleep:
     - Overview: sleep/README.md
+    - Paired A/B Evalkit: sleep/evalkit.md
     - Multi-skill Staging: sleep/multi-skill-staging.md
     - OpenAI-compatible Endpoints: sleep/openai-compatible-endpoints.md
     - Results: sleep/RESULTS.md
diff --git a/skillopt_sleep/__main__.py b/skillopt_sleep/__main__.py
index 6875ad21..e3b7794c 100644
--- a/skillopt_sleep/__main__.py
+++ b/skillopt_sleep/__main__.py
@@ -882,6 +882,19 @@ def main(argv=None) -> int:
     p_unsched = sub.add_parser("unschedule", help="remove the nightly cron entry")
     _add_common(p_unsched)
     p_unsched.add_argument("--all", action="store_true", help="remove all managed entries")
+    p_eval = sub.add_parser(
+        "evalkit",
+        help="paired A/B comparison (McNemar + bootstrap CI)",
+    )
+    p_eval.add_argument("--manifest", required=True)
+    p_eval.add_argument("--a", required=True)
+    p_eval.add_argument("--b", default=None, help="required unless --aa")
+    p_eval.add_argument("--aa", action="store_true", help="mutually exclusive with --b")
+    p_eval.add_argument("--alpha", type=float, default=0.05)
+    p_eval.add_argument("--boot", type=int, default=10000)
+    p_eval.add_argument("--seed", type=int, default=42)
+    p_eval.add_argument("--allow-graded", action="store_true")
+    p_eval.add_argument("--json", action="store_true")
 
     args = parser.parse_args(argv)
     if args.cmd == "run":
@@ -898,6 +911,19 @@ def main(argv=None) -> int:
         return cmd_schedule(args)
     if args.cmd == "unschedule":
         return cmd_unschedule(args)
+    if args.cmd == "evalkit":
+        from skillopt_sleep.evalkit import main as evalkit_main
+        argv = ["--manifest", args.manifest, "--a", args.a]
+        if args.b is not None:
+            argv.extend(["--b", args.b])
+        if args.aa:
+            argv.append("--aa")
+        argv.extend(["--alpha", str(args.alpha), "--boot", str(args.boot), "--seed", str(args.seed)])
+        if args.allow_graded:
+            argv.append("--allow-graded")
+        if args.json:
+            argv.append("--json")
+        return evalkit_main(argv)
     parser.print_help()
     return 2
 
diff --git a/skillopt_sleep/adapters/superpowers.py b/skillopt_sleep/adapters/superpowers.py
index a9ba168f..3ddf6868 100644
--- a/skillopt_sleep/adapters/superpowers.py
+++ b/skillopt_sleep/adapters/superpowers.py
@@ -717,7 +717,7 @@ def _run_scenario(
             if p.is_symlink():
                 raise ValueError(f"Refusing symlinked overlay path: {p}")
         skill_dir.mkdir(parents=True, exist_ok=True)
-        if not skill_dir.resolve().is_relative_to(workspace):
+        if not skill_dir.resolve().is_relative_to(workspace.resolve()):
             raise ValueError(f"Skill path {skill_dir} escapes workspace {workspace}")
         shutil.copy2(skill_overlay, skill_dest, follow_symlinks=False)
 
diff --git a/skillopt_sleep/evalkit.py b/skillopt_sleep/evalkit.py
new file mode 100644
index 00000000..1c3ba313
--- /dev/null
+++ b/skillopt_sleep/evalkit.py
@@ -0,0 +1,718 @@
+"""Paired A/B evaluation kit for SkillOpt-Sleep.
+
+Sleep reports (and many PRs) quote single-run success rates with no
+uncertainty and no guarantee that the two conditions saw the same tasks.
+This module is the shared instrument for those comparisons:
+
+  * one fixed task manifest, paired by task id
+  * McNemar's test on per-task binary outcomes
+  * percentile-bootstrap confidence intervals on the success-rate delta
+  * optional multi-seed repeats with task-cluster inference
+
+It does not change the nightly gate. It standardizes the evidence that
+reports and PRs cite. Pure stdlib; no numpy / scipy.
+
+Refuse comparisons whose task-id sets differ. Graded (non-binary) scores
+are bootstrap-only: McNemar is not defined for them.
+
+CLI::
+
+    python -m skillopt_sleep.evalkit --manifest M.json --a A.json --b B.json
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import random
+import sys
+from dataclasses import asdict, dataclass, field
+from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
+
+# ── errors ────────────────────────────────────────────────────────────────────
+
+class EvalkitError(ValueError):
+    """User-facing contract failure (mismatched ids, empty, etc.)."""
+
+
+MAX_BOOTSTRAPS = 1_000_000
+MAX_BOOTSTRAP_DRAWS = 50_000_000
+MAX_MCNEMAR_DISCORDANTS = 1_000_000
+
+
+def _validate_alpha(alpha: float) -> float:
+    if isinstance(alpha, bool) or not isinstance(alpha, (int, float)):
+        raise EvalkitError("alpha must be a finite number strictly between 0 and 1")
+    try:
+        value = float(alpha)
+    except OverflowError:
+        raise EvalkitError("alpha must be a finite number strictly between 0 and 1") from None
+    if not math.isfinite(value) or not 0.0 < value < 1.0:
+        raise EvalkitError("alpha must be a finite number strictly between 0 and 1")
+    return value
+
+
+def _validate_bootstraps(n_boot: int) -> int:
+    if isinstance(n_boot, bool) or not isinstance(n_boot, int):
+        raise EvalkitError("n_boot must be an integer")
+    if not 1 <= n_boot <= MAX_BOOTSTRAPS:
+        raise EvalkitError(f"n_boot must be between 1 and {MAX_BOOTSTRAPS}")
+    return n_boot
+
+
+def _validate_seed(seed: int) -> int:
+    if isinstance(seed, bool) or not isinstance(seed, int):
+        raise EvalkitError("seed must be an integer")
+    return seed
+
+
+def _validate_count(name: str, value: int) -> int:
+    if isinstance(value, bool) or not isinstance(value, int) or value < 0:
+        raise EvalkitError(f"{name} must be a non-negative integer")
+    return value
+
+
+# ── results ───────────────────────────────────────────────────────────────────
+
+@dataclass
+class McNemarResult:
+    both_success: int
+    a_only: int          # A success, B fail  (c in the usual 2x2)
+    b_only: int          # A fail, B success  (b)
+    both_fail: int
+    n: int
+    chi2: float          # uncorrected (b-c)^2 / (b+c); nan if no discordants
+    p_chi2: float
+    p_exact: float       # two-sided exact binomial on discordants
+    significant: bool
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class BootstrapCI:
+    n_boot: int
+    seed: int
+    alpha: float
+    low: float
+    high: float
+    mean: float
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class EvalReport:
+    n_tasks: int
+    rate_a: float
+    rate_b: float
+    delta: float
+    mcnemar: Optional[McNemarResult]
+    bootstrap: BootstrapCI
+    per_seed: List[Dict[str, float]] = field(default_factory=list)
+    seed_mean_delta: Optional[float] = None
+    seed_sd_delta: Optional[float] = None
+    notes: List[str] = field(default_factory=list)
+    refused: bool = False
+    refuse_reason: str = ""
+
+    def to_dict(self) -> Dict[str, Any]:
+        d = asdict(self)
+        if self.mcnemar is not None:
+            d["mcnemar"] = self.mcnemar.to_dict()
+        d["bootstrap"] = self.bootstrap.to_dict()
+        return d
+
+
+# ── statistics ────────────────────────────────────────────────────────────────
+
+def _chi2_sf_df1(x: float) -> float:
+    """Survival function of chi-square with 1 df: P(X > x) = erfc(sqrt(x/2))."""
+    if x < 0.0 or math.isnan(x):
+        return float("nan")
+    if x == 0.0:
+        return 1.0
+    return math.erfc(math.sqrt(x / 2.0))
+
+
+def _binom_pmf(k: int, n: int, p: float = 0.5) -> float:
+    if k < 0 or k > n:
+        return 0.0
+    # Retained for callers/tests that need one PMF value. Use log-gamma so the
+    # integer binomial coefficient is never coerced to an overflowing float.
+    if p == 0.5:
+        log_pmf = (
+            math.lgamma(n + 1)
+            - math.lgamma(k + 1)
+            - math.lgamma(n - k + 1)
+            - n * math.log(2.0)
+        )
+        return math.exp(log_pmf)
+    if not 0.0 < p < 1.0:
+        raise EvalkitError("binomial p must be strictly between 0 and 1")
+    log_pmf = (
+        math.lgamma(n + 1)
+        - math.lgamma(k + 1)
+        - math.lgamma(n - k + 1)
+        + k * math.log(p)
+        + (n - k) * math.log1p(-p)
+    )
+    return math.exp(log_pmf)
+
+
+def exact_mcnemar_p(b: int, c: int) -> float:
+    """Two-sided exact McNemar p-value (binomial test of discordants, p=0.5)."""
+    b = _validate_count("b", b)
+    c = _validate_count("c", c)
+    n = b + c
+    if n == 0:
+        return 1.0
+    if n > MAX_MCNEMAR_DISCORDANTS:
+        raise EvalkitError(
+            "exact McNemar workload exceeds the "
+            f"{MAX_MCNEMAR_DISCORDANTS} discordant-pair limit"
+        )
+    k = min(b, c)
+    # Start at the largest term in the requested lower tail, then recur
+    # downward. This avoids both the enormous int-to-float conversion in
+    # comb(n, k) * 0.5**n and loss from starting at an underflowed 2**-n.
+    def lower_tail_terms() -> Iterable[float]:
+        # Feed fsum lazily so memory stays bounded independently of k.
+        term = _binom_pmf(k, n, 0.5)
+        yield term
+        for i in range(k, 0, -1):
+            term *= i / (n - i + 1)
+            yield term
+
+    tail = math.fsum(lower_tail_terms())
+    return min(1.0, 2.0 * tail)
+
+
+def mcnemar_from_counts(
+    both_success: int,
+    a_only: int,
+    b_only: int,
+    both_fail: int,
+    *,
+    alpha: float = 0.05,
+) -> McNemarResult:
+    alpha = _validate_alpha(alpha)
+    both_success = _validate_count("both_success", both_success)
+    a_only = _validate_count("a_only", a_only)
+    b_only = _validate_count("b_only", b_only)
+    both_fail = _validate_count("both_fail", both_fail)
+    n = both_success + a_only + b_only + both_fail
+    if n == 0:
+        raise EvalkitError("McNemar requires at least one paired observation")
+    disc = a_only + b_only
+    if disc > MAX_MCNEMAR_DISCORDANTS:
+        raise EvalkitError(
+            "exact McNemar workload exceeds the "
+            f"{MAX_MCNEMAR_DISCORDANTS} discordant-pair limit"
+        )
+    if disc == 0:
+        chi2 = 0.0
+        p_chi2 = 1.0
+    else:
+        chi2 = (b_only - a_only) ** 2 / float(disc)
+        p_chi2 = _chi2_sf_df1(chi2)
+    p_exact = exact_mcnemar_p(b_only, a_only)
+    return McNemarResult(
+        both_success=both_success,
+        a_only=a_only,
+        b_only=b_only,
+        both_fail=both_fail,
+        n=n,
+        chi2=chi2,
+        p_chi2=p_chi2,
+        p_exact=p_exact,
+        significant=p_exact < alpha,
+    )
+
+
+def mcnemar_paired(a: Sequence[int], b: Sequence[int], *, alpha: float = 0.05) -> McNemarResult:
+    alpha = _validate_alpha(alpha)
+    if len(a) != len(b) or not a:
+        raise EvalkitError("McNemar requires a non-empty, equal-length paired sample")
+    bs = ao = bo = bf = 0
+    for x, y in zip(a, b):
+        if _as_binary(x) is None or _as_binary(y) is None:
+            raise EvalkitError("McNemar outcomes must be binary 0/1 values")
+        if x and y:
+            bs += 1
+        elif x and not y:
+            ao += 1
+        elif (not x) and y:
+            bo += 1
+        else:
+            bf += 1
+    return mcnemar_from_counts(bs, ao, bo, bf, alpha=alpha)
+
+
+def bootstrap_delta_ci(
+    a: Sequence[float],
+    b: Sequence[float],
+    *,
+    n_boot: int = 10000,
+    seed: int = 42,
+    alpha: float = 0.05,
+) -> BootstrapCI:
+    alpha = _validate_alpha(alpha)
+    n_boot = _validate_bootstraps(n_boot)
+    seed = _validate_seed(seed)
+    if len(a) != len(b) or not a:
+        raise EvalkitError("bootstrap requires a non-empty paired sample")
+    n = len(a)
+    if n_boot > MAX_BOOTSTRAP_DRAWS // n:
+        raise EvalkitError(
+            "bootstrap workload exceeds the "
+            f"{MAX_BOOTSTRAP_DRAWS} paired-draw limit (n_tasks * n_boot)"
+        )
+    if any(
+        isinstance(x, bool) or not isinstance(x, (int, float))
+        for values in (a, b)
+        for x in values
+    ):
+        raise EvalkitError("bootstrap scores must be JSON numbers, not booleans or strings")
+    try:
+        numeric_a = [float(x) for x in a]
+        numeric_b = [float(x) for x in b]
+    except OverflowError:
+        raise EvalkitError("bootstrap scores must be numeric and finite") from None
+    if any(not math.isfinite(x) for x in numeric_a + numeric_b):
+        raise EvalkitError("bootstrap scores must be numeric and finite")
+    if any(not 0.0 <= x <= 1.0 for x in numeric_a + numeric_b):
+        raise EvalkitError("bootstrap scores must be between 0 and 1")
+    rng = random.Random(seed)
+    paired_deltas = [right - left for left, right in zip(numeric_a, numeric_b)]
+    deltas: List[float] = []
+    for _ in range(n_boot):
+        deltas.append(
+            math.fsum(paired_deltas[rng.randrange(n)] for _ in range(n)) / n
+        )
+    deltas.sort()
+    # Inclusive percentile on the sorted sample.
+    lo_i = int(math.floor((alpha / 2.0) * (n_boot - 1)))
+    hi_i = int(math.ceil((1.0 - alpha / 2.0) * (n_boot - 1)))
+    lo_i = max(0, min(n_boot - 1, lo_i))
+    hi_i = max(0, min(n_boot - 1, hi_i))
+    return BootstrapCI(
+        n_boot=n_boot,
+        seed=seed,
+        alpha=alpha,
+        low=deltas[lo_i],
+        high=deltas[hi_i],
+        mean=sum(deltas) / n_boot,
+    )
+
+
+# ── pairing / loading ─────────────────────────────────────────────────────────
+
+def _as_binary(value: Any) -> Optional[int]:
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
+        return None
+    if value == 1 or value == 1.0:
+        return 1
+    if value == 0 or value == 0.0:
+        return 0
+    return None
+
+
+def _task_id(value: Any, *, context: str) -> str:
+    if not isinstance(value, str) or not value.strip():
+        raise EvalkitError(f"{context} task ids must be non-empty JSON strings")
+    return value
+
+
+def _display_task_id(value: str) -> str:
+    display = value if len(value) <= 80 else value[:77] + "..."
+    return repr(display)
+
+
+def _score(value: Any, *, task_id: str) -> float:
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
+        raise EvalkitError(
+            f"task {_display_task_id(task_id)} scores must be JSON numbers, "
+            "not booleans or strings"
+        )
+    try:
+        score = float(value)
+    except OverflowError:
+        raise EvalkitError(
+            f"task {_display_task_id(task_id)} scores must be finite and between 0 and 1"
+        ) from None
+    if not math.isfinite(score) or not 0.0 <= score <= 1.0:
+        raise EvalkitError(
+            f"task {_display_task_id(task_id)} scores must be finite and between 0 and 1"
+        )
+    return score
+
+
+def _normalize_outcomes(raw: Mapping[str, Any]) -> Dict[str, List[float]]:
+    """Map task id -> positional repeat scores (length 1 if unseeded)."""
+    if not isinstance(raw, Mapping):
+        raise EvalkitError("outcomes must be a JSON object keyed by task id")
+    out: Dict[str, List[float]] = {}
+    for tid, val in raw.items():
+        key = _task_id(tid, context="outcome")
+        if isinstance(val, Mapping):
+            if set(val) != {"seeds"} or not isinstance(val["seeds"], list):
+                raise EvalkitError(
+                    f"task {_display_task_id(key)} must contain only a seeds array"
+                )
+            val = val["seeds"]
+        if isinstance(val, (list, tuple)):
+            values = list(val)
+        else:
+            values = [val]
+        if not values:
+            raise EvalkitError(f"task {_display_task_id(key)} has an empty seed list")
+        out[key] = [_score(item, task_id=key) for item in values]
+    return out
+
+
+def align_pairs(
+    manifest_ids: Sequence[str],
+    outcomes_a: Mapping[str, Any],
+    outcomes_b: Mapping[str, Any],
+) -> Tuple[List[str], List[List[float]], List[List[float]]]:
+    """Align A and B onto the manifest. Refuse any id-set mismatch."""
+    if isinstance(manifest_ids, (str, bytes)) or not isinstance(manifest_ids, Sequence):
+        raise EvalkitError("manifest task ids must be a JSON array")
+    ids = [_task_id(item, context="manifest") for item in manifest_ids]
+    if not ids:
+        raise EvalkitError("manifest is empty")
+    if len(ids) != len(set(ids)):
+        raise EvalkitError("manifest has duplicate task ids")
+    a = _normalize_outcomes(outcomes_a)
+    b = _normalize_outcomes(outcomes_b)
+    a_ids, b_ids = set(a), set(b)
+    want = set(ids)
+    if a_ids != want or b_ids != want:
+        missing_a = sorted(want - a_ids)
+        missing_b = sorted(want - b_ids)
+        extra_a = sorted(a_ids - want)
+        extra_b = sorted(b_ids - want)
+        raise EvalkitError(
+            "outcome task ids must equal the manifest "
+            f"(missing_a={missing_a[:8]}, missing_b={missing_b[:8]}, "
+            f"extra_a={extra_a[:8]}, extra_b={extra_b[:8]})"
+        )
+    n_seed_a = {len(a[i]) for i in ids}
+    n_seed_b = {len(b[i]) for i in ids}
+    if len(n_seed_a) != 1 or n_seed_a != n_seed_b:
+        raise EvalkitError("every task must have the same number of seed repeats in A and B")
+    return ids, [a[i] for i in ids], [b[i] for i in ids]
+
+
+def _is_binary_matrix(rows: Sequence[Sequence[float]]) -> bool:
+    for row in rows:
+        for x in row:
+            if _as_binary(x) is None:
+                return False
+    return True
+
+
+def _mean(xs: Iterable[float]) -> float:
+    seq = list(xs)
+    return sum(seq) / len(seq) if seq else float("nan")
+
+
+def _sd(xs: Sequence[float]) -> float:
+    if len(xs) < 2:
+        return 0.0
+    m = _mean(xs)
+    return math.sqrt(sum((x - m) ** 2 for x in xs) / (len(xs) - 1))
+
+
+def reconstruct_paired_from_rates(n: int, rate_a: float, rate_b: float) -> Tuple[List[int], List[int]]:
+    """Deterministic maximum-concordance reconstruction of paired binaries.
+
+    First ``round(n * rate)`` tasks succeed in each condition, same id order.
+    This is a published-rate replay convention, not original microdata.
+    """
+    if isinstance(n, bool) or not isinstance(n, int) or n < 1:
+        raise EvalkitError("n must be >= 1")
+    if any(
+        isinstance(rate, bool) or not isinstance(rate, (int, float))
+        for rate in (rate_a, rate_b)
+    ):
+        raise EvalkitError("rates must be JSON numbers, finite, and between 0 and 1")
+    try:
+        numeric_a, numeric_b = float(rate_a), float(rate_b)
+    except OverflowError:
+        raise EvalkitError("rates must be finite and between 0 and 1") from None
+    if not all(math.isfinite(rate) and 0.0 <= rate <= 1.0 for rate in (numeric_a, numeric_b)):
+        raise EvalkitError("rates must be finite and between 0 and 1")
+    ka = int(round(n * numeric_a))
+    kb = int(round(n * numeric_b))
+    a = [1 if i < ka else 0 for i in range(n)]
+    b = [1 if i < kb else 0 for i in range(n)]
+    return a, b
+
+
+def compare(
+    manifest_ids: Sequence[str],
+    outcomes_a: Mapping[str, Any],
+    outcomes_b: Mapping[str, Any],
+    *,
+    alpha: float = 0.05,
+    n_boot: int = 10000,
+    seed: int = 42,
+    allow_graded: bool = False,
+) -> EvalReport:
+    alpha = _validate_alpha(alpha)
+    n_boot = _validate_bootstraps(n_boot)
+    if not isinstance(allow_graded, bool):
+        raise EvalkitError("allow_graded must be a boolean")
+    ids, a_rows, b_rows = align_pairs(manifest_ids, outcomes_a, outcomes_b)
+    n_seed = len(a_rows[0])
+    notes: List[str] = []
+
+    # Per-task mean across seeds (the headline paired sample).
+    a_mean = [_mean(row) for row in a_rows]
+    b_mean = [_mean(row) for row in b_rows]
+    rate_a = _mean(a_mean)
+    rate_b = _mean(b_mean)
+    delta = rate_b - rate_a
+    boot = bootstrap_delta_ci(a_mean, b_mean, n_boot=n_boot, seed=seed, alpha=alpha)
+
+    binary = _is_binary_matrix(a_rows) and _is_binary_matrix(b_rows)
+    mcnemar: Optional[McNemarResult] = None
+    if binary and n_seed == 1:
+        # One independent binary observation per task: McNemar's intended unit.
+        mcnemar = mcnemar_paired(
+            [int(_as_binary(row[0]) or 0) for row in a_rows],
+            [int(_as_binary(row[0]) or 0) for row in b_rows],
+            alpha=alpha,
+        )
+    elif binary:
+        # Repeated seeds within a task are clustered measurements, not
+        # independent observations. The task-level bootstrap above is the
+        # inferential result; pooling here would create pseudoreplication.
+        notes.append(
+            "multi-seed binary scores: McNemar omitted; task-cluster bootstrap CI is authoritative"
+        )
+    elif allow_graded:
+        notes.append("graded scores: McNemar omitted; bootstrap CI only")
+    else:
+        raise EvalkitError(
+            "non-binary scores require --allow-graded (McNemar is undefined)"
+        )
+
+    per_seed: List[Dict[str, float]] = []
+    seed_mean = seed_sd = None
+    if n_seed > 1:
+        for s in range(n_seed):
+            da = _mean(row[s] for row in a_rows)
+            db = _mean(row[s] for row in b_rows)
+            per_seed.append(
+                {"seed_index": s, "rate_a": da, "rate_b": db, "delta": db - da}
+            )
+        deltas = [row["delta"] for row in per_seed]
+        seed_mean = _mean(deltas)
+        seed_sd = _sd(deltas)
+        notes.append(
+            f"multi-seed positional repeats: {n_seed}; mean delta={seed_mean:.6f} "
+            f"sample sd={seed_sd:.6f}; descriptive only, not an uncertainty estimate"
+        )
+        notes.append(
+            "seed lists are positional repeats; seed_index is not a verified RNG seed id"
+        )
+
+    return EvalReport(
+        n_tasks=len(ids),
+        rate_a=rate_a,
+        rate_b=rate_b,
+        delta=delta,
+        mcnemar=mcnemar,
+        bootstrap=boot,
+        per_seed=per_seed,
+        seed_mean_delta=seed_mean,
+        seed_sd_delta=seed_sd,
+        notes=notes,
+    )
+
+
+def compare_aa(
+    manifest_ids: Sequence[str],
+    outcomes: Mapping[str, Any],
+    **kwargs: Any,
+) -> EvalReport:
+    """A/A identity smoke check: identical conditions must not reject."""
+    report = compare(manifest_ids, outcomes, outcomes, **kwargs)
+    report.notes.append("A/A identity smoke check (identical conditions)")
+    return report
+
+
+# ── I/O ───────────────────────────────────────────────────────────────────────
+
+def _reject_duplicate_object_keys(pairs: Sequence[Tuple[str, Any]]) -> Dict[str, Any]:
+    obj: Dict[str, Any] = {}
+    for key, value in pairs:
+        if key in obj:
+            display_key = key if len(key) <= 80 else key[:77] + "..."
+            raise EvalkitError(f"duplicate JSON object key: {display_key!r}")
+        obj[key] = value
+    return obj
+
+
+def _reject_json_constant(value: str) -> Any:
+    raise EvalkitError(f"non-standard JSON constant {value} is not allowed")
+
+
+def _load_json(path: str, *, label: str) -> Any:
+    try:
+        with open(path, encoding="utf-8") as f:
+            return json.load(
+                f,
+                object_pairs_hook=_reject_duplicate_object_keys,
+                parse_constant=_reject_json_constant,
+            )
+    except EvalkitError as exc:
+        raise EvalkitError(f"invalid JSON in {label}: {exc}") from None
+    except (json.JSONDecodeError, UnicodeError, ValueError, RecursionError) as exc:
+        if isinstance(exc, json.JSONDecodeError):
+            detail = exc.msg
+        elif isinstance(exc, UnicodeError):
+            detail = "invalid text encoding"
+        elif isinstance(exc, RecursionError):
+            detail = "nesting is too deep"
+        else:
+            detail = "invalid numeric literal"
+        raise EvalkitError(f"invalid JSON in {label}: {detail}") from None
+    except OSError as exc:
+        detail = exc.strerror or type(exc).__name__
+        raise EvalkitError(f"could not read {label}: {detail}") from None
+
+
+def _manifest_ids(obj: Any) -> List[str]:
+    if isinstance(obj, list):
+        return [_task_id(item, context="manifest") for item in obj]
+    if isinstance(obj, Mapping):
+        forms = [name for name in ("ids", "tasks", "outcomes") if name in obj]
+        if len(forms) > 1:
+            raise EvalkitError("manifest must use exactly one of ids, tasks, or outcomes")
+        if "ids" in obj:
+            values = obj["ids"]
+            if not isinstance(values, list):
+                raise EvalkitError("manifest ids must be a JSON array")
+            return [_task_id(item, context="manifest") for item in values]
+        if "tasks" in obj:
+            tasks = obj["tasks"]
+            if not isinstance(tasks, list):
+                raise EvalkitError("manifest tasks must be a JSON array")
+            ids: List[str] = []
+            for task in tasks:
+                if not isinstance(task, Mapping) or "id" not in task:
+                    raise EvalkitError("every manifest task must be an object containing id")
+                ids.append(_task_id(task["id"], context="manifest"))
+            return ids
+        if "outcomes" in obj:
+            outcomes = obj["outcomes"]
+            if not isinstance(outcomes, Mapping):
+                raise EvalkitError("manifest outcomes must be a JSON object")
+            return [_task_id(key, context="manifest") for key in outcomes]
+    raise EvalkitError("manifest must be a list of ids or an object with ids/tasks")
+
+
+def _outcomes(obj: Any, manifest_ids: Sequence[str]) -> Dict[str, Any]:
+    if not isinstance(obj, Mapping):
+        raise EvalkitError("outcomes file must be an object mapping task id to score")
+    # A direct mapping wins when its keys exactly match the manifest. This makes
+    # a task literally named ``outcomes`` unambiguous even when its score is a
+    # seeded object. Otherwise the one-key wrapper is recognized.
+    if set(obj) == set(manifest_ids):
+        return dict(obj)
+    if set(obj) == {"outcomes"} and isinstance(obj["outcomes"], Mapping):
+        values = obj["outcomes"]
+        return dict(values)
+    return dict(obj)
+
+
+def format_markdown(report: EvalReport) -> str:
+    lines = [
+        "# Paired A/B evalkit report",
+        "",
+        f"- n_tasks: {report.n_tasks}",
+        f"- rate_a: {report.rate_a:.6f}",
+        f"- rate_b: {report.rate_b:.6f}",
+        f"- delta (B-A): {report.delta:+.6f}",
+        (
+            f"- bootstrap {100 * (1 - report.bootstrap.alpha):g}% CI: "
+            f"[{report.bootstrap.low:+.6f}, {report.bootstrap.high:+.6f}] "
+            f"(n_boot={report.bootstrap.n_boot}, seed={report.bootstrap.seed})"
+        ),
+    ]
+    if report.mcnemar is not None:
+        m = report.mcnemar
+        lines.append(
+            f"- McNemar 2x2: both+={m.both_success} a_only={m.a_only} "
+            f"b_only={m.b_only} both-={m.both_fail}"
+        )
+        lines.append(
+            f"- McNemar chi2={m.chi2:.4f} p_chi2={m.p_chi2:.6g} "
+            f"p_exact={m.p_exact:.6g} significant={m.significant}"
+        )
+    if report.seed_mean_delta is not None:
+        lines.append(
+            f"- positional-repeat mean delta: {report.seed_mean_delta:+.6f} "
+            f"(descriptive sample sd {report.seed_sd_delta:.6f}, "
+            f"k={len(report.per_seed)}; not an uncertainty estimate)"
+        )
+    for note in report.notes:
+        lines.append(f"- note: {note}")
+    return "\n".join(lines) + "\n"
+
+
+def main(argv: Optional[Sequence[str]] = None) -> int:
+    p = argparse.ArgumentParser(
+        prog="skillopt_sleep.evalkit",
+        description="Paired A/B comparison with McNemar and bootstrap CIs",
+    )
+    p.add_argument("--manifest", required=True, help="JSON list of task ids (or {ids,tasks})")
+    p.add_argument("--a", required=True, help="JSON outcomes for condition A")
+    p.add_argument("--b", default=None, help="JSON outcomes for condition B (required unless --aa)")
+    p.add_argument("--aa", action="store_true", help="A/A identity smoke check (mutually exclusive with --b)")
+    p.add_argument("--alpha", type=float, default=0.05)
+    p.add_argument("--boot", type=int, default=10000)
+    p.add_argument("--seed", type=int, default=42)
+    p.add_argument("--allow-graded", action="store_true")
+    p.add_argument("--json", action="store_true")
+    args = p.parse_args(list(argv) if argv is not None else None)
+
+    try:
+        if (args.b is not None) == args.aa:
+            raise EvalkitError("exactly one of --b or --aa is required")
+        ids = _manifest_ids(_load_json(args.manifest, label="--manifest"))
+        a = _outcomes(_load_json(args.a, label="--a"), ids)
+        if args.aa:
+            report = compare_aa(
+                ids, a, alpha=args.alpha, n_boot=args.boot,
+                seed=args.seed, allow_graded=args.allow_graded,
+            )
+        else:
+            b = _outcomes(_load_json(args.b, label="--b"), ids)
+            report = compare(
+                ids, a, b, alpha=args.alpha, n_boot=args.boot,
+                seed=args.seed, allow_graded=args.allow_graded,
+            )
+    except EvalkitError as exc:
+        print(f"ERR_EVALKIT {exc}", file=sys.stderr)
+        return 2
+
+    if args.json:
+        try:
+            print(json.dumps(report.to_dict(), indent=2, sort_keys=True, allow_nan=False))
+        except (TypeError, ValueError) as exc:
+            print(f"ERR_EVALKIT report is not strict JSON: {exc}", file=sys.stderr)
+            return 2
+    else:
+        print(format_markdown(report), end="")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/skillopt_sleep/staging.py b/skillopt_sleep/staging.py
index e2ecfbd5..615c5ac8 100644
--- a/skillopt_sleep/staging.py
+++ b/skillopt_sleep/staging.py
@@ -439,6 +439,7 @@ def _write_atomic_bytes(
     *,
     create_parents: bool = True,
     mode: Optional[int] = None,
+    replace_permission_retries: int = 0,
 ) -> None:
     """Write raw bytes atomically, optionally restoring an exact file mode."""
     directory = os.path.dirname(path) or "."
@@ -458,7 +459,18 @@ def _write_atomic_bytes(
                 else:
                     os.chmod(tmp, existing_mode)
             os.fsync(f.fileno())
-        os.replace(tmp, path)
+        # A caller may explicitly tolerate a bounded transient Windows sharing
+        # violation. Live files, WALs, receipts, backups, and rollback always
+        # use the zero-retry default so a concurrent editor cannot be overwritten
+        # after the caller's compare-and-swap validation.
+        for attempt in range(replace_permission_retries + 1):
+            try:
+                os.replace(tmp, path)
+                break
+            except PermissionError:
+                if os.name != "nt" or attempt == replace_permission_retries:
+                    raise
+                time.sleep(0.005 * (attempt + 1))
         _fsync_parent(path)
     except BaseException:
         try:
@@ -535,7 +547,10 @@ def _remove_private_temp_aliases(path: str) -> None:
         if not entry.name.startswith(".tmp-new-") or entry.path == path:
             continue
         try:
-            candidate = entry.stat(follow_symlinks=False)
+            # ``DirEntry.stat()`` reports zeroed file IDs/link counts on the
+            # Windows GitHub runner.  A path-based lstat returns the actual
+            # NTFS identity and keeps the hard-link comparison meaningful.
+            candidate = os.lstat(entry.path)
             if (
                 stat.S_ISREG(candidate.st_mode)
                 and (candidate.st_dev, candidate.st_ino) == file_id
@@ -546,6 +561,18 @@ def _remove_private_temp_aliases(path: str) -> None:
             continue
     if removed:
         _fsync_directory(directory)
+        if os.name == "nt":
+            # NTFS can report the pre-unlink link count briefly after the
+            # directory entry is gone. Wait only for metadata convergence;
+            # any surviving hard link still leaves nlink > 1 and the caller
+            # will continue to fail closed.
+            for attempt in range(20):
+                try:
+                    if os.lstat(path).st_nlink <= 1:
+                        break
+                except OSError:
+                    break
+                time.sleep(0.005 * (attempt + 1))
 
 
 def _artifact_snapshot(path: str) -> Optional[tuple[bytes, int]]:
@@ -915,7 +942,14 @@ def _publish_latest(root: str, out: str) -> None:
             or info.st_nlink != 1
         ):
             raise StagingError(f"latest-staging pointer is unsafe: {pointer}")
-    _write_atomic_bytes(pointer, f"{name}\n".encode("utf-8"), mode=0o600)
+    # Concurrent staging publishers may briefly retain the replace destination
+    # on Windows. Retrying is safe only for this derived, last-writer-wins pointer.
+    _write_atomic_bytes(
+        pointer,
+        f"{name}\n".encode("utf-8"),
+        mode=0o600,
+        replace_permission_retries=20,
+    )
 
 
 def _staging_order(path: str) -> tuple:
@@ -1902,14 +1936,32 @@ def _existing_path_is_canonical_staging_descendant(
     The staging root itself may be supplied through a symlink, so compare the
     resolved candidate with the same relative path beneath the resolved root.
     """
-    try:
-        relative = os.path.relpath(path, staging_dir)
-    except ValueError:
-        return False
-    if relative == os.pardir or relative.startswith(os.pardir + os.sep):
-        return False
-    expected_real = os.path.join(os.path.realpath(staging_dir), relative)
-    return _path_identity_key(os.path.realpath(path)) == _path_identity_key(expected_real)
+    candidate = os.path.abspath(path)
+    root = os.path.abspath(staging_dir)
+    # Compare actual directory identities while walking upward. This tolerates
+    # equivalent root spellings (/var vs /private/var and Windows 8.3 vs long
+    # names) without resolving away a symlink/junction *below* the root. The
+    # supplied root itself may be an alias, so test its identity before applying
+    # the descendant-link refusal.
+    current = candidate
+    while True:
+        try:
+            if os.path.samefile(current, root):
+                if (
+                    _path_identity_key(current) != _path_identity_key(root)
+                    and _path_is_within(current, root)
+                    and _is_link_or_junction(current)
+                ):
+                    return False
+                return current != candidate
+        except OSError:
+            return False
+        if _is_link_or_junction(current):
+            return False
+        parent = os.path.dirname(current)
+        if parent == current:
+            return False
+        current = parent
 
 
 def _immutable_backup_snapshot(
diff --git a/tests/fixtures/evalkit/aa_manifest.json b/tests/fixtures/evalkit/aa_manifest.json
new file mode 100644
index 00000000..05448f5b
--- /dev/null
+++ b/tests/fixtures/evalkit/aa_manifest.json
@@ -0,0 +1,8 @@
+{
+  "ids": [
+    "t00", "t01", "t02", "t03", "t04", "t05", "t06", "t07", "t08", "t09",
+    "t10", "t11", "t12", "t13", "t14", "t15", "t16", "t17", "t18", "t19",
+    "t20", "t21", "t22", "t23", "t24", "t25", "t26", "t27", "t28", "t29",
+    "t30", "t31", "t32", "t33", "t34", "t35", "t36", "t37", "t38", "t39"
+  ]
+}
diff --git a/tests/fixtures/evalkit/aa_outcomes.json b/tests/fixtures/evalkit/aa_outcomes.json
new file mode 100644
index 00000000..444ad43f
--- /dev/null
+++ b/tests/fixtures/evalkit/aa_outcomes.json
@@ -0,0 +1,9 @@
+{
+  "outcomes": {
+    "t00": 1, "t01": 1, "t02": 1, "t03": 1, "t04": 1, "t05": 1, "t06": 1, "t07": 1,
+    "t08": 1, "t09": 1, "t10": 1, "t11": 1, "t12": 1, "t13": 1, "t14": 1, "t15": 1,
+    "t16": 1, "t17": 1, "t18": 1, "t19": 1, "t20": 0, "t21": 0, "t22": 0, "t23": 0,
+    "t24": 0, "t25": 0, "t26": 0, "t27": 0, "t28": 0, "t29": 0, "t30": 0, "t31": 0,
+    "t32": 0, "t33": 0, "t34": 0, "t35": 0, "t36": 0, "t37": 0, "t38": 0, "t39": 0
+  }
+}
diff --git a/tests/fixtures/evalkit/mcnemar_textbook.json b/tests/fixtures/evalkit/mcnemar_textbook.json
new file mode 100644
index 00000000..f6019ccb
--- /dev/null
+++ b/tests/fixtures/evalkit/mcnemar_textbook.json
@@ -0,0 +1,10 @@
+{
+  "name": "textbook-2x2",
+  "both_success": 40,
+  "a_only": 2,
+  "b_only": 12,
+  "both_fail": 46,
+  "chi2": 7.142857142857143,
+  "p_chi2": 0.007526315166457887,
+  "p_exact": 0.012939453125
+}
diff --git a/tests/fixtures/evalkit/results_searchqa_nano_gated.json b/tests/fixtures/evalkit/results_searchqa_nano_gated.json
new file mode 100644
index 00000000..3122c7fe
--- /dev/null
+++ b/tests/fixtures/evalkit/results_searchqa_nano_gated.json
@@ -0,0 +1,10 @@
+{
+  "cell_id": "results-searchqa-nano-gated-cumulative-nights5",
+  "source": "docs/sleep/RESULTS.md section 2",
+  "n": 1400,
+  "baseline": 0.560,
+  "after": 0.679,
+  "published_delta": 0.119,
+  "reconstruction": "maximum-concordance: first round(n*rate) tasks succeed in each condition",
+  "inference": "unsupported: reconstructed pairs recover the point delta only"
+}
diff --git a/tests/test_devin_plugin.py b/tests/test_devin_plugin.py
index cc2fe637..0d449f7a 100644
--- a/tests/test_devin_plugin.py
+++ b/tests/test_devin_plugin.py
@@ -364,6 +364,7 @@ def test_env_tilde_is_expanded(self):
             importlib.reload(mcp_server)
 
 
+@unittest.skipIf(os.name == "nt", "Devin installer and hook are POSIX shell scripts")
 class TestDevinInstaller(unittest.TestCase):
     def _run_installer(self, project, home, installer=INSTALLER):
         env = os.environ.copy()
diff --git a/tests/test_evalkit.py b/tests/test_evalkit.py
new file mode 100644
index 00000000..556925e9
--- /dev/null
+++ b/tests/test_evalkit.py
@@ -0,0 +1,686 @@
+"""Paired A/B evalkit: known-answer stats, seeded null calibration, RESULTS replay."""
+from __future__ import annotations
+
+import io
+import json
+import math
+import os
+import random
+import subprocess
+import sys
+import tempfile
+import unittest
+from contextlib import redirect_stderr, redirect_stdout
+from unittest.mock import patch
+
+from skillopt_sleep.evalkit import (
+    MAX_BOOTSTRAP_DRAWS,
+    MAX_MCNEMAR_DISCORDANTS,
+    EvalkitError,
+    bootstrap_delta_ci,
+    compare,
+    compare_aa,
+    exact_mcnemar_p,
+    format_markdown,
+    mcnemar_from_counts,
+    mcnemar_paired,
+    reconstruct_paired_from_rates,
+)
+from skillopt_sleep.evalkit import (
+    main as evalkit_main,
+)
+
+FIXTURE_DIR = os.path.join(os.path.dirname(__file__), "fixtures", "evalkit")
+
+
+def _load(name: str):
+    with open(os.path.join(FIXTURE_DIR, name), encoding="utf-8") as f:
+        return json.load(f)
+
+
+class TestMcNemarKnownAnswer(unittest.TestCase):
+    def test_textbook_2x2_chi2_and_exact(self):
+        fx = _load("mcnemar_textbook.json")
+        res = mcnemar_from_counts(
+            fx["both_success"], fx["a_only"], fx["b_only"], fx["both_fail"],
+        )
+        self.assertAlmostEqual(res.chi2, fx["chi2"], places=12)
+        self.assertAlmostEqual(res.p_chi2, fx["p_chi2"], places=12)
+        self.assertAlmostEqual(res.p_exact, fx["p_exact"], places=12)
+        self.assertTrue(res.significant)
+        self.assertEqual(res.n, 100)
+
+    def test_zero_discordants_is_not_significant(self):
+        res = mcnemar_from_counts(20, 0, 0, 5)
+        self.assertEqual(res.chi2, 0.0)
+        self.assertEqual(res.p_chi2, 1.0)
+        self.assertEqual(res.p_exact, 1.0)
+        self.assertFalse(res.significant)
+
+    def test_paired_vectors_match_counts(self):
+        a = [1, 1, 1, 0, 0]
+        b = [1, 0, 1, 1, 0]
+        res = mcnemar_paired(a, b)
+        self.assertEqual(res.both_success, 2)
+        self.assertEqual(res.a_only, 1)
+        self.assertEqual(res.b_only, 1)
+        self.assertEqual(res.both_fail, 1)
+        self.assertAlmostEqual(res.p_exact, exact_mcnemar_p(1, 1))
+
+    def test_exact_tail_is_stable_above_one_thousand_discordants(self):
+        self.assertEqual(exact_mcnemar_p(700, 700), 1.0)
+        value = exact_mcnemar_p(590, 611)
+        # Independent reference from scipy.stats.binomtest(590, 1201, 0.5).
+        self.assertAlmostEqual(value, 0.563883319454372, places=10)
+        self.assertEqual(value, exact_mcnemar_p(611, 590))
+
+    def test_exact_tail_has_a_documented_resource_limit(self):
+        with self.assertRaisesRegex(EvalkitError, "discordant-pair limit"):
+            exact_mcnemar_p(MAX_MCNEMAR_DISCORDANTS + 1, 0)
+        with self.assertRaisesRegex(EvalkitError, "discordant-pair limit"):
+            mcnemar_from_counts(0, MAX_MCNEMAR_DISCORDANTS, 1, 0)
+
+    def test_exact_tail_accumulates_from_a_lazy_stream(self):
+        real_fsum = math.fsum
+
+        def consume(values):
+            self.assertNotIsInstance(values, (list, tuple))
+            return real_fsum(values)
+
+        with patch("skillopt_sleep.evalkit.math.fsum", side_effect=consume):
+            self.assertAlmostEqual(exact_mcnemar_p(590, 611), 0.563883319454372)
+
+    def test_invalid_counts_and_binary_vectors_are_refused(self):
+        for args in ((-1, 0), (True, 0), (1.5, 0)):
+            with self.subTest(args=args), self.assertRaises(EvalkitError):
+                exact_mcnemar_p(*args)
+        with self.assertRaises(EvalkitError):
+            mcnemar_from_counts(1, -1, 0, 1)
+        with self.assertRaises(EvalkitError):
+            mcnemar_paired([], [])
+        with self.assertRaises(EvalkitError):
+            mcnemar_paired([0, 2], [0, 1])
+
+
+class TestBootstrapCoverage(unittest.TestCase):
+    def test_identical_series_ci_collapses_to_zero(self):
+        a = [1, 0, 1, 0, 1, 0, 1, 0]
+        ci = bootstrap_delta_ci(a, a, n_boot=2000, seed=7)
+        self.assertEqual(ci.low, 0.0)
+        self.assertEqual(ci.high, 0.0)
+        self.assertEqual(ci.mean, 0.0)
+
+    def test_known_shift_ci_excludes_zero(self):
+        # A always 0, B always 1: delta = 1 exactly, CI is [1, 1].
+        a = [0] * 30
+        b = [1] * 30
+        ci = bootstrap_delta_ci(a, b, n_boot=1000, seed=1)
+        self.assertEqual(ci.low, 1.0)
+        self.assertEqual(ci.high, 1.0)
+
+    def test_seed_is_deterministic(self):
+        a = [1, 0, 1, 1, 0, 0, 1, 0, 1, 0]
+        b = [1, 1, 1, 0, 0, 1, 1, 0, 0, 1]
+        x = bootstrap_delta_ci(a, b, n_boot=500, seed=99)
+        y = bootstrap_delta_ci(a, b, n_boot=500, seed=99)
+        self.assertEqual((x.low, x.high, x.mean), (y.low, y.high, y.mean))
+
+    def test_invalid_alpha_and_bootstrap_counts_are_refused(self):
+        for alpha in (0, 1, -0.1, 1.1, float("nan"), float("inf"), True, "0.05"):
+            with self.subTest(alpha=alpha), self.assertRaises(EvalkitError):
+                bootstrap_delta_ci([0], [1], alpha=alpha)
+        for n_boot in (0, -1, 1.5, True, 1_000_001):
+            with self.subTest(n_boot=n_boot), self.assertRaises(EvalkitError):
+                bootstrap_delta_ci([0], [1], n_boot=n_boot)
+        for seed in (True, 1.5, "7"):
+            with self.subTest(seed=seed), self.assertRaises(EvalkitError):
+                bootstrap_delta_ci([0], [1], n_boot=10, seed=seed)
+
+    def test_total_bootstrap_draws_are_bounded(self):
+        n_tasks = 100
+        excessive_bootstraps = MAX_BOOTSTRAP_DRAWS // n_tasks + 1
+        with self.assertRaisesRegex(EvalkitError, "paired-draw limit"):
+            bootstrap_delta_ci(
+                [0] * n_tasks,
+                [1] * n_tasks,
+                n_boot=excessive_bootstraps,
+            )
+
+    def test_malformed_direct_api_scores_are_contract_errors(self):
+        for value in (
+            "not-a-number", "1", True, None, object(), 10**1000,
+            float("nan"), float("inf"),
+        ):
+            with self.subTest(value=value), self.assertRaises(EvalkitError):
+                bootstrap_delta_ci([0], [value], n_boot=10)
+
+    def test_direct_api_scores_must_be_in_the_unit_interval(self):
+        for value in (-1, -0.00001, 1.00001, 2):
+            with self.subTest(value=value), self.assertRaisesRegex(
+                EvalkitError, "between 0 and 1"
+            ):
+                bootstrap_delta_ci([0], [value], n_boot=10)
+
+
+class TestAACalibration(unittest.TestCase):
+    def test_aa_does_not_reject(self):
+        man = _load("aa_manifest.json")
+        out = _load("aa_outcomes.json")
+        report = compare_aa(man["ids"], out["outcomes"], n_boot=2000, seed=42)
+        self.assertEqual(report.delta, 0.0)
+        self.assertIsNotNone(report.mcnemar)
+        self.assertFalse(report.mcnemar.significant)
+        self.assertEqual(report.mcnemar.p_exact, 1.0)
+        self.assertLessEqual(report.bootstrap.low, 0.0)
+        self.assertGreaterEqual(report.bootstrap.high, 0.0)
+
+    def test_exact_test_controls_type_one_error_under_a_seeded_null(self):
+        # Unlike comparing an array with itself, this exercises non-zero,
+        # symmetrically distributed discordance and can catch p-value inflation.
+        rng = random.Random(20260824)
+        trials = 500
+        rejected = 0
+        for _ in range(trials):
+            a = []
+            b = []
+            for _task in range(80):
+                left = int(rng.random() < 0.5)
+                right = 1 - left if rng.random() < 0.30 else left
+                a.append(left)
+                b.append(right)
+            rejected += int(mcnemar_paired(a, b, alpha=0.05).significant)
+        rate = rejected / trials
+        self.assertGreaterEqual(rate, 0.025)
+        self.assertLessEqual(rate, 0.075)
+
+    def test_paired_bootstrap_has_nominal_coverage_under_a_seeded_null(self):
+        rng = random.Random(20260825)
+        trials = 160
+        covered = 0
+        for trial in range(trials):
+            a = []
+            b = []
+            for _task in range(80):
+                left = int(rng.random() < 0.5)
+                right = 1 - left if rng.random() < 0.30 else left
+                a.append(left)
+                b.append(right)
+            ci = bootstrap_delta_ci(a, b, n_boot=300, seed=10_000 + trial)
+            covered += int(ci.low <= 0.0 <= ci.high)
+        coverage = covered / trials
+        self.assertGreaterEqual(coverage, 0.90)
+        self.assertLessEqual(coverage, 0.99)
+
+
+class TestCompareContracts(unittest.TestCase):
+    def test_mismatched_ids_are_refused(self):
+        with self.assertRaises(EvalkitError) as ctx:
+            compare(["t1", "t2"], {"t1": 1, "t2": 0}, {"t1": 1, "t3": 0})
+        self.assertIn("must equal the manifest", str(ctx.exception))
+
+    def test_duplicate_manifest_ids_refused(self):
+        with self.assertRaises(EvalkitError):
+            compare(["t1", "t1"], {"t1": 1}, {"t1": 0})
+
+    def test_empty_count_table_refused(self):
+        with self.assertRaisesRegex(EvalkitError, "at least one paired observation"):
+            mcnemar_from_counts(0, 0, 0, 0)
+
+    def test_empty_manifest_refused(self):
+        with self.assertRaises(EvalkitError):
+            compare([], {}, {})
+
+    def test_task_ids_are_nonempty_strings_without_coercion(self):
+        for task_id in (1, {}, [], None, True, "", "   "):
+            with self.subTest(task_id=task_id), self.assertRaisesRegex(
+                EvalkitError, "non-empty JSON strings"
+            ):
+                compare([task_id], {}, {})
+        with self.assertRaisesRegex(EvalkitError, "non-empty JSON strings"):
+            compare(["t1"], {"": 0}, {"t1": 1})
+
+    def test_scores_are_json_numbers_without_coercion(self):
+        for score in ("0", "1.0", True, False, None):
+            with self.subTest(score=score), self.assertRaisesRegex(
+                EvalkitError, "JSON numbers"
+            ):
+                compare(["t1"], {"t1": score}, {"t1": 1})
+        with self.assertRaisesRegex(EvalkitError, "finite and between 0 and 1"):
+            compare(["t1"], {"t1": 10**1000}, {"t1": 1})
+        with self.assertRaisesRegex(EvalkitError, "contain only a seeds array"):
+            compare(
+                ["t1"],
+                {"t1": {"seeds": [0, 1], "ignored": 1}},
+                {"t1": [0, 1]},
+            )
+
+    def test_graded_refused_without_flag(self):
+        with self.assertRaises(EvalkitError) as ctx:
+            compare(["t1", "t2"], {"t1": 0.4, "t2": 0.9}, {"t1": 0.5, "t2": 0.8})
+        self.assertIn("allow-graded", str(ctx.exception))
+
+    def test_graded_bootstrap_only(self):
+        report = compare(
+            ["t1", "t2"],
+            {"t1": 0.4, "t2": 0.9},
+            {"t1": 0.5, "t2": 0.8},
+            allow_graded=True,
+            n_boot=500,
+            seed=3,
+        )
+        self.assertIsNone(report.mcnemar)
+        self.assertTrue(any("graded" in n for n in report.notes))
+        self.assertAlmostEqual(report.delta, 0.0, places=12)
+
+    def test_multi_seed_variance_band(self):
+        report = compare(
+            ["t1", "t2"],
+            {"t1": [1, 0, 1], "t2": [0, 0, 1]},
+            {"t1": [1, 1, 1], "t2": [1, 0, 1]},
+            n_boot=400,
+            seed=2,
+        )
+        self.assertEqual(len(report.per_seed), 3)
+        self.assertEqual(
+            [row["seed_index"] for row in report.per_seed],
+            [0, 1, 2],
+        )
+        self.assertTrue(any("positional repeats" in note for note in report.notes))
+        self.assertIsNotNone(report.seed_mean_delta)
+        self.assertGreaterEqual(report.seed_sd_delta, 0.0)
+        self.assertAlmostEqual(report.rate_a, (2 / 3 + 1 / 3) / 2)
+        self.assertAlmostEqual(report.rate_b, (1.0 + 2 / 3) / 2)
+        self.assertIsNone(report.mcnemar)
+        self.assertTrue(any("task-cluster" in note for note in report.notes))
+
+    def test_empty_nonfinite_and_out_of_range_seed_scores_are_refused(self):
+        bad_values = ([], [float("nan")], [float("inf")], [-0.01], [1.01])
+        for value in bad_values:
+            with self.subTest(value=value), self.assertRaises(EvalkitError):
+                compare(["t1"], {"t1": value}, {"t1": [1]}, allow_graded=True)
+
+    def test_duplicating_seeds_within_tasks_does_not_inflate_inference(self):
+        ids = ["t1", "t2", "t3", "t4"]
+        a_two = {tid: [0, 0] for tid in ids}
+        b_two = {tid: [1, 1] for tid in ids}
+        a_many = {tid: [0] * 100 for tid in ids}
+        b_many = {tid: [1] * 100 for tid in ids}
+        two = compare(ids, a_two, b_two, n_boot=500, seed=8)
+        many = compare(ids, a_many, b_many, n_boot=500, seed=8)
+        self.assertIsNone(two.mcnemar)
+        self.assertIsNone(many.mcnemar)
+        self.assertEqual(two.delta, many.delta)
+        self.assertEqual(two.bootstrap.to_dict(), many.bootstrap.to_dict())
+
+    def test_heterogeneous_seed_duplication_does_not_change_cluster_inference(self):
+        ids = ["t1", "t2", "t3", "t4"]
+        a = {
+            "t1": [0, 1],
+            "t2": [1, 0],
+            "t3": [0, 0],
+            "t4": [1, 1],
+        }
+        b = {
+            "t1": [1, 1],
+            "t2": [0, 0],
+            "t3": [0, 1],
+            "t4": [1, 0],
+        }
+        duplicated_a = {task_id: values * 50 for task_id, values in a.items()}
+        duplicated_b = {task_id: values * 50 for task_id, values in b.items()}
+        original = compare(ids, a, b, n_boot=500, seed=8)
+        duplicated = compare(ids, duplicated_a, duplicated_b, n_boot=500, seed=8)
+        self.assertEqual(original.delta, duplicated.delta)
+        self.assertEqual(original.bootstrap.to_dict(), duplicated.bootstrap.to_dict())
+        self.assertIsNone(original.mcnemar)
+        self.assertIsNone(duplicated.mcnemar)
+
+    def test_repeats_cannot_be_inflated_for_only_one_task(self):
+        with self.assertRaisesRegex(
+            EvalkitError,
+            "every task must have the same number of seed repeats",
+        ):
+            compare(
+                ["t1", "t2"],
+                {"t1": [0, 1] * 50, "t2": [0, 1]},
+                {"t1": [1, 1] * 50, "t2": [1, 0]},
+                n_boot=20,
+            )
+
+    def test_positional_repeat_sd_is_explicitly_noninferential(self):
+        report = compare(
+            ["t1", "t2"],
+            {"t1": [0, 1], "t2": [1, 1]},
+            {"t1": [1, 1], "t2": [0, 1]},
+            n_boot=20,
+        )
+        self.assertTrue(any("not an uncertainty estimate" in note for note in report.notes))
+        self.assertIn("descriptive sample sd", format_markdown(report))
+
+    def test_invalid_compare_parameters_are_refused(self):
+        for alpha in (0, 1, float("nan"), "0.05", 10**1000):
+            with self.subTest(alpha=alpha), self.assertRaises(EvalkitError):
+                compare(["t1"], {"t1": 0}, {"t1": 1}, alpha=alpha)
+        for n_boot in (0, -4, True):
+            with self.subTest(n_boot=n_boot), self.assertRaises(EvalkitError):
+                compare(["t1"], {"t1": 0}, {"t1": 1}, n_boot=n_boot)
+
+
+class TestResultsCellReplay(unittest.TestCase):
+    def test_malformed_reconstruction_inputs_are_contract_errors(self):
+        for rates in (
+            ("bad", 0.5), ("0.5", 0.5), (None, 0.5),
+            (10**1000, 0.5), (float("nan"), 0.5), (0.5, 1.1),
+        ):
+            with self.subTest(rates=rates), self.assertRaises(EvalkitError):
+                reconstruct_paired_from_rates(10, *rates)
+        for n in (True, 0, 1.5):
+            with self.subTest(n=n), self.assertRaises(EvalkitError):
+                reconstruct_paired_from_rates(n, 0.5, 0.5)
+
+    def test_published_searchqa_nano_gated_point_delta(self):
+        cell = _load("results_searchqa_nano_gated.json")
+        a, b = reconstruct_paired_from_rates(cell["n"], cell["baseline"], cell["after"])
+        self.assertEqual(len(a), cell["n"])
+        rate_a = math.fsum(a) / cell["n"]
+        rate_b = math.fsum(b) / cell["n"]
+        self.assertAlmostEqual(rate_a, cell["baseline"], places=3)
+        self.assertAlmostEqual(rate_b, cell["after"], places=3)
+        self.assertAlmostEqual(rate_b - rate_a, cell["published_delta"], places=3)
+
+
+class TestCLI(unittest.TestCase):
+    def test_aa_cli_exit_zero(self):
+        rc = evalkit_main([
+            "--manifest", os.path.join(FIXTURE_DIR, "aa_manifest.json"),
+            "--a", os.path.join(FIXTURE_DIR, "aa_outcomes.json"),
+            "--aa",
+            "--boot", "300",
+            "--json",
+        ])
+        self.assertEqual(rc, 0)
+
+    def test_mismatch_cli_exit_two(self):
+        with tempfile.TemporaryDirectory() as td:
+            man = os.path.join(td, "m.json")
+            a = os.path.join(td, "a.json")
+            b = os.path.join(td, "b.json")
+            with open(man, "w", encoding="utf-8") as f:
+                json.dump(["t1", "t2"], f)
+            with open(a, "w", encoding="utf-8") as f:
+                json.dump({"t1": 1, "t2": 0}, f)
+            with open(b, "w", encoding="utf-8") as f:
+                json.dump({"t1": 1, "t3": 0}, f)
+            rc = evalkit_main(["--manifest", man, "--a", a, "--b", b])
+            self.assertEqual(rc, 2)
+
+    def test_exactly_one_of_b_or_aa_is_required(self):
+        manifest = os.path.join(FIXTURE_DIR, "aa_manifest.json")
+        outcomes = os.path.join(FIXTURE_DIR, "aa_outcomes.json")
+        for extra in ([], ["--b", outcomes, "--aa"], ["--b", "", "--aa"]):
+            with self.subTest(extra=extra):
+                stdout = io.StringIO()
+                stderr = io.StringIO()
+                with redirect_stdout(stdout), redirect_stderr(stderr):
+                    rc = evalkit_main([
+                        "--manifest", manifest,
+                        "--a", outcomes,
+                        "--json",
+                        *extra,
+                    ])
+                self.assertEqual(rc, 2)
+                self.assertEqual(stdout.getvalue(), "")
+                self.assertIn("exactly one of --b or --aa", stderr.getvalue())
+
+    def test_umbrella_cli_enforces_b_or_aa_exclusivity(self):
+        manifest = os.path.join(FIXTURE_DIR, "aa_manifest.json")
+        outcomes = os.path.join(FIXTURE_DIR, "aa_outcomes.json")
+        base = [
+            sys.executable,
+            "-m",
+            "skillopt_sleep",
+            "evalkit",
+            "--manifest",
+            manifest,
+            "--a",
+            outcomes,
+            "--json",
+        ]
+        for extra in ([], ["--b", outcomes, "--aa"], ["--b", "", "--aa"]):
+            with self.subTest(extra=extra):
+                proc = subprocess.run(
+                    [*base, *extra],
+                    capture_output=True,
+                    text=True,
+                    check=False,
+                )
+                self.assertEqual(proc.returncode, 2)
+                self.assertEqual(proc.stdout, "")
+                self.assertIn("exactly one of --b or --aa", proc.stderr)
+
+    def test_umbrella_cli_accepts_explicit_aa(self):
+        proc = subprocess.run(
+            [
+                sys.executable,
+                "-m",
+                "skillopt_sleep",
+                "evalkit",
+                "--manifest",
+                os.path.join(FIXTURE_DIR, "aa_manifest.json"),
+                "--a",
+                os.path.join(FIXTURE_DIR, "aa_outcomes.json"),
+                "--aa",
+                "--boot",
+                "20",
+                "--json",
+            ],
+            capture_output=True,
+            text=True,
+            check=False,
+        )
+        self.assertEqual(proc.returncode, 0, proc.stderr)
+        self.assertEqual(json.loads(proc.stdout)["delta"], 0.0)
+
+    def test_duplicate_json_object_keys_are_refused(self):
+        with tempfile.TemporaryDirectory() as td:
+            man = os.path.join(td, "m.json")
+            a = os.path.join(td, "a.json")
+            b = os.path.join(td, "b.json")
+            with open(man, "w", encoding="utf-8") as handle:
+                handle.write('["t1"]')
+            with open(a, "w", encoding="utf-8") as handle:
+                handle.write('{"t1": 0, "t1": 1}')
+            with open(b, "w", encoding="utf-8") as handle:
+                handle.write('{"t1": 1}')
+            stdout = io.StringIO()
+            stderr = io.StringIO()
+            with redirect_stdout(stdout), redirect_stderr(stderr):
+                rc = evalkit_main([
+                    "--manifest", man,
+                    "--a", a,
+                    "--b", b,
+                    "--json",
+                ])
+            self.assertEqual(rc, 2)
+            self.assertEqual(stdout.getvalue(), "")
+            self.assertIn("duplicate JSON object key: 't1'", stderr.getvalue())
+
+    def test_task_named_outcomes_is_not_mistaken_for_wrapper(self):
+        with tempfile.TemporaryDirectory() as td:
+            paths = []
+            for name, content in (
+                ("m.json", '["outcomes"]'),
+                ("a.json", '{"outcomes": 0}'),
+                ("b.json", '{"outcomes": 1}'),
+            ):
+                path = os.path.join(td, name)
+                with open(path, "w", encoding="utf-8") as handle:
+                    handle.write(content)
+                paths.append(path)
+            stdout = io.StringIO()
+            stderr = io.StringIO()
+            with redirect_stdout(stdout), redirect_stderr(stderr):
+                rc = evalkit_main([
+                    "--manifest", paths[0],
+                    "--a", paths[1],
+                    "--b", paths[2],
+                    "--boot", "20",
+                    "--json",
+                ])
+            self.assertEqual(rc, 0, stderr.getvalue())
+            self.assertEqual(json.loads(stdout.getvalue())["delta"], 1.0)
+
+    def test_task_named_outcomes_accepts_seeded_object_values(self):
+        with tempfile.TemporaryDirectory() as td:
+            paths = []
+            for name, content in (
+                ("m.json", '["outcomes"]'),
+                ("a.json", '{"outcomes": {"seeds": [0, 1]}}'),
+                ("b.json", '{"outcomes": {"seeds": [1, 1]}}'),
+            ):
+                path = os.path.join(td, name)
+                with open(path, "w", encoding="utf-8") as handle:
+                    handle.write(content)
+                paths.append(path)
+            stdout = io.StringIO()
+            stderr = io.StringIO()
+            with redirect_stdout(stdout), redirect_stderr(stderr):
+                rc = evalkit_main([
+                    "--manifest", paths[0], "--a", paths[1], "--b", paths[2],
+                    "--boot", "20", "--json",
+                ])
+            self.assertEqual(rc, 0, stderr.getvalue())
+            self.assertEqual(json.loads(stdout.getvalue())["delta"], 0.5)
+
+    def test_malformed_json_and_shapes_are_clean_contract_errors(self):
+        cases = (
+            ("{", '{"t1": 1}', '{"t1": 1}'),
+            ('{"ids": "t1"}', '{"t1": 1}', '{"t1": 1}'),
+            ('["t1"]', '{"outcomes": []}', '{"t1": 1}'),
+        )
+        for manifest_text, a_text, b_text in cases:
+            with self.subTest(manifest=manifest_text), tempfile.TemporaryDirectory() as td:
+                paths = []
+                for name, content in (("m.json", manifest_text), ("a.json", a_text), ("b.json", b_text)):
+                    path = os.path.join(td, name)
+                    with open(path, "w", encoding="utf-8") as handle:
+                        handle.write(content)
+                    paths.append(path)
+                stderr = io.StringIO()
+                with redirect_stderr(stderr):
+                    rc = evalkit_main(["--manifest", paths[0], "--a", paths[1], "--b", paths[2]])
+                self.assertEqual(rc, 2)
+                self.assertTrue(stderr.getvalue().startswith("ERR_EVALKIT "))
+                self.assertNotIn("Traceback", stderr.getvalue())
+
+    def test_module_entrypoint(self):
+        proc = subprocess.run(
+            [
+                sys.executable, "-m", "skillopt_sleep.evalkit",
+                "--manifest", os.path.join(FIXTURE_DIR, "aa_manifest.json"),
+                "--a", os.path.join(FIXTURE_DIR, "aa_outcomes.json"),
+                "--aa",
+                "--boot", "200",
+            ],
+            capture_output=True,
+            text=True,
+            check=False,
+        )
+        self.assertEqual(proc.returncode, 0, proc.stderr)
+        self.assertIn("delta (B-A): +0.000000", proc.stdout)
+
+    def test_nonfinite_input_is_refused_without_nonstandard_json_output(self):
+        with tempfile.TemporaryDirectory() as td:
+            man = os.path.join(td, "m.json")
+            a = os.path.join(td, "a.json")
+            b = os.path.join(td, "b.json")
+            with open(man, "w", encoding="utf-8") as f:
+                json.dump(["t1"], f)
+            with open(a, "w", encoding="utf-8") as f:
+                f.write('{"t1": NaN}')
+            with open(b, "w", encoding="utf-8") as f:
+                json.dump({"t1": 1}, f)
+            stdout = io.StringIO()
+            stderr = io.StringIO()
+            with redirect_stdout(stdout), redirect_stderr(stderr):
+                rc = evalkit_main([
+                    "--manifest", man, "--a", a, "--b", b, "--json",
+                ])
+            self.assertEqual(rc, 2)
+            self.assertEqual(stdout.getvalue(), "")
+            self.assertIn("non-standard JSON constant", stderr.getvalue())
+            self.assertNotIn("NaN", stdout.getvalue())
+
+    def test_nonstandard_json_constants_are_rejected_even_in_ignored_metadata(self):
+        for constant in ("NaN", "Infinity", "-Infinity"):
+            with self.subTest(constant=constant), tempfile.TemporaryDirectory() as td:
+                man = os.path.join(td, "m.json")
+                a = os.path.join(td, "a.json")
+                b = os.path.join(td, "b.json")
+                with open(man, "w", encoding="utf-8") as handle:
+                    handle.write('{"ids": ["t1"], "ignored": ' + constant + "}")
+                with open(a, "w", encoding="utf-8") as handle:
+                    handle.write('{"t1": 0}')
+                with open(b, "w", encoding="utf-8") as handle:
+                    handle.write('{"t1": 1}')
+                stdout = io.StringIO()
+                stderr = io.StringIO()
+                with redirect_stdout(stdout), redirect_stderr(stderr):
+                    rc = evalkit_main([
+                        "--manifest", man, "--a", a, "--b", b, "--json",
+                    ])
+                self.assertEqual(rc, 2)
+                self.assertEqual(stdout.getvalue(), "")
+                self.assertIn("non-standard JSON constant", stderr.getvalue())
+
+    def test_malformed_manifest_ids_are_rejected_by_the_cli(self):
+        invalid_ids = (1, {}, [], None, True, "", "   ")
+        for task_id in invalid_ids:
+            with self.subTest(task_id=task_id), tempfile.TemporaryDirectory() as td:
+                man = os.path.join(td, "m.json")
+                a = os.path.join(td, "a.json")
+                with open(man, "w", encoding="utf-8") as handle:
+                    json.dump({"tasks": [{"id": task_id}]}, handle)
+                with open(a, "w", encoding="utf-8") as handle:
+                    json.dump({"t1": 0}, handle)
+                stderr = io.StringIO()
+                with redirect_stderr(stderr):
+                    rc = evalkit_main(["--manifest", man, "--a", a, "--aa"])
+                self.assertEqual(rc, 2)
+                self.assertIn("non-empty JSON strings", stderr.getvalue())
+
+    def test_conflicting_manifest_forms_are_rejected(self):
+        with tempfile.TemporaryDirectory() as td:
+            man = os.path.join(td, "m.json")
+            a = os.path.join(td, "a.json")
+            with open(man, "w", encoding="utf-8") as handle:
+                json.dump({"ids": ["t1"], "tasks": [{"id": "t1"}]}, handle)
+            with open(a, "w", encoding="utf-8") as handle:
+                json.dump({"t1": 0}, handle)
+            stderr = io.StringIO()
+            with redirect_stderr(stderr):
+                rc = evalkit_main(["--manifest", man, "--a", a, "--aa"])
+            self.assertEqual(rc, 2)
+            self.assertIn("exactly one of ids, tasks, or outcomes", stderr.getvalue())
+
+    def test_input_errors_do_not_disclose_paths(self):
+        with tempfile.TemporaryDirectory() as td:
+            secret_name = "secret-customer-path.json"
+            path = os.path.join(td, secret_name)
+            stderr = io.StringIO()
+            with redirect_stderr(stderr):
+                rc = evalkit_main(["--manifest", path, "--a", path, "--aa"])
+            self.assertEqual(rc, 2)
+            self.assertNotIn(td, stderr.getvalue())
+            self.assertNotIn(secret_name, stderr.getvalue())
+            self.assertIn("--manifest", stderr.getvalue())
+
+    def test_confidence_label_is_not_truncated_by_float_roundoff(self):
+        report = compare_aa(["t1"], {"t1": 1}, alpha=0.34, n_boot=20)
+        rendered = format_markdown(report)
+        self.assertIn("bootstrap 66% CI", rendered)
+        self.assertNotIn("bootstrap 65% CI", rendered)
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/tests/test_sleep_adopt_skill_subset.py b/tests/test_sleep_adopt_skill_subset.py
index 9c26c588..f6c32af5 100644
--- a/tests/test_sleep_adopt_skill_subset.py
+++ b/tests/test_sleep_adopt_skill_subset.py
@@ -33,14 +33,22 @@ def _sha(text):
     return hashlib.sha256(text.encode("utf-8")).hexdigest()
 
 
+def _canonical(path):
+    return os.path.realpath(os.path.abspath(path))
+
+
+def _same_path(left, right):
+    return os.path.normcase(_canonical(left)) == os.path.normcase(_canonical(right))
+
+
 def _read(path):
-    with open(path, encoding="utf-8") as f:
+    with open(path, encoding="utf-8", newline="") as f:
         return f.read()
 
 
 def _write(path, text):
     os.makedirs(os.path.dirname(path), exist_ok=True)
-    with open(path, "w", encoding="utf-8") as f:
+    with open(path, "w", encoding="utf-8", newline="") as f:
         f.write(text)
 
 
@@ -55,7 +63,9 @@ def __init__(
         beta_body="# beta v1\n",
     ):
         self.tmp = tmp
-        self.live_root = os.path.join(tmp, "live")
+        # Keep the project's lexical spelling so status/latest receipts remain
+        # user-facing, but pin live targets to one canonical filesystem identity.
+        self.live_root = os.path.join(_canonical(tmp), "live")
         self.alpha_live = os.path.join(self.live_root, "alpha", "SKILL.md")
         self.beta_live = os.path.join(self.live_root, "beta", "SKILL.md")
         for path, body in (
@@ -90,6 +100,7 @@ class TestAdoptionIsConfinedToTheStagedRoots(unittest.TestCase):
     """
 
     def _retarget(self, staging, skill_name, new_live):
+        new_live = _canonical(new_live)
         manifest_path = os.path.join(staging, "manifest.json")
         with open(manifest_path, encoding="utf-8") as handle:
             manifest = json.load(handle)
@@ -324,6 +335,8 @@ def test_manifest_proposal_filename_cannot_escape_staging(self):
             self.assertEqual(_read(night.alpha_live), "# alpha v1\n")
 
     def test_adoption_preserves_existing_live_file_mode(self):
+        if os.name == "nt":
+            self.skipTest("Windows does not provide POSIX file-mode semantics")
         with tempfile.TemporaryDirectory() as tmp:
             night = TwoSkillNight(tmp)
             os.chmod(night.alpha_live, 0o640)
@@ -338,7 +351,7 @@ def test_a_failed_write_rolls_the_whole_selection_back(self):
             real_write = staging_mod._write_atomic
 
             def boom(path, text, *, create_parents=True):
-                if path == night.beta_live:
+                if _same_path(path, night.beta_live):
                     raise OSError("disk full")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -359,7 +372,7 @@ def test_post_commit_live_write_error_rolls_the_whole_selection_back(self):
 
             def commit_then_fail(path, text, *, create_parents=True):
                 result = real_write(path, text, create_parents=create_parents)
-                if path == night.beta_live:
+                if _same_path(path, night.beta_live):
                     raise OSError("late close failure")
                 return result
 
@@ -385,7 +398,7 @@ def test_rollback_removes_files_that_did_not_exist_before(self):
             real_write = staging_mod._write_atomic
 
             def boom(path, text, *, create_parents=True):
-                if path == night.beta_live:
+                if _same_path(path, night.beta_live):
                     raise OSError("disk full")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -543,6 +556,8 @@ def test_repeated_noop_adoption_cannot_rewrite_receipt_or_backup(self):
             self.assertEqual(_read(backup_path), backup_before)
 
     def test_rollback_restores_original_mode_as_well_as_bytes(self):
+        if os.name == "nt":
+            self.skipTest("Windows does not provide POSIX file-mode semantics")
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
@@ -551,7 +566,7 @@ def test_rollback_restores_original_mode_as_well_as_bytes(self):
             real_write = staging_mod._write_atomic
 
             def boom(path, text, *, create_parents=True):
-                if path == night.beta_live:
+                if _same_path(path, night.beta_live):
                     raise OSError("disk full")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -574,7 +589,7 @@ def test_backup_failure_rolls_back_prior_live_writes(self):
             )
 
             def boom(path, data, *, mode=None):
-                if path == beta_backup:
+                if _same_path(path, beta_backup):
                     raise OSError("backup device full")
                 return real_write_new(path, data, mode=mode)
 
@@ -1561,7 +1576,7 @@ def test_concurrent_adoption_cleanly_refuses_one_writer(self):
             real_write = staging_mod._write_atomic
 
             def pause_first_live_write(path, text, *, create_parents=True):
-                if path == night.alpha_live and not entered.is_set():
+                if _same_path(path, night.alpha_live) and not entered.is_set():
                     entered.set()
                     if not release.wait(5):
                         raise RuntimeError("test timed out waiting for release")
@@ -1593,7 +1608,7 @@ def test_separate_nights_share_the_same_live_target_lock(self):
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
-            live = os.path.join(tmp, "live", "alpha", "SKILL.md")
+            live = os.path.join(_canonical(tmp), "live", "alpha", "SKILL.md")
             _write(live, "# alpha v1\n")
 
             def stage(proposal):
@@ -1616,7 +1631,7 @@ def stage(proposal):
             real_write = staging_mod._write_atomic
 
             def pause_first_live_write(path, text, *, create_parents=True):
-                if path == live and not entered.is_set():
+                if _same_path(path, live) and not entered.is_set():
                     entered.set()
                     if not release.wait(5):
                         raise RuntimeError("test timed out waiting for release")
@@ -1712,8 +1727,9 @@ def test_cycle_skips_empty_proposed_skill_with_a_note(self):
 
 class TestDurableAdoptionTransaction(unittest.TestCase):
     def _legacy_night(self, tmp):
-        skill = os.path.join(tmp, "live", "skill", "SKILL.md")
-        memory = os.path.join(tmp, "live", "CLAUDE.md")
+        live_root = os.path.join(_canonical(tmp), "live")
+        skill = os.path.join(live_root, "skill", "SKILL.md")
+        memory = os.path.join(live_root, "CLAUDE.md")
         _write(skill, "# skill v1\n")
         _write(memory, "# memory v1\n")
         staging = write_staging(
@@ -1737,7 +1753,7 @@ def test_wal_is_durable_before_first_backup_and_removed_at_commit(self):
             real_write_new = staging_mod._write_new_bytes
 
             def observe_backup(path, data, *, mode=None):
-                if path == wal_path:
+                if _same_path(path, wal_path):
                     return real_write_new(path, data, mode=mode)
                 with open(wal_path, encoding="utf-8") as handle:
                     wal = json.load(handle)
@@ -1762,7 +1778,7 @@ def test_interrupted_transaction_is_recovered_before_retry(self):
 
             def commit_then_fail(path, text, *, create_parents=True):
                 result = real_write(path, text, create_parents=create_parents)
-                if path == night.alpha_live:
+                if _same_path(path, night.alpha_live):
                     raise OSError("simulated process interruption")
                 return result
 
@@ -1798,7 +1814,7 @@ def test_interrupted_transaction_recovers_before_corrupt_manifest_read(self):
 
             def commit_then_fail(path, text, *, create_parents=True):
                 result = real_write(path, text, create_parents=create_parents)
-                if path == night.alpha_live:
+                if _same_path(path, night.alpha_live):
                     raise OSError("simulated interruption")
                 return result
 
@@ -1827,7 +1843,7 @@ def test_interrupted_relative_staging_recovers_via_absolute_path(self):
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
-            live = os.path.join(tmp, "live", "alpha", "SKILL.md")
+            live = os.path.join(_canonical(tmp), "live", "alpha", "SKILL.md")
             _write(live, "# alpha v1\n")
             previous_cwd = os.getcwd()
             try:
@@ -1849,7 +1865,7 @@ def test_interrupted_relative_staging_recovers_via_absolute_path(self):
 
                 def commit_then_fail(path, text, *, create_parents=True):
                     result = real_write(path, text, create_parents=create_parents)
-                    if path == live:
+                    if _same_path(path, live):
                         raise OSError("simulated interruption")
                     return result
 
@@ -1881,7 +1897,7 @@ def test_restart_cleans_own_hardlink_publication_temp(self):
 
             def commit_then_fail(path, text, *, create_parents=True):
                 result = real_write(path, text, create_parents=create_parents)
-                if path == night.alpha_live:
+                if _same_path(path, night.alpha_live):
                     raise OSError("simulated interruption")
                 return result
 
@@ -1917,7 +1933,7 @@ def test_rollback_preserves_concurrent_human_edit_and_retains_wal(self):
             real_write = staging_mod._write_atomic
 
             def fail_beta_after_human_edit(path, text, *, create_parents=True):
-                if path == night.beta_live:
+                if _same_path(path, night.beta_live):
                     _write(night.alpha_live, "# concurrent human edit\n")
                     raise OSError("beta disk failure")
                 return real_write(path, text, create_parents=create_parents)
@@ -1947,7 +1963,7 @@ def test_edit_during_receipt_publication_never_commits_a_false_receipt(self):
             real_write = staging_mod._write_atomic
 
             def edit_live_before_receipt(path, text, *, create_parents=True):
-                if path == receipt_path:
+                if _same_path(path, receipt_path):
                     _write(night.alpha_live, "# concurrent human edit\n")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -2060,7 +2076,7 @@ def test_legacy_manifest_is_pinned_and_adoption_has_a_receipt(self):
 
     def test_legacy_missing_targets_can_share_one_new_parent(self):
         with tempfile.TemporaryDirectory() as tmp:
-            live_root = os.path.join(tmp, "new-live")
+            live_root = os.path.join(_canonical(tmp), "new-live")
             skill = os.path.join(live_root, "SKILL.md")
             memory = os.path.join(live_root, "CLAUDE.md")
             staging = write_staging(
@@ -2080,7 +2096,7 @@ def test_failed_legacy_adoption_removes_its_exact_new_directory_tree(self):
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
-            live_root = os.path.join(tmp, "new", "nested", "live")
+            live_root = os.path.join(_canonical(tmp), "new", "nested", "live")
             skill = os.path.join(live_root, "SKILL.md")
             memory = os.path.join(live_root, "CLAUDE.md")
             staging = write_staging(
@@ -2096,7 +2112,7 @@ def test_failed_legacy_adoption_removes_its_exact_new_directory_tree(self):
             real_write = staging_mod._write_atomic
 
             def fail_receipt(path, text, *, create_parents=True):
-                if path == receipt:
+                if _same_path(path, receipt):
                     raise OSError("receipt device full")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -2113,7 +2129,7 @@ def test_recovery_never_removes_a_replaced_created_directory(self):
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
-            live_root = os.path.join(tmp, "new-live")
+            live_root = os.path.join(_canonical(tmp), "new-live")
             skill = os.path.join(live_root, "SKILL.md")
             memory = os.path.join(live_root, "CLAUDE.md")
             staging = write_staging(
@@ -2130,7 +2146,7 @@ def test_recovery_never_removes_a_replaced_created_directory(self):
             real_write = staging_mod._write_atomic
 
             def fail_receipt(path, text, *, create_parents=True):
-                if path == receipt:
+                if _same_path(path, receipt):
                     raise OSError("receipt device full")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -2158,7 +2174,7 @@ def test_restart_recovery_removes_journaled_created_directories(self):
         from skillopt_sleep import staging as staging_mod
 
         with tempfile.TemporaryDirectory() as tmp:
-            live_root = os.path.join(tmp, "restart", "live")
+            live_root = os.path.join(_canonical(tmp), "restart", "live")
             skill = os.path.join(live_root, "SKILL.md")
             memory = os.path.join(live_root, "CLAUDE.md")
             staging = write_staging(
@@ -2174,7 +2190,7 @@ def test_restart_recovery_removes_journaled_created_directories(self):
             real_write = staging_mod._write_atomic
 
             def fail_receipt(path, text, *, create_parents=True):
-                if path == receipt:
+                if _same_path(path, receipt):
                     raise OSError("simulated interruption")
                 return real_write(path, text, create_parents=create_parents)
 
@@ -2231,7 +2247,7 @@ def test_legacy_second_target_failure_rolls_back_first(self):
             real_write = staging_mod._write_atomic
 
             def fail_memory(path, text, *, create_parents=True):
-                if path == memory:
+                if _same_path(path, memory):
                     raise OSError("memory disk failure")
                 return real_write(path, text, create_parents=create_parents)
 
diff --git a/tests/test_sleep_engine.py b/tests/test_sleep_engine.py
index ba985ea5..3f975361 100644
--- a/tests/test_sleep_engine.py
+++ b/tests/test_sleep_engine.py
@@ -1375,7 +1375,9 @@ def test_cycle_stage_then_adopt_with_backup(self):
 
     def test_cycle_can_target_repo_scoped_skill_path(self):
         with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
-            target = os.path.abspath(os.path.join(proj, ".agents/skills/taste-skill/SKILL.md"))
+            target = os.path.realpath(
+                os.path.abspath(os.path.join(proj, ".agents/skills/taste-skill/SKILL.md"))
+            )
             cfg = load_config(
                 invoked_project=proj,
                 projects="invoked",
diff --git a/tests/test_sleep_scheduler_safety.py b/tests/test_sleep_scheduler_safety.py
index f830ef90..992b19cd 100644
--- a/tests/test_sleep_scheduler_safety.py
+++ b/tests/test_sleep_scheduler_safety.py
@@ -11,6 +11,7 @@
 
 
 class TestSleepSchedulerSafety(unittest.TestCase):
+    @unittest.skipIf(os.name == "nt", "POSIX runner quoting is not used on Windows")
     def test_posix_runner_quotes_every_path_and_argument(self):
         with tempfile.TemporaryDirectory(prefix="sleep $' quote ") as project:
             command = scheduler._runner_cmd(
diff --git a/tests/test_sleep_skill_resolver.py b/tests/test_sleep_skill_resolver.py
index e52beabe..423c50d8 100644
--- a/tests/test_sleep_skill_resolver.py
+++ b/tests/test_sleep_skill_resolver.py
@@ -29,6 +29,10 @@ def _write_skill(root, name, body="# skill\n"):
     return path
 
 
+def _canonical(path):
+    return os.path.realpath(os.path.abspath(path))
+
+
 def _symlink(test, source, link_name):
     """Create a symlink, or skip the test where the platform refuses one.
 
@@ -178,7 +182,9 @@ def test_user_skills_root_comes_first_then_plugin_cache(self):
             )
             os.makedirs(plugin_skills)
             cfg = load_config(claude_home=claude_home)
-            self.assertEqual(skill_search_roots(cfg), [skills, plugin_skills])
+            self.assertEqual(
+                skill_search_roots(cfg), [_canonical(skills), _canonical(plugin_skills)]
+            )
 
     def test_absent_roots_are_skipped(self):
         with tempfile.TemporaryDirectory() as tmp:
@@ -249,7 +255,7 @@ def test_unreadable_plugin_cache_does_not_break_discovery(self):
                 self.skipTest("directory is still readable after chmod 000")
             try:
                 cfg = load_config(claude_home=claude_home)
-                self.assertEqual(skill_search_roots(cfg), [skills])
+                self.assertEqual(skill_search_roots(cfg), [_canonical(skills)])
             finally:
                 try:
                     os.chmod(cache, 0o700)
@@ -279,7 +285,9 @@ def test_versioned_marketplace_layout_is_discovered(self):
             )
             cfg = load_config(claude_home=claude_home)
             self.assertIn(
-                os.path.join(cache, "claude-plugins-official", "superpowers", "5.0.7", "skills"),
+                _canonical(os.path.join(
+                    cache, "claude-plugins-official", "superpowers", "5.0.7", "skills"
+                )),
                 skill_search_roots(cfg),
             )
             res = resolve_skill("brainstorming", skill_search_roots(cfg))
@@ -292,11 +300,13 @@ def test_multiple_installed_versions_resolve_to_the_newest_not_ambiguous(self):
         # peer root would make an ordinary upgrade resolve AMBIGUOUS.
         with tempfile.TemporaryDirectory() as tmp:
             claude_home = os.path.join(tmp, ".claude")
-            plugin = os.path.join(claude_home, "plugins", "cache",
-                                  "claude-plugins-official", "chrome-devtools-mcp")
+            plugin = _canonical(os.path.join(
+                claude_home, "plugins", "cache",
+                "claude-plugins-official", "chrome-devtools-mcp"
+            ))
             for version in ["1.1.1", "1.5.0", "1.6.0"]:
                 _write_skill(os.path.join(plugin, version, "skills"), "chrome-devtools")
-            newest = os.path.join(plugin, "1.6.0", "skills")
+            newest = _canonical(os.path.join(plugin, "1.6.0", "skills"))
 
             cfg = load_config(claude_home=claude_home)
             roots = skill_search_roots(cfg)
@@ -310,13 +320,15 @@ def test_multiple_installed_versions_resolve_to_the_newest_not_ambiguous(self):
     def test_version_ordering_is_numeric_not_lexicographic(self):
         with tempfile.TemporaryDirectory() as tmp:
             claude_home = os.path.join(tmp, ".claude")
-            plugin = os.path.join(claude_home, "plugins", "cache", "market", "plugin")
+            plugin = _canonical(os.path.join(
+                claude_home, "plugins", "cache", "market", "plugin"
+            ))
             for version in ["1.9.0", "1.10.0"]:
                 _write_skill(os.path.join(plugin, version, "skills"), "example-skill")
             cfg = load_config(claude_home=claude_home)
             roots = [r for r in skill_search_roots(cfg) if r.startswith(plugin)]
             # "1.10.0" < "1.9.0" as strings; it must still win as a version.
-            self.assertEqual(roots, [os.path.join(plugin, "1.10.0", "skills")])
+            self.assertEqual(roots, [_canonical(os.path.join(plugin, "1.10.0", "skills"))])
 
     def test_stable_release_beats_an_installed_prerelease(self):
         # Segment lists alone would rank 2.0.0-beta above 2.0.0, because a
@@ -324,12 +336,14 @@ def test_stable_release_beats_an_installed_prerelease(self):
         # must never be preferred over the stable release it precedes.
         with tempfile.TemporaryDirectory() as tmp:
             claude_home = os.path.join(tmp, ".claude")
-            plugin = os.path.join(claude_home, "plugins", "cache", "market", "plugin")
+            plugin = _canonical(os.path.join(
+                claude_home, "plugins", "cache", "market", "plugin"
+            ))
             for version in ["2.0.0", "2.0.0-beta"]:
                 _write_skill(os.path.join(plugin, version, "skills"), "example-skill")
             cfg = load_config(claude_home=claude_home)
             roots = [r for r in skill_search_roots(cfg) if r.startswith(plugin)]
-            self.assertEqual(roots, [os.path.join(plugin, "2.0.0", "skills")])
+            self.assertEqual(roots, [_canonical(os.path.join(plugin, "2.0.0", "skills"))])
 
     def test_version_key_orders_release_forms_sensibly(self):
         from skillopt_sleep.skill_resolver import _version_sort_key as key
@@ -346,7 +360,7 @@ def test_legacy_unversioned_layout_still_works(self):
             plugin = os.path.join(claude_home, "plugins", "cache", "market", "plugin")
             expected = _write_skill(os.path.join(plugin, "skills"), "example-skill")
             cfg = load_config(claude_home=claude_home)
-            self.assertIn(os.path.join(plugin, "skills"), skill_search_roots(cfg))
+            self.assertIn(_canonical(os.path.join(plugin, "skills")), skill_search_roots(cfg))
             res = resolve_skill("example-skill", skill_search_roots(cfg))
             self.assertEqual(res.status, FOUND)
             self.assertEqual(res.path, os.path.realpath(expected))
@@ -361,8 +375,8 @@ def test_two_marketplaces_each_contribute_a_root(self):
             _write_skill(cognee, "cognee-remember")
             cfg = load_config(claude_home=claude_home)
             roots = skill_search_roots(cfg)
-            self.assertIn(official, roots)
-            self.assertIn(cognee, roots)
+            self.assertIn(_canonical(official), roots)
+            self.assertIn(_canonical(cognee), roots)
 
     def test_legacy_target_skill_path_behavior_is_untouched(self):
         with tempfile.TemporaryDirectory() as tmp:
diff --git a/tests/test_sleep_staging_fanout.py b/tests/test_sleep_staging_fanout.py
index f7ad4b76..32121ac4 100644
--- a/tests/test_sleep_staging_fanout.py
+++ b/tests/test_sleep_staging_fanout.py
@@ -32,6 +32,10 @@ def _proposal(name="example-skill", body="# example\n", live=None, root="/tmp/li
     return SkillProposal(name, body, live)
 
 
+def _canonical(path):
+    return os.path.realpath(os.path.abspath(os.path.normpath(path)))
+
+
 def _report():
     return SleepReport(night=1, project="/repo/example", accepted=True,
                        gate_action="accept_new_best")
@@ -44,7 +48,7 @@ def test_one_row_per_skill_in_order(self):
         self.assertEqual([r["proposed_file"] for r in rows],
                          ["proposed_SKILL.alpha.md", "proposed_SKILL.beta.md"])
         self.assertEqual(rows[0]["live_skill_path"],
-                         os.path.normpath("/tmp/live/alpha/SKILL.md"))
+                         _canonical("/tmp/live/alpha/SKILL.md"))
         self.assertEqual(
             rows[0]["sha256"],
             hashlib.sha256(b"# example\n").hexdigest(),
@@ -126,14 +130,14 @@ def test_absolute_paths_needing_normalisation_are_accepted(self):
         # duplicate separators everywhere, and every forward-slash absolute
         # path on Windows. Normalising first keeps the traversal guard.
         rows = skill_proposal_rows([_proposal("alpha", live="/tmp/live//alpha/SKILL.md")])
-        self.assertEqual(rows[0]["live_skill_path"], os.path.normpath("/tmp/live/alpha/SKILL.md"))
+        self.assertEqual(rows[0]["live_skill_path"], _canonical("/tmp/live/alpha/SKILL.md"))
 
     def test_current_directory_segments_are_normalised_not_refused(self):
         rows = skill_proposal_rows([
             _proposal("alpha", live="/tmp/live/./alpha/SKILL.md")
         ])
         self.assertEqual(rows[0]["live_skill_path"],
-                         os.path.normpath("/tmp/live/alpha/SKILL.md"))
+                         _canonical("/tmp/live/alpha/SKILL.md"))
 
     def test_two_skills_targeting_one_file_are_refused(self):
         shared = "/tmp/live/shared/SKILL.md"
@@ -297,7 +301,7 @@ def test_fan_out_adds_files_and_manifest_rows(self):
             rows = self._manifest(out)["skills"]
             self.assertEqual([r["skill_name"] for r in rows], ["alpha", "beta"])
             self.assertEqual(rows[1]["live_skill_path"],
-                             os.path.join(live_root, "beta", "SKILL.md"))
+                             _canonical(os.path.join(live_root, "beta", "SKILL.md")))
             self.assertEqual(
                 rows[0]["sha256"],
                 hashlib.sha256(b"# alpha\n").hexdigest(),
@@ -368,6 +372,122 @@ def publish(index):
             ) as handle:
                 self.assertEqual(handle.read().strip(), os.path.basename(latest))
 
+    def test_latest_pointer_retries_a_transient_windows_sharing_violation(self):
+        from skillopt_sleep import staging as staging_mod
+
+        with tempfile.TemporaryDirectory() as tmp:
+            root = os.path.join(tmp, ".skillopt-sleep", "staging")
+            night = os.path.join(root, "20260815-010203")
+            os.makedirs(night)
+            with open(os.path.join(night, "manifest.json"), "w", encoding="utf-8") as f:
+                f.write("{}")
+            real_replace = os.replace
+            calls = []
+
+            def transient_replace(source, destination):
+                calls.append((source, destination))
+                if len(calls) == 1:
+                    raise PermissionError("simulated Windows sharing violation")
+                return real_replace(source, destination)
+
+            with mock.patch.object(staging_mod.os, "name", "nt"), mock.patch.object(
+                staging_mod.os, "replace", side_effect=transient_replace
+            ), mock.patch.object(staging_mod.time, "sleep") as sleep:
+                staging_mod._publish_latest(root, night)
+
+            self.assertEqual(len(calls), 2)
+            sleep.assert_called_once_with(0.005)
+            with open(os.path.join(root, ".latest"), encoding="utf-8") as handle:
+                self.assertEqual(handle.read(), "20260815-010203\n")
+
+    def test_generic_atomic_write_never_retries_permission_error(self):
+        from skillopt_sleep import staging as staging_mod
+
+        with tempfile.TemporaryDirectory() as tmp:
+            destination = os.path.join(tmp, "live.md")
+            with open(destination, "wb") as handle:
+                handle.write(b"original")
+            with mock.patch.object(staging_mod.os, "name", "nt"), mock.patch.object(
+                staging_mod.os,
+                "replace",
+                side_effect=PermissionError("live file is busy"),
+            ) as replace, self.assertRaises(PermissionError):
+                staging_mod._write_atomic_bytes(destination, b"proposal")
+
+            replace.assert_called_once()
+            with open(destination, "rb") as handle:
+                self.assertEqual(handle.read(), b"original")
+            self.assertFalse(any(name.startswith(".tmp-") for name in os.listdir(tmp)))
+
+    def test_latest_pointer_persistent_error_preserves_destination_and_cleans_temp(self):
+        from skillopt_sleep import staging as staging_mod
+
+        with tempfile.TemporaryDirectory() as tmp:
+            root = os.path.join(tmp, ".skillopt-sleep", "staging")
+            night = os.path.join(root, "20260815-010203")
+            os.makedirs(night)
+            with open(os.path.join(night, "manifest.json"), "w", encoding="utf-8") as f:
+                f.write("{}")
+            pointer = os.path.join(root, ".latest")
+            with open(pointer, "w", encoding="utf-8") as handle:
+                handle.write("20260814-010203\n")
+
+            with mock.patch.object(staging_mod.os, "name", "nt"), mock.patch.object(
+                staging_mod.os,
+                "replace",
+                side_effect=PermissionError("pointer remains busy"),
+            ) as replace, mock.patch.object(staging_mod.time, "sleep"), self.assertRaises(
+                PermissionError
+            ):
+                staging_mod._publish_latest(root, night)
+
+            self.assertEqual(replace.call_count, 21)
+            with open(pointer, encoding="utf-8") as handle:
+                self.assertEqual(handle.read(), "20260814-010203\n")
+            self.assertFalse(any(name.startswith(".tmp-") for name in os.listdir(root)))
+
+    def test_staging_descendant_accepts_root_alias_but_rejects_child_symlink(self):
+        from skillopt_sleep import staging as staging_mod
+
+        with tempfile.TemporaryDirectory() as tmp:
+            real_root = os.path.join(tmp, "real-staging")
+            os.makedirs(real_root)
+            alias_root = os.path.join(tmp, "staging-alias")
+            outside = os.path.join(tmp, "outside")
+            os.makedirs(outside)
+            try:
+                os.symlink(real_root, alias_root, target_is_directory=True)
+                os.symlink(
+                    outside,
+                    os.path.join(real_root, "child-alias"),
+                    target_is_directory=True,
+                )
+            except OSError:
+                self.skipTest("directory symlinks unavailable")
+
+            ordinary_lexical = os.path.join(real_root, "backup.md")
+            with open(ordinary_lexical, "w", encoding="utf-8") as handle:
+                handle.write("backup")
+            # WAL/manifest paths are canonical, while callers can still supply
+            # the same staging root through a lexical alias.
+            ordinary = os.path.realpath(ordinary_lexical)
+            escaped = os.path.join(alias_root, "child-alias", "outside.md")
+            with open(os.path.join(outside, "outside.md"), "w", encoding="utf-8") as handle:
+                handle.write("outside")
+
+            self.assertTrue(
+                staging_mod._existing_path_is_canonical_staging_descendant(
+                    ordinary,
+                    alias_root,
+                )
+            )
+            self.assertFalse(
+                staging_mod._existing_path_is_canonical_staging_descendant(
+                    escaped,
+                    alias_root,
+                )
+            )
+
     def test_latest_ignores_a_symlinked_night(self):
         with tempfile.TemporaryDirectory() as tmp:
             real_night = write_staging(
diff --git a/tests/test_superpowers_scenarios.py b/tests/test_superpowers_scenarios.py
index be6e9ec0..a724a1c4 100644
--- a/tests/test_superpowers_scenarios.py
+++ b/tests/test_superpowers_scenarios.py
@@ -1,5 +1,6 @@
 """Tests for Superpowers skill evaluation (offline, no API)."""
 import os
+import re as _re
 import subprocess
 import tempfile
 from pathlib import Path
@@ -7,8 +8,6 @@
 
 import pytest
 
-import re as _re
-
 from skillopt_sleep.adapters.superpowers import (
     VERIFICATION_SCENARIOS,
     _get_scenarios,
@@ -269,6 +268,7 @@ def test_flaky_scenario_requires_observed_failure_before_success(self):
         results = [_score_check(c, "1 passed", None, evidence) for c in flaky["judge"]["checks"]]
         assert all(results) is False
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX pytest shims")
     def test_shim_counts_real_invocations(self):
         """The shim logs every pytest run, including `python -m pytest`."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -288,6 +288,7 @@ def test_shim_counts_real_invocations(self):
                 "failures": 0,
             }
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX pytest shims")
     def test_shim_handles_shell_metacharacters_in_paths(self):
         with tempfile.TemporaryDirectory() as tmpdir:
             ws = Path(tmpdir) / "space $HOME"
@@ -302,6 +303,7 @@ def test_shim_handles_shell_metacharacters_in_paths(self):
             assert _pytest_run_count(log, "abc123") == 1
             assert _pytest_outcome_counts(log, "abc123")["successes"] == 1
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX pytest shims")
     def test_python_shim_matches_module_arguments_not_command_text(self):
         with tempfile.TemporaryDirectory() as tmpdir:
             ws = Path(tmpdir)
@@ -323,6 +325,7 @@ def test_python_shim_matches_module_arguments_not_command_text(self):
             assert _pytest_run_count(log, "abc123") == 1
             assert _pytest_outcome_counts(log, "abc123")["successes"] == 1
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX pytest shims")
     def test_zero_work_and_skipped_runs_are_not_successes(self):
         """Exit code zero alone is not evidence that a test actually passed."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -362,6 +365,7 @@ def test_pytest_after_edit_fails_closed_on_broken_source_symlink(self):
             (ws / "broken.py").symlink_to(ws / "missing.py")
             assert _pytest_after_edit(log, ws) is False
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX pytest shims")
     def test_shim_stamps_attempt_number(self):
         """SKILLOPT_ATTEMPT is set by the shim, so the flaky test can't be faked."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -427,6 +431,7 @@ def test_harness_verify_ignores_project_pytest_hooks_and_config(self):
             )
             assert _harness_verify(ws, dict(os.environ), test_paths=["test_guard.py"]) is True
 
+    @pytest.mark.skipif(os.name != "posix", reason="test executes POSIX agent shims")
     def test_agent_shim_does_not_reuse_stale_bytecode(self):
         with tempfile.TemporaryDirectory() as tmpdir:
             ws = Path(tmpdir)
@@ -455,6 +460,7 @@ def test_agent_shim_does_not_reuse_stale_bytecode(self):
             }
 
 
+@pytest.mark.skipif(os.name != "posix", reason="Superpowers adapter requires POSIX bash")
 class TestOverlayIntegration:
     """Mocked tests proving skill overlay and bootstrap are set up correctly."""
 
@@ -712,6 +718,7 @@ def _run(self, workspace):
             )
         return result, mock_run
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_no_host_credentials_by_default(self):
         """Regression: host ~/.claude auth/config is never linked into scenario HOME."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -720,6 +727,7 @@ def test_no_host_credentials_by_default(self):
             claude_dir = workspace / "home-test" / ".claude"
             assert list(claude_dir.iterdir()) == []
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_env_is_scrubbed(self, monkeypatch):
         monkeypatch.setenv("SECRET_TOKEN", "leak-me")
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -729,6 +737,7 @@ def test_env_is_scrubbed(self, monkeypatch):
             assert "SECRET_TOKEN" not in env
             assert env["HOME"] == str(workspace / "home-test")
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_path_is_minimal_by_default(self, monkeypatch):
         """Host PATH is not inherited unless SKILLOPT_INHERIT_PATH=1."""
         monkeypatch.setenv("PATH", f"/opt/hostonly/bin{os.pathsep}/usr/bin")
@@ -741,6 +750,7 @@ def test_path_is_minimal_by_default(self, monkeypatch):
             assert ".skillopt" in path  # shim dir still present
             assert "/usr/bin" in path
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_path_inherit_opt_in(self, monkeypatch):
         monkeypatch.setenv("PATH", f"/opt/hostonly/bin{os.pathsep}/usr/bin")
         monkeypatch.setenv("SKILLOPT_INHERIT_PATH", "1")
@@ -749,6 +759,7 @@ def test_path_inherit_opt_in(self, monkeypatch):
             _, mock_run = self._run(workspace)
             assert "/opt/hostonly/bin" in mock_run.call_args.kwargs["env"]["PATH"]
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_skill_name_traversal_rejected(self):
         """A skill_name with path separators must not redirect the overlay write."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -763,6 +774,7 @@ def test_skill_name_traversal_rejected(self):
                             skill_overlay=None, workspace=workspace,
                         )
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_fails_closed_without_auth(self, monkeypatch):
         monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -792,6 +804,7 @@ def test_harness_verify_drops_credential(self):
             assert mock_run.call_args.kwargs["env"]["PATH"] == "/scrubbed/bin"
             assert "-m" in mock_run.call_args[0][0]
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_missing_bootstrap_flags_error(self):
         """Absent using-superpowers SKILL.md must surface a distinct error."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -820,6 +833,7 @@ def test_harness_verify_respects_timeout(self, monkeypatch):
                 _harness_verify(ws / "p", {}, timeout=600)
             assert mock_run.call_args.kwargs["timeout"] == 600
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_claude_bin_override(self, monkeypatch):
         monkeypatch.setenv("SKILLOPT_CLAUDE_BIN", "/custom/claude")
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -859,6 +873,7 @@ def test_symlinked_candidate_refused(self):
             with pytest.raises(ValueError, match="must not be a symlink"):
                 SuperpowersEvaluator().evaluate(candidate_skill_path=str(link))
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_private_runner_also_refuses_symlinked_candidate(self):
         with tempfile.TemporaryDirectory() as tmpdir:
             workspace = Path(tmpdir)
@@ -884,6 +899,7 @@ def test_private_runner_also_refuses_symlinked_candidate(self):
                     workspace=workspace,
                 )
 
+    @pytest.mark.skipif(os.name != "posix", reason="scenario runner requires POSIX bash")
     def test_symlinked_overlay_path_refused(self):
         """A symlinked skills/ component in the checkout must be refused, no write."""
         with tempfile.TemporaryDirectory() as tmpdir:
@@ -935,6 +951,7 @@ def test_git_timeout_has_clear_error(self):
                     _run_git_step(["fetch", "origin"], Path(tmpdir), timeout=12)
 
 
+@pytest.mark.skipif(os.name != "posix", reason="Superpowers adapter requires POSIX bash")
 class TestPermissionModes:
     """Tests for permission handling in cmd construction."""