Repository navigation
Close WaveStar NMPC portable physical closed-loop gap #623
Description
Published gap
WECCCOMP/WECCCOMP_Nonlinear_Model_Predictive remains qualified. An independent 225 s opt-in source-ode8 Python run passes the existing pitch/pitch-speed/PTO force/work numerical gates on Apple silicon. Two Linux x86 runners with the same phases and package versions disagree enough that one passes and the other fails the same pointwise limits; local versus archived MATLAB sources also differ after feedback begins. The public physical path passes seven 0.5% aggregate split-step measures locally, but its autonomous pointwise pitch error is 0.007289 rad versus the unchanged 0.0005 rad gate.
Next work
- Reproduce and isolate runner-dependent recursive controller amplification using identical plant states and commands; distinguish QP, AR predictor, observer, and floating-point dispatch effects. Existing prescribed-command tests already show nearly identical plants, so do not repeat them unchanged.
- Establish a stable platform-matched MATLAB target from the pinned HDF5 and sea. Then independently run the full 225 s Python plant and NMPC from those inputs, with no saved MATLAB stroke, torque request, body state, or force driving feedback.
- If deterministic numerical changes are justified, keep source-ode8 compatibility opt-in and preserve the physical default. Gate complete motion, controller requests, PTO force, and work on supported CI runners.
Closeout
A durable full-duration autonomous physical gate must pass the predeclared pointwise and aggregate limits on supported platforms against a checked source baseline. Source-driven component replay and an Apple-silicon-only source-ode8 pass do not promote the strict case badge. If the MATLAB baseline itself remains runner-sensitive, document the required upstream or numerical correction and keep the case qualified.
Evidence: ledger, local full source gate, physical split-step portable test.
Activity
cmccomb commentedon Oct 23, 2026
Reference-generation update from Python PR #629: I generated fresh independent 225 s MATLAB R2025b WaveStar NMPC traces with the pinned HDF5 and sea, using workflows and stored diagnostics only in
wec-sim-python. Neither WEC-Sim MATLAB repository received a change or PR.The first 0.5 and 0.25 ms source runs differ by 0.004763 rad pitch, above the existing 0.0005 rad pointwise limit. Two 0.25 ms runs on different Intel/AMD hosted CPUs differ by 0.012014 rad pitch and 0.732573 N m request even with one MATLAB computational thread. A compatible-BLAS repeat produced identical full-duration source traces on two AMD EPYC 7763 runners; that same-CPU result does not establish cross-CPU portability.
Against that repeated source trace, an independent physical Python run at the matching 0.25 ms plant step still misses all five pointwise limits: maximum pitch gap 0.017938 rad versus 0.0005 rad, and request gap 0.937424 N m versus 0.1 N m. Pitch-peak aggregate difference is 0.6762% versus 0.5%. A frozen-input QP probe in the PR reproduces each earlier source's requests closely, narrowing the investigation toward closed-loop numerical amplification.
The same physical plant does pass a separate 225 s prescribed-request component check when given the source torque with its one-sample actuator delay: maximum pitch gap 0.0000625 rad, pitch-speed gap 0.000368 rad/s, stroke gap 0.0000125 m, force gap 0.000962 N, and PTO-work gap 0.0233%. This checked component audit passed on Linux x86 and Apple silicon with matching results. Its source requests are supplied as inputs, so this result does not meet the autonomous closeout gate.
A complete 4,501-sample controller replay on the checked compatible source also passes on Linux x86 and Apple silicon. On source stroke and preceding requests, the observer gap is at most 1.75e-12, the active AR forecast gap 2.77e-11 N m, and the complete requested-torque gap 0.001070 N m peak / 0.0000335 N m RMS, within the established component limits. A QP replay on the source's logged states and forecasts has a similar 0.001071 N m peak gap. This narrows the fixed-input controller discrepancy to the QP side; it still does not remove autonomous closed-loop amplification.
The checked compact traces, audits, workflow, and probe are in PR #629. The documented closeout gate has not passed, so this issue remains open. The remaining work is a stable source target across supported platforms and a justified numerical correction that yields a full-duration independent physical pass at the existing limits.
Further QP isolation: a direct 4,201-sample MATLAB replay on the checked source states/forecasts and controller matrices reproduces the source requests within 6.43e-11 N m. Its complete 40-step plans are checked in PR #629. Python's independently carried plan differs by up to 0.002058 N m and its first request by 0.001071 N m. When each Python QP starts from the preceding MATLAB plan, the one-step request and plan gaps drop to 1.76e-7 N m on macOS ARM. The fixed-input difference is therefore amplified by the persistent nominal-plan recursion. This sharpens the Python-side defect but does not yet satisfy the independent physical closeout gate.
Merge update (2026-10-23): PR #629 passed every check and merged to
masterase5d065da. The checked source traces, controller plans, reproducible audits, and Linux x86 / Apple silicon component gates are now durable in this Python repository. A local exploratory Python interior-point QP trial at the source solver's default optimality tolerance still missed the independent 225 s physical limits (maximum pitch gap 0.007413 rad versus 0.0005; request gap 0.611864 N m versus 0.1). The production controller was not changed. The original autonomous closeout gate is still unmet, so #623 remains open and the case qualified.cmccomb commentedon Oct 23, 2026
Python-only PR #650 adds checked platform diagnostics without changing the MATLAB repositories or the original acceptance limits.
The 225 s MATLAB source is exactly repeatable within each tested native environment: compatible oneMKL on two AMD EPYC models, Apple Accelerate on Apple silicon, and native oneMKL on Intel macOS. Yet the Apple and Intel traces differ from the checked AMD source by up to 0.006697/0.006825 rad pitch and 0.472857/0.507981 N m requested torque, exceeding the original 0.0005 rad and 0.1 N m pointwise limits. Their pitch traces agree to about 2e-15 rad before NMPC activation and diverge after feedback begins.
Independent physical Python against its own matching source still fails all five pointwise limits on both Apple and Intel. Apple pitch/request maxima are 0.005875 rad / 0.542261 N m, and its pitch-peak aggregate gap is 0.6555% versus the original 0.5% limit. Both Intel physical runs fail pointwise (pitch 0.007962/0.007568 rad; request 0.722569/0.605466 N m), although their seven aggregate gaps pass. Refining the Python plant step through startup from 1 ms to 0.0625 ms leaves the checked-source pitch gap near 1.06414e-5 rad, so Python RK4 step truncation alone does not remove the seed discrepancy.
A direct 4,201-sample MATLAB QP replay on identical checked states, forecasts, and matrices succeeded on compatible Linux, Intel macOS, and Apple silicon. Its first commands differ across platforms by at most 8.75e-11 N m and complete plans by at most 1.68e-10 N m. The large source differences therefore develop through autonomous feedback from much smaller numerical differences.
An additional first-active-sample probe of those source controller logs narrows the initial command separation: at 15 s, Intel/Apple observer-state differences from the AMD source are at most 2.4e-14/1.84e-13, while their AR forecast differences are 3.4e-12/5.72e-12 N m. With the prior QP plan fixed at zero, swapping only the forecast accounts for nearly all the roughly 6.4e-14/8.2e-14 N m first-command differences. This points to the predictor path as the earliest active platform perturbation, while leaving its exact floating-point operation and the physical Python correction unresolved.
The checked fixtures, source/physical comparisons, runtime provenance, and reproducing tools are in PR #650. The predeclared full-duration physical pointwise and aggregate gate remains unmet. #623 remains open and the WaveStar NMPC case remains qualified. Closing it still requires a source target whose numerical behavior is justified across supported platforms and plant steps, plus a Python correction that passes the unchanged autonomous gate on those platforms.
cmccomb commentedon Oct 23, 2026
Further Python-only isolation of the first autonomous pointwise failure, using the checked 0.25 ms AMD source fixture and an independent physical Python run at the same plant step:
6.4114e-6rad and requested torque by3.364e-5N m. Refining the Python plant step through 0.0625 ms leaves that startup pitch gap essentially unchanged (the checked startup-refinement report in PR Audit WaveStar NMPC source across CPU models and startup steps #650).0.107743N m. At 17.35 s, the independent Python request is2.223799N m versus the source's2.120644N m, a0.103155N m gap that first exceeds the unchanged0.1N m request limit. The simultaneous observer-state and AR-forecast maximum differences are only0.000398and0.016268N m respectively.2.120523N m, just0.000121N m from source. Feeding the source state and forecast with that MATLAB plan gives2.120644N m, within4e-9N m of the source request. Conversely, feeding source state and forecast with the Python-carried plan still gives2.223921N m. This isolates the immediate0.103N m request crossing to accumulated plan history, not a large one-step optimization error at 17.35 s.The checked Apple and Intel MATLAB source requests are within
3.2e-8N m of the AMD request at 17.35 s, so this early finding also applies to the native Apple source comparison. It does not identify a justified change to the published controller or remove later platform-dependent feedback amplification. The complete independent 225 s physical pointwise and aggregate gate remains unmet, and #623 stays open.cmccomb commentedon Oct 23, 2026
Python-only PR #651, merged to
masteras15abf142, adds checked 0.125 ms MATLAB source-step and fixed-input AR diagnostics, using the pinned WEC-Sim and Applications commits without changing either MATLAB repository.The two 225 s Apple silicon source runs at 0.125 ms are identical. A corrected archived-artifact comparison confirms this after fixing the repeat comparator's hardcoded 0.25 ms assumption. However, changing the same source from 0.25 to 0.125 ms changes pitch by up to 0.006753 rad and requested torque by 0.687034 N m; its pitch-peak aggregate differs by 0.5691%. These exceed the unchanged 0.0005 rad, 0.1 N m, and 0.5% limits.
On the matching 0.125 ms source, the independent 225 s physical Python run still misses all five pointwise limits: maximum pitch 0.006806 rad, pitch speed 0.036390 rad/s, stroke 0.001356 m, axial force 2.115091 N, and requested torque 0.434049 N m. All seven aggregates pass. The source traces supply only the pinned hydrodynamics and sea to this Python run; saved source motion and requests do not drive its feedback.
On an identical checked 4,501-sample input sequence, the pinned MATLAB AR predictor's active 15–225 s forecasts differ by at most 2.84e-12 N m among compatible Linux, Intel macOS, and Apple silicon. The much larger autonomous trajectory differences therefore arise through recursive feedback, not a large fixed-input active predictor mismatch.
This is diagnostic evidence, not a closeout pass. The original portable full-duration independent physical pointwise and aggregate gate remains required; #623 stays open and the WaveStar NMPC case stays qualified.
cmccomb commentedon Oct 23, 2026
The 62.5 µs follow-up is now checked in the Python repository as PR #652. Its MATLAB workflow passed: two full 225 s Apple silicon source traces match exactly at all 4,501 samples, so the new reference is reproducible on that runner.
The original autonomous gate still fails. Halving the MATLAB source plant step from 125 to 62.5 µs changes pitch by 0.010800 rad (the 0.0005 rad limit is first crossed at 17.95 s) and requested torque by 0.721063 N·m (0.1 N·m limit). All seven source-to-source aggregate gaps remain under 0.5%, which does not satisfy the pointwise gate. On the matched 62.5 µs source, the independent experimental Python physical run misses all five pointwise limits: 0.008512 rad pitch, 0.040802 rad/s pitch speed, 0.001701 m stroke, 3.1800 N axial force, and 0.659784 N·m requested torque. Its pitch-peak aggregate gap is 0.8871% versus the unchanged 0.5% limit. The light-body correction used in that audit also regresses the existing 1 ms aggregate test and was not merged.
The checked compact source, exact-repeat report, step comparison, and fixed-input controller probe are in PR #652. This closes the missing-reference investigation, but not the issue's required portable physical closed-loop gate. The blocker is pointwise sensitivity in the source/controller feedback itself: a finer, repeatable source trace is still not step-stable at the predeclared limits. The case remains qualified and this issue remains open. No MATLAB repository change was made.
cmccomb commentedon Oct 24, 2026
The new draft diagnostic PR #657 keeps this issue open and preserves the original acceptance limits and qualified status. Its checked 225 s report is reproducible from the pinned HDF5, source traces, and wave phases.
The two MATLAB source runs have identical checked 500-bin sea components and first cross the pitch limit at 17.95 s. Each Python run advances its own plant and controller for all 225 s, with no saved source stroke, pose, force, or request driving feedback; the two Python runs first cross the pitch limit at 43.5 s. These local measurements used macOS ARM64, NumPy 2.5.4, and SciPy 1.18.1. The repo test preserves the source self-change as a strict expected failure.
A separate checked QP report shows Hessian minimum eigenvalues 0.886–0.899 and condition numbers 1.33–1.41 at four sensitive samples. Given the same logged state, forecast, and preceding MATLAB plan, the Python QP has zero bound violations, near-zero residuals, and first requests within 3.1e-9 N m of the direct MATLAB QP at the three non-bound samples. Resetting the preceding plan at 30 s changes the first request from 5.92177 to 0.681924 N m because it changes the RTI linearization; it is not a tighter solve of the published controller.
The remaining correction is a numerically stable, checked 225 s source controller/plant target whose own step and supported-platform variation meet the original pointwise limits, followed by a passing independent physical Python run against that target on supported runners. A solver swap or 30 s prefix does not establish that result. No case status or badge promotion is proposed by PR #657.
cmccomb commentedon Oct 24, 2026
PR #657 has now merged into this Python repository as
e928adc9after all exact-head checks passed. It adds the 225 s source/Python step-sensitivity and QP diagnostics described above. The pinned MATLAB source and Applications repositories were not changed.The result remains diagnostic: source 0.125 versus 0.0625 ms pitch differs by 0.010800 rad against the retained 0.0005 rad limit, and the independent physical Python step pair also exceeds all pointwise limits. The WaveStar case remains qualified; this issue and the original full-duration physical gate remain open.