Skip to content

feat: P8 runtime performance and resilience hardening - #303

Open
masarray wants to merge 42 commits into
fix/fat-workstation-convergencefrom
feat/p8-runtime-performance-resilience
Open

feat: P8 runtime performance and resilience hardening#303
masarray wants to merge 42 commits into
fix/fat-workstation-convergencefrom
feat/p8-runtime-performance-resilience

Conversation

@masarray

@masarray masarray commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

Implements the P8 Runtime Performance & Resilience hardening layer on top of the FAT workstation convergence work in #296.

This PR preserves a strict protection-workstation invariant: Report/GOOSE/SOE/process evidence remains event-by-event; only the visual latest-value projection may be coalesced.

A second regression audit was completed before closure. That audit deliberately removed two unused experimental helpers (LatestValueUiBatcher and PooledByteBufferLease) rather than adding parallel authorities or speculative pooling to a production path that is already working.

Final P8 coverage

  • P8.1 — per-IED async isolation: regression-locks the existing per-device lifecycle gates, pre-emptive stop lane, per-device monitor CTS/client/session, independent reconnect state, and existing multi-IED FAT isolation behavior.
  • P8.2 — UI throttling/batching: regression-locks the production 200 ms WPF latest-value coalescing path while SOE remains a separate FIFO ConcurrentQueue. No second batching framework is introduced.
  • P8.3 — virtual scrolling: regression-locks FAT row+column virtualization, recycling mode, and content scrolling; existing RCB virtualization behavior remains untouched.
  • P8.4 — leak/shutdown prevention: regression-locks timer shutdown, application cancellation, bounded async runtime disposal, active-operation cancellation, and per-device ownership boundaries.
  • P8.5 — defensive telemetry envelope: separates process value, quality, relay/source timestamp, and local receive timestamp. Missing values are Invalid; unknown/vendor quality is never promoted to Good; malformed/missing relay timestamps remain unknown rather than becoming PC time.
  • P8.6 — allocation profiling: adds low-overhead GC/allocation snapshots and defensive deltas without forcing collections or introducing flaky CI timing thresholds.
  • P8.7 — pooling gate: no production pooling is introduced because the audit found no measured ARSAS-owned hotspot with a proven lifetime boundary. Critical ARIEC61850/Npcap/Report/GOOSE runtime paths are regression-guarded against speculative ArrayPool insertion.

Regression coverage

  • null/malformed telemetry normalization
  • no fabricated relay timestamp
  • unproven quality cannot become Good
  • ARIEC zone-less IEC UtcTime display remains UTC, not local PC time
  • production latest-value coalescing vs lossless SOE queue
  • independent per-IED lifecycle/reconnect state
  • existing independent multi-IED FAT behavior
  • FAT virtualization contract
  • bounded shutdown/disposal ownership
  • allocation delta invariants and no forced GC
  • no speculative pooling in ambiguous native/process-bus ownership paths

Safety decisions from the second audit

  • Removed the unused generic UI batcher instead of creating a second presentation authority.
  • Removed the preliminary pooled-buffer helper because P8.6 profiling had not proven a production hotspot.
  • Kept the existing production MainWindow batching path unchanged because it already separates visual coalescing from authoritative SOE evidence.
  • No value ?? 0 or equivalent missing-data promotion is introduced.
  • ReceivedAtUtc is local receipt metadata and is never substituted for missing relay time.

See docs/P8_RUNTIME_PERFORMANCE_RESILIENCE.md for the final invariants and audit decisions.

Stacked base: fix/fat-workstation-convergence / #296. Do not merge into the lower stack until CI and field gates are accepted.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78c5d009f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Services/Iec61850TelemetryEnvelope.cs Outdated
Comment thread Services/Iec61850TelemetryEnvelope.cs Outdated
Comment thread Services/LatestValueUiBatcher.cs Outdated
Comment thread Services/LatestValueUiBatcher.cs Outdated
Comment thread Services/LatestValueUiBatcher.cs Outdated

Copy link
Copy Markdown
Owner Author

@codex review

Please review the current head after the P8 second-audit cleanup, with extra attention to telemetry timestamp/quality semantics, allocation metrics, concurrency/regression risks, and whether the production batching/virtualization/reconnect contracts remain unchanged.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4445f6d44

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Services/Iec61850TelemetryEnvelope.cs Outdated
Comment thread Services/RuntimeAllocationSnapshot.cs Outdated
Comment thread Services/RuntimeAllocationSnapshot.cs

Copy link
Copy Markdown
Owner Author

@codex review

Please review the current head after addressing all three second-review P2 findings: strict complete relay timestamp parsing, monotonic allocation-rate timing, and explicit current-managed vs last-GC heap semantics. Focus on any remaining regression or correctness risks before field test.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ad2babfc1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Services/Iec61850ReadValue.cs

Copy link
Copy Markdown
Owner Author

@codex address that feedback

Please fix the remaining P1 on the current PR branch, not by weakening tests and not by adding a parallel runtime. Requirements:

  1. Integrate Iec61850TelemetryEnvelope into the actual production decoder/read-to-runtime path used by live MMS polling/discovery, so normalization is not test-only.
  2. ApplyDiscoveryReadValue must never promote absent/undecoded quality to Good; use the envelope's conservative quality semantics.
  3. The live monitor path must consume the normalized envelope before writing RuntimePointState, Iec61850PointSnapshot, and SOE/event metadata. Missing/malformed/incomplete relay timestamps must remain unknown (-/null as appropriate) and must never become PC time. Preserve a valid complete relay timestamp and explicit Good quality unchanged.
  4. Do not batch/drop/reorder authoritative Report/GOOSE/SOE events and do not change acquisition mode selection/reconnect behavior.
  5. Add behavioral regression tests that prove the production path (not only ToTelemetryEnvelope() itself): missing quality is not Good; malformed/partial source timestamp stays unknown; valid Good + complete source timestamp passes through.
  6. Keep the existing P8 architecture invariants and full build compatibility.

After implementing, please update the P8 docs if needed. Do not merge the PR.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

Copy link
Copy Markdown
Owner Author

Cross-thread integration note: COMTRADE P1D.7 is now on main at c3e2ac3ef6e6a3a49f47c5155a0ba4a7c00bcb6b. No overlapping runtime files were found between this P8/FAT stack and the COMTRADE integration. Keep this PR stacked/unmerged until the FAT field gate is accepted; before final landing, refresh the lower FAT stack against current main and rerun CI to prove the combined tree.

Copy link
Copy Markdown
Owner Author

Cross-thread coordination note added to main: docs/WORKSTREAM_COORDINATION.md (commit b0ec4828755272c3331680098d0c95d9f467d84b). COMTRADE P1D.7 is already integrated in main via PR #300. Before P8/FAT final landing, integrate the latest main, preserve the COMTRADE no-regression boundary documented there, rerun exact-head CI on the combined codebase, and repeat the FAT field gate. Do not merge this stack from a stale pre-P1D.7 base.

Copy link
Copy Markdown
Owner Author

Final coordination update from the completed COMTRADE lane: latest main is now 3ad21a631d09a175a8030a0764fd880d0ee77560 (runtime COMTRADE integration at c3e2ac3e... plus durable coordination/lock cleanup). Before P8/FAT final landing, integrate this main or newer, read root AGENTS.md §17 and docs/WORKSTREAM_COORDINATION.md, preserve accepted COMTRADE behavior, then rerun exact combined-head CI + the FAT field gate. Do not merge the stale stacked head directly over current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant