Repository navigation
Attribute protocol-property timeouts to a replayable run - #4
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Attribute protocol-property timeouts to a replayable run
Summary
Replace the single opaque protocol-property namespace run with a discovered,
source-ordered run of its individual
deftestvars. Before each property starts,the runner now records:
:hegel-run.Each property has a 60-second watchdog inside the existing 300-second aggregate
watchdog. A timeout stops the remaining suite, because Jolt does not expose
cooperative future cancellation and starting another property beside the stuck
worker would make both results suspect.
Why
On exact head
c30804f98cc9bb9428a556745d4d1a36f9190ebb, one macOS CI attempt reachedthe protocol-property namespace and hit its 300000 ms watchdog. The failure said
only
protocol properties timed out; it did not identify which of 18 propertieswas active or the Hegel seed. A same-SHA PR-event run and failed-job rerun passed,
confirming that the symptom was intermittent but leaving no evidence with which
to isolate it.
Hegel already resolves and supplies a fresh seed internally, but the pinned API
returns that seed only after a run completes. Choosing the seed at the jolt-http
runner boundary preserves randomized coverage while making a hung invocation
identifiable and replayable before control enters Hegel.
The reported phase is intentionally coarse. The current API does not expose
whether libhegel is generating, executing, or shrinking, so this change reports
only the source-supported fact that the var entered its Hegel run.
Fail-closed and non-vacuous controls
running a normal green suite.
:onceor:eachfixtures that it does not yet know how to apply.nonempty property/seed/phase context, ordinary-scenario continuation, and
protocol-timeout abort; it releases and joins its blocked worker.
JOLT_HTTP_FAULT_HANG_PROPERTYdeliberately blocks a selected real propertyonly after its context is durable, exercising the production watchdog and
report path.
JOLT_HTTP_FAULT_THROW_PROPERTYproves an unexpected real property-bodythrow reaches the same attributed report path instead of being swallowed by
clojure.test/test-var.Chiasmus model
The bounded model covers the 18 current property IDs, a 60000 ms property bound,
and the existing 300000 ms aggregate bound. Its violation query asks whether a
timeout or throw can pass the gate or be reported without exact property, seed,
and observable phase.
The model, controls, source facts, assumptions, and semantic gaps are checked in
under
docs/proofs/protocol-property-attribution.md.Objective measure
The target was set before implementation: preserve all 18 properties and 23
assertions, keep the healthy focused run at or below 31 seconds (20 percent over
the 25.82-second baseline), localize an injected hang at a 100 ms test bound,
and retain the 300-second aggregate ceiling.
The final comparison alternated three base and three candidate runs on the same
machine and dependency cache rather than selecting a best run:
The focused healthy median is 5.5 percent faster than the paired base median;
every candidate sample also passed the precommitted 31-second ceiling. The
eight added checks are watchdog, timeout-policy, fixture, and fault-selector
semantic controls.
Validation
Exact base:
c73db8da2218e32b8a1b94ea777ce00a9563b3f6Exact head:
849db5b7cfd1ff6b4a09eb486d8088cb9533debeAll Jolt commands ran through the workspace's Chez 10.4.1 fail-closed wrapper.
All four SMT-LIB files were linted and verified from disk with Chiasmus/Z3.
Exact-head hosted validation also passed on both workflow triggers:
34005859972: canonical policy, Linux x86_64, macOS arm64;34005858250: canonical policy, Linux x86_64, macOS arm64.Scope
This changes only the test runner, protocol-property test support, and bounded
proof documentation. It does not change HTTP production behavior, loosen a
timeout, require aspects/instrumentation, or claim to reproduce the original
macOS scheduling condition.
Tracks chucklehead-dev/jolt-aspect-packs#101.