Selective Recovery for Microsoft Agent Framework Workflows

[Oluseyi Christopher Olusola (Tarchy01)]

Hi Agent Framework team,

I’m Oluseyi, founder of Consistency.

We’ve built a recovery layer for AI agents and multi-step workflows that focuses on a specific problem after partial failure: deciding which completed work is still valid and which downstream work actually needs to run again.

We have already tested Consistency on Microsoft Agent Framework DAG workflows, including multi-fault recovery scenarios.

The basic idea is:

When part of a workflow fails, Consistency examines the surviving state, preserves the work that is still valid, repairs the failed boundary, invalidates only the downstream state affected by that failure, and reruns only what needs reconstruction.

We also tested the same recovery approach in a live OpenHands agent workflow:

Metric	Restart	Consistency
Model calls	3	2
Prompt tokens	1,961	223
Total tokens	2,043	236
External tool calls	3	1
Recovery time	11.261s	6.451s
Final result	HydroLink	HydroLink

Same final answer, with less repeated work.

Demo:
https://consistency-runtime.netlify.app

Given Agent Framework’s work on durable, restartable workflows, I think Consistency could add another layer of recovery by helping determine which completed steps are still safe to keep after a failure and which ones need to run again.

I’d be interested in feedback from the Agent Framework team on whether this could fit naturally into the framework, and whether there might be room for a small integration or design-partner experiment.

Best,
Oluseyi Christopher Olusola
Founder, Consistency

[Huzaifa Asim (huzaifaasim017)]

A useful integration boundary here is the superstep checkpoint. MAF checkpoints are taken after a superstep finishes and include executor state and pending messages (docs). Selective recovery would need to show what happens when one branch has finished but a sibling branch fails before the next checkpoint; otherwise a “reused” result may exist only in process memory.

For a small design-partner experiment, I would publish a deterministic DAG test with A → {B, C} → D and inject failures at three points: (1) before C's external call, (2) after the call succeeds but before its result is checkpointed, and (3) after the checkpoint but before D. Track per-node input fingerprints, code/model version, side-effect receipts/idempotency keys, and a validity decision (reuse/recompute/unknown). At the B+C fan-in, D must rerun if either effective input changes, even if B's output was reusable. A lost response after an external side effect should be “unknown” until reconciled with the destination, not automatically “safe to rerun.”

The reported token/time reduction is interesting; this failure matrix plus exact external-call counts and final artifact hashes would make the correctness claim reviewable alongside the savings. Does Consistency currently persist those validity decisions in MAF's checkpoint state, or in a separate store tied to the checkpoint ID?

[Oluseyi Christopher Olusola (Tarchy01)]

Thank you for the detailed feedback. You've identified an important boundary that deserves to be made explicit.

Consistency's existing Microsoft Agent Framework integration demonstrates dependency-aware selective recovery across a fan-out/fan-in workflow, including preservation of a healthy sibling branch and regeneration of affected descendants.

However, the current pilot uses MAF's in-memory checkpoint storage and evaluates semantic invalidity after workflow completion.

We have not yet demonstrated that validity decisions are durably persisted within MAF's checkpoint state or in a separate store keyed to checkpoint IDs.

Consistency does support application-provided side-effect reconciliation, including cases where an external action may have completed before its acknowledgement was received. But that is not equivalent to proving crash-safe recovery across all three failure points you've described.

Your proposed A → {B, C} → D experiment is a useful way to expose that distinction.

In particular, I agree that an external action with an uncertain outcome must not automatically be classified as safe to repeat, and that downstream execution must reflect changes to its effective inputs.

The existing benchmark establishes selective recovery under explicit validity and dependency contracts. It does not yet establish the stronger durability guarantees you're asking about.

I appreciate you identifying the distinction. It's exactly the kind of scrutiny needed to establish where Consistency's demonstrated capabilities end and where additional engineering would be required.