Skip to content

fix(flagd): reconnect without error when sync stream deadline is reached - #1955

Draft
alxckn wants to merge 1 commit into
open-feature:mainfrom
alxckn:fix/flagd-stream-deadline-reconnect
Draft

alxckn wants to merge 1 commit into
open-feature:mainfrom
alxckn:fix/flagd-stream-deadline-reconnect

Conversation

@alxckn

@alxckn alxckn commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

This PR

Treats a DEADLINE_EXCEEDED on the in-process sync stream as a routine reconnect rather than a stream error, when the stream already delivered data.

Why

streamDeadlineMs (FLAGD_STREAM_DEADLINE_MS, 10 min by default) is documented as an application-layer keepalive, so a healthy stream reaching it is expected. Today it goes through the error path in SyncStreamQueueSource.handleObserverError, which on every instance, every 10 minutes:

  1. enqueues ERROR → the provider emits PROVIDER_STALE and schedules PROVIDER_ERROR after the grace period;
  2. sets shouldThrottle → waits retryBackoffMaxMs before reconnecting, so the provider stays STALE without a stream for the full backoff;
  3. if retryBackoffMaxMs exceeds the grace period, logs Provider did not reconnect successfully within Xs and emits PROVIDER_ERROR for what was a planned recycle.

With this change, a deadline on a stream that received at least one payload is handled like onCompleted(): immediate reconnect, no event. flagd re-sends the full flag state on reconnect, so nothing is missed.

Keeping the liveness check

The deadline exists to detect hung streams, so a stream that reaches its deadline without ever delivering data is still reported as an error (with backoff), exactly as before. Both cases are covered by new tests.

Consistency across providers

The Python provider made the matching change in open-feature/python-sdk-contrib#405 ("log expected sync-stream recycles at debug, not warning"), based on the same reasoning: with a 10 min default deadline, many stream "errors" are expected, and only the stale -> error transition is a real error. This PR brings the Java provider's in-process mode in line, and also goes one step further by skipping the STALE event and the backoff.

Related discussion: open-feature/flagd#1959. A server-side complement (closing streams with OK at the deadline, which fixes all providers and tracing) is proposed in open-feature/flagd#2098.

Scope / notes

  • In-process (SyncStreamQueueSource) only. The RPC resolver (RpcResolver.observeEventStream) deliberately treats even a graceful onCompleted as an error (it clears the evaluation cache), so changing that is a separate discussion. Happy to follow up if maintainers want the two modes aligned.
  • No new option. Fatal status codes are still checked first, so DEADLINE_EXCEEDED listed in fatalStatusCodes still shuts down as before.

How to test

mvn --projects providers/flagd verify

deadlineExceededAfterDataReconnectsWithoutErrorOrBackoff fails without the fix. deadlineExceededWithoutDataEnqueuesErrorPayload guards the liveness behavior.

🤖 Generated with Claude Code

The stream deadline (FLAGD_STREAM_DEADLINE_MS, 10 min by default) is an
application-layer keepalive: a healthy stream reaching it is expected.
It was nevertheless handled as a stream error, emitting STALE (and ERROR
after the grace period) and waiting retryBackoffMaxMs before
reconnecting, every 10 minutes on every instance.

A DEADLINE_EXCEEDED on a stream that already delivered data is now
handled like a completed stream: immediate reconnect, no error event.
A stream reaching its deadline without delivering any data is still
reported as an error.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Alexandre Chakroun <achakroun@macmail.fr>
@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants