[kinyoklion]

Summary

Applications and editors rewrite data files in place: they truncate the file and then write the new content. A reload that reads the file in between finds it empty. On this branch, that reload applied an empty document, so the client briefly served no flags, and the override source briefly removed every override, until the next reload. A slow rewrite (truncate, pause 300 ms, write) made a client serve a -> default -> b. The legacy file data source on main serves a -> b for the same rewrite, because it never accepted an empty file.

The OVERRIDE spec's retry requirement exists to keep "a partially written or briefly invalid file from emptying the layer" (section 1.3). This change makes both file consumers of the shared engine, FileDataSource and FileOverrideSource, keep their data through such a rewrite.

An empty JSON file is invalid

The parser for a file is chosen by its extension, as the OVERRIDE spec recommends (requirement 1.3.3, sdk-specs #272), so whether an empty file is valid is up to the format, like the rest of the content. The JSON parser still treated a file with no content, or white space only, as a document with no entries. That rule came from before formats were chosen by extension. An empty JSON file now fails to parse, with the message the file has no content, which is not a JSON document, and the last good data stays, as for any other content that cannot be parsed. A registered parser that returns null for an empty file, as YAML parsers do, still makes it a document with no entries. This applies to a file with no extension and to any extension without a registered parser, since those are parsed as JSON.

This changes behavior from 9.0.0-alpha.1, where an empty JSON file loaded as no flags or no overrides. To remove every override, write {} or delete the file.

I audited the rest of the shared file engine for other rules that look at content instead of the extension, and found none:

• The parser comes from the extension, case-insensitively.
• A YAML extension without a registered parser is a configuration error.
• The reader removes a byte order mark.
• A missing file is skipped explicitly before parsing, so it never reaches the JSON rule. That doc comment now says "contributes no data" instead of "a file with no content".

A file that is emptied is held back until it has stayed empty

With the JSON rule alone, a JSON file that is empty for a moment fails to parse and is retried. A YAML file can legitimately be empty, so its parser cannot tell an emptied file from one in the middle of a rewrite. The reloader now holds back a file that a load finds with no entries while its content is in effect, that is, while the data in effect came from a load that read content from that file. A file has no entries when it is empty or white space only, or when a registered parser reads it as nothing, such as YAML with only a comment or ---, which a read can see of a file that is being written. Whatever the format:

• The load keeps the current data, logs the hold at debug level, and arms the retry. It reports no failure and uses up none of FileDataSource's five load attempts.
• Every load that finds the file still so within the retry delay (1 second) of when a load first found it so holds it back too, whether it is the retry, a poll, or a change signal for this or another configured file. Once the file has stayed so for the retry delay, the next load parses it: as invalid JSON, which is reported and keeps the data, or as a YAML document with no entries.
• If the file was written since a load last found it empty, which its state shows (its modification time and size), the delay starts over. That covers a file that is rewritten and truncated again between two loads. The states are only compared with each other, never with the clock. On .NET 6 and later the state is read through a symbolic link; on netstandard2.0 and net462 a link is observed by its content, which is the same for every empty state. The time is read after each file's own read, so a slow read of another file does not shorten its delay.
• The retry that a hold arms comes when the first held file's delay or bound is over, plus a tenth of the delay, so that a timer that fires a little early still finds it over. A file that stays empty is applied about 1.1 seconds after a load first found it so. The retry logs that it reads the files again after a file was found empty.
• A file stops being held back once loads have held it back for five retry delays. That bounds a file that keeps being written while every load finds it empty, and files that are emptied in turn so that every load holds one of them back. The bound is the file's own: another file's hold does not shorten it. It starts over only at a load that holds nothing back, whether that load applies, is skipped because nothing changed, or fails, so that a later rewrite is held back again; while some file is held back in every load, no file's bound starts over. A load reads every file even after one fails to read or parse, and fails with the first failure, so that no file keeps a bound from before the failure. A tool that rewrites a file now and then with the same content gets a fresh bound for each rewrite, and so does an operator who saves a file with a mistake and later saves the fix.
• A read that finds content in the file, even content that does not parse, or finds the file missing, ends its empty state, so that its next emptying starts a new delay even where a coarse modification time makes it look as it did before.
• A load reads every file before it holds any back, so files that are emptied together are applied together. Changes to the other files wait while a file is held back, since a reload applies all the files or none.
• A file whose content is not in effect is parsed right away and never delays the other files: an empty placeholder YAML file next to the data, a file whose empty state has been applied, or, where missing files are skipped, a file that is deleted and then created empty. {} is a whole document and is applied right away.
• With retries off (FileDataSource without auto-update), nothing would read the file again, so an empty file is parsed right away. The initial load has no data in effect, so it parses an empty file right away too.

A rewrite is covered when the writer leaves the file empty for less than the retry delay, including two rewrites in quick succession and a change to another file during the gap. A writer that leaves it empty for longer gets the empty file applied (YAML) or reported (JSON), as before. So does a program that regenerates the file in place over and over without a pause, so that every load finds it empty: after five retry delays its empty file is applied. Several files that are each kept written while empty, starting at different times, hold the data back for about five retry delays each, because each file's bound runs on its own. Holding the data back for as long as the file keeps changing would keep stale data without end, which the debounce also refuses to do.

The rule took eight rounds of adversarial review to get right; the Java and Ruby ports' reviews found the round-7 hole:

• Round 1 read an emptied file again on every load that a change started. A placeholder delayed every change, and a stream of changes held the data back without end.
• Round 2 read it again once, on the next load. Two quick rewrites, a change to another file during the gap, and polling with two rewrites a second apart still applied an empty YAML file.
• Round 3 held it for a retry delay, with a bound per file. Its elapsed-time arithmetic overflowed after about 15 minutes on Linux and macOS, two files rewritten in turn held the data without end, and its allowance for early timers let another load apply a gap slightly shorter than the delay.
• Round 4 bounded the run of holding loads, made the delay check exact, and moved to Stopwatch.Elapsed, which converts without overflow and which tests can replace. It read the clock once per load, so a slow read of one file shortened another's delay, and its shared bound was restarted by any failed load and could cut short another file's ordinary rewrite.
• Round 5 read the time per file and gave each file its own bound, which started over when data was applied or when the file had stayed empty for the whole delay. A file rewritten with the same content kept its old bound (fixed during the round), files kept empty and truncated again in staggered turns were held back without end, and a bound left from a long failure of another file cut short a later rewrite.
• Round 6 started a file's bound over only at a load that holds nothing back. A file whose read or parse failed was not counted as read, so an operator who saved a mistake and saved the fix in place seconds later had the fix's truncation applied empty.
• Round 7 counted the file that failed as read, and ended the empty state on content that fails to parse or on a missing file. A load still stopped at the first file that failed, so a file configured after it kept a bound from before the failure, and its next rewrite, during a deploy that fixed the other file, was applied empty.
• Round 8 is the rule above: a load reads every file even after a failure, and fails with the first one.

Documentation

The public docs say what an application needs to know, without the details of the hold:

• Parser on FileDataSourceBuilder and FileOverrideSourceBuilder: a null result means the file has no entries.
• FileOverrides.Source: files can be rewritten in place, and the overrides stay in effect while a file is empty during a rewrite; to remove a file's overrides, delete it or write {}; an empty JSON file is not valid; replace a YAML file in one operation to avoid applying a partly written file.
• FileDataSourceBuilder.AutoUpdate no longer tells applications to replace files atomically whenever possible. It says files can be rewritten in place, and keeps the advice for YAML only (see below).

Not changed: YAML cut off between two entries

A read that overlaps a write can see the first part of a YAML file. If the cut falls between two entries, the part is valid YAML with fewer entries, and it is applied until the next reload. JSON cut off anywhere fails to parse. This predates this change and is the same on main. Telling a part from a whole file needs a content rule this change does not add, so the docs recommend replacing a YAML file in one operation.

Not changed: the first load of a new source

A new reloader has no data in effect, so its first load parses an empty file right away. For FileDataSource that is the start of the client, as before. Under FDv2, a file data source used as a synchronizer is created again when the data system starts, falls back or recovers, and its first load can then read a file that is being rewritten. This predates this change; the other SDKs' reviews found the same.

Difference from the other SDKs

Go still detects the format from the content (internal/filedata/document.go): a file that does not start with { is YAML, so an empty file of any name loads as no overrides. The OVERRIDE spec recommends choosing the format by extension but lets an SDK whose file data source already inspects the content keep doing so (requirement 1.3.3), and it does not say what an empty file means, so the SDKs can differ until it does.

Testing

• The parser rejects an empty or white-space JSON file, including one that is parsed as JSON by default, and keeps returning an empty document for a registered parser's null.
• The reloader tests of how long files are held back run on a clock of their own and run each load with ReloadNow, with a retry delay of an hour on that clock, so their timing does not depend on the machine. They cover:
  • an emptied YAML file held until exactly the retry delay, and applied once it is over; a load five minutes short of it still holds it;
  • an emptied JSON file reported once the delay is over, keeping the data;
  • a file written and truncated again between two loads held for another delay, also when the configured path is a symbolic link and the writes go to its target;
  • two quick rewrites, and a change to another file during the gap, applying nothing empty;
  • three files emptied together applied together;
  • the five-delay bound, for a file kept written while empty, for two files rewritten in turn, and for two files kept empty and truncated again in staggered turns; a failed load of another file does not restart it, another file reaching its bound does not cut short an ordinary rewrite, and a bound from a hold during a failure of another file starts over once a load that holds nothing back reads the file;
  • a file rewritten now and then with the same content held back at each rewrite;
  • the fix of a file saved in place with a mistake held back, for JSON and YAML; the bound of a file configured after a failing file starting over once a failing load reads it, for a parse failure and a read failure; a failing load reporting the first failure; a file kept written while empty applied after its bound while an earlier file fails; a load that holds a file back and then fails starting no bound over; a file deleted and created empty again with the state it had before held back with a new delay;
  • a slow read of an earlier file does not shorten a later file's delay;
  • the retry that a hold asks for: the rest of the delay, the bound when it comes first, the first file due among several, measured from the end of the load and never less than the margin, the same from a change signal, and a whole delay after a failure;
  • a JSON file that was empty and reported for hours held back again when it is rewritten, rather than reported again;
  • a file emptied again long after it had content held back again;
  • white space, a YAML comment and --- held back, and {} applied right away;
  • an empty placeholder, a file deleted and created empty, content moved to another configured file, and retries off.
• On the system clock: the retry that a hold arms applies the rewritten file, and applies a YAML file that stays empty; a retry left over from a failure holds an emptied file back; a retry left over from a hold does nothing once a load on demand has read the file; holding a file back uses up no load attempt.
• End-to-end, with real in-place rewrites (InPlaceWrites.Rewrite in TestUtils: truncate, pause, write):
  • A client with a file data source serves a -> b and never the empty file, for JSON and YAML (300 ms pause).
  • The override source never applies a snapshot without the override, for JSON and YAML, in watching and polling modes: for one rewrite, for two rewrites 30 ms apart, and for a second file written in the middle of the first file's 400 ms rewrite.
• The reloader tests now rewrite their file in place with a 30 ms pause after truncating, so they exercise this case deliberately instead of by chance. With these writes, three of them failed on the base of this branch on every run (ReportsIdenticalFailureOnlyOnce, RetriesAfterFailureWithoutFurtherTriggers, RetryLeftOverAfterASuccessfulReloadNowDoesNotLoad). ReportsIdenticalFailureOnlyOnce now also checks that each report is the one it expects, not a report of the file being empty mid-write.
• RetryBudgetIsPerEpisodeNotPerPath (from feat: Move the file data source onto the shared file data engine #376) counted the reads of each path and expected an attempt that fails at the first path not to read the second; every attempt now reads both.
• ParseFailureFromPartialReadIsRetriedUntilContentIsComplete (from feat: Move the file data source onto the shared file data engine #376) asserted that the source was initialized right after it received the data, which the retry's thread sets just after; it failed once in a loop on a loaded machine and now waits for it.
• To check that the tests catch what they target, each part of the change was altered on its own, with a clean build each time, against the file loading, file data source and override source tests. Every variant fails at least one test:

Variant	Tests failed
The round-7 rule	2 (a file after a failing file, for a parse failure and a read failure)
Stop reading at a read failure	1
Stop reading at a parse failure	2
The last failure reported instead of the first	1
No bound on a file's hold	7
No fresh delay when the file was written since	7
The delay check shortened by a tenth	3
A parser's "nothing" counted as content	2 (the comment and --- cases)
A file that stayed empty for the whole delay clearing its bound	1
Bounds kept after a load that holds nothing back	7
A file that failed to load not counted as reached	2
Bounds started over by a load that held a file back and then failed	1
A missing file keeping its empty state	1
Bounds cleared by every load, holding or not	4
A file's bound cleared when it is read with content	2
The clock read once per load	1
The hold's retry armed for a whole delay instead of the time left	5
No margin on the hold's retry	6
No clamp at zero on the hold's retry	1
The hold's retry ignoring the bound	1
The last held file deciding the retry	1
ReloadNow arming a whole delay	4
A change signal arming a whole delay	1
Files whose content is not in effect held back	3
Content in effect not recorded	46
A file's empty state kept after it is read with content	2 (tests that pin the modification time, as a coarse file system would)
Stop at the first held-back file	2
Hold back with retries off	1
The hold arms no retry	10
The retry after a hold treated as stale	3
A symbolic link observed by its own metadata	1
White space counted as content	1
The JSON rule reverted	10

Leaving out the update of what each file contributed when a load is skipped as unchanged changes nothing: the skip happens only when every file's presence and content equal those of the data in effect.

• The file loading, file data source and override source tests (280) passed 10 runs in a row. An earlier loop failed once in FileDataWatcherTest.DirectoryRemovedWithoutNotificationsIsReportedOnce, a watcher test that this change does not touch; that was a race in the watcher's periodic check, fixed in fix: Report changes to the files of directories that have no watch #396. The full server SDK suite passes on net8.0 (2031 tests), and the SDK builds for netstandard2.0, net462 and net8.0.

Relation to #384

#384 fixes flakes in the reloader tests by reading their content through a scripted reader, so that no read sees a half-written file. That hides the case these tests need to handle. This change fixes the engine instead, and the reloader tests keep real, deliberately slow, in-place writes. #384 is no longer needed.

_{Opened from termag}

---

Note

Overview
Fixes a gap where a reload that read a file during an in-place rewrite (truncate then write) could apply an empty document, briefly clearing flags or overrides until the next reload.

JSON parsing: Whitespace-only or zero-byte JSON no longer loads as an empty document; it throws and leaves the last good data in place. Registered parsers that return null for empty YAML still mean “no entries.” Clearing overrides intentionally uses {} or deleting the file.

FileDataReloader hold-back: When a file that previously contributed content is read with no entries, the load keeps current data, schedules a variable retry (based on how long the file has stayed empty, capped at five retry delays), and does not consume failure attempt budget. Loads read every configured path even after an earlier path fails, reporting the first failure. Debounce timing uses an injectable monotonic clock for tests.

Docs on FileDataSource, file overrides, and YAML Parser describe in-place rewrites, empty-file semantics, and that atomic replace is still recommended for YAML partial reads.

Tests add InPlaceWrites.Rewrite, broad reloader hold-back coverage, and end-to-end checks that clients and override sources never serve an empty layer mid-rewrite (JSON/YAML, watch/poll).

^{Reviewed by Cursor Bugbot for commit 0e6c6dd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0e6c6dd. Configure here.}