[kinyoklion]

Summary

The server SDK's contract test service had no persistent store support: the data system store configuration in SdkClientEntity.cs was commented out with a TODO about the store packages, and the FDv1 persistentDataStore configuration was not handled. CI therefore ran both harness versions with enable_persistence_tests: 'false'. This change makes the service drive real Redis, DynamoDB, and Consul stores and runs the persistence suites in CI.

It is separate from the persistence fix in #389 (and #390 on dotnet-server-sdk-v9), which these tests do not cover yet; the harness gains tests for that behavior in a separate change.

Test service

• The FDv2 dataSystem.store and storeMode configuration (harness v3) and the FDv1 top-level persistentDataStore configuration (harness v2) build the store with Components.PersistentDataStore, with the requested prefix and cache mode.
• An FDv1 configuration with a store and no streaming, polling, or service endpoint configuration runs in daemon mode (Components.ExternalUpdatesOnly), which the v2 suite expects. Without this, the client tried to reach LaunchDarkly and failed to initialize.
• An empty prefix means the store's default prefix, and a store configuration without a DSN is rejected.
• The Redis and DynamoDB clients fail fast (500 ms timeouts, no DynamoDB retries), as the store recovery tests require. The Consul client does not retry.
• The service advertises persistent-data-store-redis, persistent-data-store-dynamodb, persistent-data-store-consul, and persistent-data-store-recovery.
• The service takes its port as an optional first argument, as on dotnet-server-sdk-v9. Its default stays 8000.

Building against the local SDK

The store projects in pkgs/ reference the published LaunchDarkly.ServerSdk package, so a reference to them would bring a second copy of the SDK into the service. Instead, a small project per store under contract-tests/Stores/ compiles that store's sources against the SDK project, with the same client library and version range as the store's own project. There is one project per store because the Redis and DynamoDB stores share internal type names. The store projects themselves are not changed, and neither are their packages or their builds.

CI

• .github/actions/contract-tests gains enable_persistence_tests (default false) and test_service_port (default 8000). With persistence enabled, it starts the stores with launchdarkly/gh-actions/actions/persistent-stores, waits until they accept connections and Consul has a leader, and enables the tests in both harness runs.
• The server SDK workflow enables the persistence tests on the Linux runner only, where the stores start in Docker, and passes port 9000 on every runner: the harness expects DynamoDB on port 8000, the service's default. The client SDK workflow uses the same action with the default port, because its test service always listens on 8000.
• The nightly enables them too, also with port 9000.

Suppressed tests

The persistence suites found three pre-existing defects in the store integrations. They are suppressed with a comment in each suppression file. The harness reads every non-blank line as a test ID, so a line that starts with # matches no test.

1. A deleted flag is stored without a "key" property: {"version":N,"deleted":true}, from the ItemDescriptor serializer in DataStoreTypes.cs (Redis and Consul store that string), or as the placeholder "null" in the item attribute (DynamoDBDataStoreImpl.MarshalItem). The harness expects a JSON tombstone with a key property, the item key or $deleted. This fails read-write/data source deletions respect versioning for each store (both harness versions) and, in v3, the five store recovery tests that check a deleted flag after the write back. With a one-line serializer change that writes "key": "$deleted", the Redis and Consul tests pass, and the recovery write back itself works; DynamoDB needs its own fix.
2. ConsulDataStoreImpl.GetAllAsync iterates result.Response, which is null when no key has the collection's prefix, so reading an empty collection throws. This fails collection cardinality/collection holds no flags for Consul.

These need SDK and store changes, which this PR does not make. For the keyless tombstone, the v3 harness's own tombstone suite lists {"version":N,"deleted":true} as a shape that .NET, Java, and Node write and that readers must accept, so for Redis and Consul the fix may belong in the harness's read-write matcher instead. DynamoDB's "null" placeholder is not one of the accepted shapes. Until one of these lands, CI does not check the store write back after recovery or the store side of deletions for any store; the only recovery test that runs checks evaluations from memory during the outage.

Testing

With Redis, DynamoDB local, and Consul running in Docker on their default ports:

• Harness v2 (2.41.1), full suite with persistence enabled and the suppression file: all tests passed (4989 ran).
• Harness v3 (3.2.0-alpha.15, from v3 at 58f4122), full suite with persistence enabled and the FDv2 suppression file: all tests passed (5095 ran).
• After moving the port to the first argument: both persistence suites with the service on port 9000 passed.

Without the suppressions, the persistence suites fail exactly the tests listed above (4 in v2, 19 in v3).

After the review changes (the store projects, the port default, the readiness wait), both persistence suites pass again with the suppressions (v3: 141 ran; v2: 111 ran), and LaunchDarkly.ServerSdk.sln builds with no errors.

#392 makes the same change on dotnet-server-sdk-v9.

_{Opened from termag}

---

Note

Overview
Enables persistent store contract tests in CI by wiring the server SDK contract test service to real Redis, DynamoDB, and Consul, and turning those suites on in GitHub Actions.

The contract test service now builds Redis, DynamoDB, and Consul integrations from in-repo sources (via Stores/* wrapper projects) against the local SDK, implements CreatePersistentDataStore for harness FDv1 persistentDataStore and FDv2 dataSystem.store, supports daemon mode when a store is configured without streaming/polling, advertises the new store capabilities, and accepts an optional listen port (9000 in CI so DynamoDB can use 8000). Store clients use short timeouts for recovery tests.

The contract-tests composite action gains enable_persistence_tests and test_service_port, starts stores and waits for readiness when enabled, and forwards those flags to both harness runs. sdk-server-ci runs persistence tests on Linux only; nightly always enables them with port 9000.

Known failures from pre-existing tombstone / empty-collection bugs in the store integrations are listed in test-supressions*.txt so CI stays green until separate fixes land.

^{Reviewed by Cursor Bugbot for commit fd9c0d6. Bugbot is set up for automated code reviews on this repo. Configure here.}

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

bugbot review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 649f89f. Configure here.}