[tomdesair]

Summary

This pull request refactors cluster-safe upload deduplication across all supported storage backends, enforces a mandatory locking service architecture with a Null Object fallback, consolidates upload lock cleanup, and standardizes multi-backend integration testing.

1. Upload Checksum Deduplication Architecture

• Orchestration: Introduced DeduplicationService to manage thread-safe and cluster-safe deduplication lifecycles, CAS race resolution, missing parent detection, and reference-counted cleanups.
• Backend Checksum Indices:
  • DiskChecksumIndex: Filesystem-based index storing parent references and child links under <storagePath>/checksums/<algo>/<checksum>.
  • S3ChecksumIndex: S3 object-based index leveraging conditional writes (If-None-Match: *) with automatic fallback to unconditional put-object, parent tombstoning, and child reference tracking.
  • AzureBlobChecksumIndex: Azure Blob-based index using conditional blob uploads (If-None-Match: *) and blob prefixes for child trackers.
• Dual-Protocol Support: Seamlessly deduplicates completed uploads created via either the Tus 1.0.0 protocol (Upload-Checksum) or the IETF Resumable Uploads for HTTP (RUFH) specification (Digest / Content-Digest).
• Lifecycle & Read/Write Safety:
  • Supports multi-child linking: Child uploads link to parent data files/objects without duplicating physical storage.
  • Parent tombstoning: Deleting a parent upload while children are active marks the parent tombstoned without breaking child access; final child deletion completely purges the parent data and checksum index.
  • Read/Write Isolation: Read operations dynamically resolve to parent files/objects, whereas write or truncation operations strictly target child-owned storage to prevent parent corruption.
• Documentation: Added comprehensive documentation in docs/DEDUPLICATION.md and updated README.md, docs/ARCHITECTURE.md, docs/CONFIGURATION.md, docs/S3_STORAGE.md, and docs/AZURE_BLOB_STORAGE.md.

2. Mandatory UploadLockingService & Null Object Pattern

• Enforced UploadLockingService as mandatory across all storage backends (DiskStorageService, S3StorageService, AzureBlobStorageService, and ThreadLocalCachedStorageAndLockingService).
• Introduced NoOpUploadLockingService implementing UploadLockingService (Null Object Pattern) for unit testing and custom embedding environments where locking is handled externally.
• Added centralized, null-safe helper methods Utils.getLockingServiceOrDefault(...) to eliminate redundant null checks across request handlers and storage services.

3. Storage Service Lifecycle & Lock Cleanup Consolidation

• Added getUploadLockingService() to UploadStorageService interface and its implementations.
• Deprecated cleanupExpiredUploads(UploadLockingService) in favor of zero-argument cleanupExpiredUploads().
• Consolidated lock cleanup into terminateUpload(UploadInfo) across all storage backends to ensure locks are cleanly and unconditionally removed upon upload termination.
• Maintained serialization backward compatibility on UploadInfo (serialVersionUID = -8751200491586638308L).
• Documented in AGENTS.md and architecture docs that UploadLock implements AutoCloseable and should always be managed with try-with-resources.

[coveralls]

coverage: 94.4% (-0.3%) from 94.73% — feature/checksum-deduplication-cleanup into master