Repository navigation
Conversation
A stale group share is a share ID in the group cache that has no share in the provider cache. ListReceivedShares found the stale ID on each listing. It removed the ID from received.json, but the ID is not in that file. So each listing uploaded received.json again with the same content, and the stale ID stayed forever. Read side: - receivedsharecache and sharecache: Remove does not upload when the share is not in the cache. When the share is not in memory, Remove syncs first and checks again. - sharecache: Remove does not panic when the user or group has no entry for the space of the share. - ListReceivedShares records which groups give each share ID. Before the cleanup, it reads the share again from storage. When the share is still missing, it removes the ID from the user received states and from each group that gave the ID. Write side: - Share writes the provider cache and created.json first. It adds the grantee entry only after both writes succeed. - removeShare removes the grantee entry first. When that fails, Unshare returns the error and the share stays, so the caller can try again. - The grantee step runs alone, so a failed sibling write cannot cancel it. Space delete: - ProcessEvents puts its logger into the context. Before, purgeSpace logged to a disabled logger. - purgeSpace returns all errors with the share ID, and ProcessEvents logs them. - In a purge, removeShare continues after a grantee error and removes the created.json entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
… share fix - ListReceivedShares cleans up stale entries only when it read the space file. A NotFound for the space file can be temporary. Before, the cleanup then removed valid group shares for all members of the group. - Remove the second provider read before the cleanup. It used the same sync path, so it gave no protection. - Unshare removes the share also when the grantee entry cannot be removed. The gateway removes the storage grant in this case too. Before, a broken grantee file blocked the unshare and left a share without a grant. - Share removes the entries that it wrote when a later write fails. Before, the share stayed, and a new Share failed with AlreadyExists. - The writes in Share and removeShare do not cancel each other. - In a purge, the grantee removal and the created.json removal run at the same time again. - The logger in the ProcessEvents context writes only warnings and errors, because it does not use the configured log level. - Fix the Unshare test. It checked nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Send a SpaceDeleted event to ProcessEvents, as in production, and check the result of the purge goroutine. - A normal delete removes the group shares. The listings after the delete do no uploads. - When the group file cannot be written during the purge, the group entry stays. The first listing of a member removes it with one upload. The listings after that do no uploads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…of shares A loop that removes many shares uploaded each touched document once for each share. Each upload sends the whole document. A purge of 20 group shares did 41 uploads, and a listing that finds 20 expired shares did 60. These tests fail now: - providercache, sharecache, receivedsharecache: RemoveMany removes many share IDs with one upload, and does no upload when no ID is present. - purgeSpace, listCreatedShares, listSharesByIDs and ListReceivedShares upload each touched document once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r document A loop that removed many shares called removeShare for each share. Each removal uploaded every touched document again, with the full content. A purge of 20 group shares did 41 uploads, and a listing that found 20 expired shares did 60 uploads. The bytes grew with the square of the share count, and each upload held the lock of the space or the user. - providercache, sharecache and receivedsharecache get RemoveMany. It removes many IDs with one upload, and does no upload when no ID is present. Remove calls RemoveMany with one ID. - providercache.Cache.Remove now checks the result of Spaces.Load and does not upload when the share is not in the space. - removeShares groups the shares per document and uploads each document once. removeShare calls it with one share. - purgeSpace removes all shares of the space with one removeShares call. - listSharesByIDs, listCreatedShares and ListReceivedShares collect the expired shares of each space and remove them together. One helper, expireShares, replaces the five copies of the expiry code. The event now always has the ShareID. - The stale cleanup in ListReceivedShares removes the stale IDs of a space with one upload per user or group document. A purge of 20 group shares now does 3 uploads, and a listing that finds 20 expired shares also does 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…che Remove panic sharecache.Cache.Remove read us.UserShares[ssid].IDs with no nil check. When the user had no entry for the space of the share, the entry was nil and Remove panicked. The panic ran in an errgroup goroutine, so it stopped the process. Production saw it in CreatedCache.Remove during the cleanup of orphaned shares, and the same path runs in each unshare. The nil guard is already in this pull request. Add the two cases from the report: - a share in space-a, then a Remove of a share in space-b, on a warm cache - a share ID with no delimiter, so the space key is ":" On the sharecache.go of stable-8.0, both tests and the existing nil test fail with a panic. Correct the changelog: the panic was in the created cache too, not only in the group cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e manager Bump reva to v2.0.0-20261112224757-ebbb52833e5f, the head of owncloud/reva#813. A stale group share is a share ID in the group cache that has no share. On each listing of the received shares, the jsoncs3 share manager uploaded the received.json file of the user again with the same content, and the stale ID stayed. The new reva: - removes stale group shares from the group cache, only when it read the share file of the space - does not upload a file when a Remove changes nothing - writes a share before the grantee entry, and rolls back after a failure - removes the share in an unshare also when the grantee entry fails - removes many shares with one upload for each document - logs the share cleanup errors after a space delete The bump also brings owncloud/reva#796 into stable-8.0. The previous reva pin did not have it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…he workers stop The listing workers in listCreatedShares and ListReceivedShares send the expired shares to a channel. The main goroutine reads this channel and the results channel in one select loop. After all workers stop, the manager calls expireShares one time with the parent context. listSharesByIDs collects the expired shares of all spaces and calls expireShares one time after the loop. removeShares groups the shares by document across spaces. Thus expired shares of one creator in different spaces now cause one upload of created.json. The errgroup context is now gctx. errgroup cancels this context when Wait returns, so the expiration uses the parent context. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and the stale cleanup The stale cleanup in ListReceivedShares took the etag from the ListSpace snapshot, but it read each share from the live provider cache. A parallel listing that got a temporary NotFound reset the live cache between the two reads. Then all shares looked stale, and the cleanup removed valid shares from the group cache. The cleanup now reads each share from the snapshot. The rollback of a failed Share used the request context. A cancelled request is a common cause of the failure, and then each rollback write failed at once. The share stayed, and a new Share failed with AlreadyExists. The rollback now uses a context without the cancellation, with a timeout. A failed grantee write keeps the share ID in memory. The rollback did not remove it, so the next write of the grantee document uploaded the ID without a share behind it. The rollback now removes the grantee entry first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…share cleanup An older version writes the grantee entry and the share at the same time. During a rolling upgrade, a listing on a new pod can find the grantee entry before the share is on storage. The listing removed this entry at once, and no group member saw the share. A listing now notes the time when this pod first finds a stale share. It removes the entry only on a listing after StaleShareGracePeriod. The default is 5 minutes. Zero removes the entry at the first listing. The rule applies to user entries and to group entries. The tests of the cleanup set the grace period to zero. The changelog describes the fixes of the last commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stale share fix Bump reva to v2.0.0-20261116153307-8f53b9a6148f, the head of owncloud/reva#813. The new reva commits change only the jsoncs3 share manager: - The stale cleanup reads each share from the same copy of the space file. A parallel listing can no longer make valid shares look stale. - The cleanup removes a stale entry only after a grace period of 5 minutes. During a rolling upgrade, an older version can write the grantee entry before the share. - The rollback of a failed share also removes the grantee entry, and it runs also when the request was cancelled. - A listing removes the expired shares of all spaces together. The changelog describes these fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e manager Bump reva to v2.0.0-20261112224757-ebbb52833e5f, the head of owncloud/reva#813. A stale group share is a share ID in the group cache that has no share. On each listing of the received shares, the jsoncs3 share manager uploaded the received.json file of the user again with the same content, and the stale ID stayed. The new reva: - removes stale group shares from the group cache, only when it read the share file of the space - does not upload a file when a Remove changes nothing - writes a share before the grantee entry, and rolls back after a failure - removes the share in an unshare also when the grantee entry fails - removes many shares with one upload for each document - logs the share cleanup errors after a space delete The bump also brings owncloud/reva#796 into stable-8.0. The previous reva pin did not have it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…stale share fix Bump reva to v2.0.0-20261116153307-8f53b9a6148f, the head of owncloud/reva#813. The new reva commits change only the jsoncs3 share manager: - The stale cleanup reads each share from the same copy of the space file. A parallel listing can no longer make valid shares look stale. - The cleanup removes a stale entry only after a grace period of 5 minutes. During a rolling upgrade, an older version can write the grantee entry before the share. - The rollback of a failed share also removes the grantee entry, and it runs also when the request was cancelled. - A listing removes the expired shares of all spaces together. The changelog describes these fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…timeout and the stale cleanup - removeShares returns an error only when it cannot remove the shares. When the shares are gone, a retry gets NotFound. The next listing removes a stale grantee entry, and ListShares skips a stale created entry. - Each rollback removal in Share gets its own timeout. A slow removal does not use the time of the next one. - When the download of the space file gets NotFound, the listing runs a Stat. A second NotFound proves that the file is missing, and the stale entry is removed after the grace period. - A listing that finds a share again restarts its grace period. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hare cache sharecache RemoveMany kept the space entry after it removed the last share of the space. Each listing of a group member then read the space file of a purged space again. RemoveMany now deletes the empty space entry, the same as receivedsharecache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tale group entries The stale cleanup sent the stale IDs of a group also to the received.json of the user. These IDs are usually not in that file, so the user cache downloaded the file again only to find nothing to remove. The cleanup now sends only the IDs that are in received.json of the user. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
removeShares started one goroutine for each grantee and creator document, with no limit. A purge of a space with many grantees then sent hundreds of parallel writes to the metadata storage. Now MaxConcurrency limits the parallel removals. When MaxConcurrency is not set, the limit is 10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ch expired share Each listing that found an expired share published a ShareExpired event, also when another listing removed the share first or when the removal failed. Each event can send a notification. providercache RemoveMany now returns the IDs that its upload removed. The CAS upload makes sure that only one caller removes a share, also across pods. expireShares publishes the event only for these shares. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gofmt reported four files of the jsoncs3 share manager. Sort the imports of jsoncs3.go, split a long closure in providercache.go and remove an empty line in providercache_test.go. Move a comment in batchremove_internal_test.go to its own line, so that gofmt does not align it with the next line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…moval upload RemoveMany in the provider cache, the created and group cache and the user cache deletes the shares from memory before the upload. When the upload failed with an error that is not a CAS conflict, the memory state was different from the file. The etag did not change, so a sync got NotModified and kept that state. A retry then found nothing to remove, did not upload and reported success. The share stayed in the file, and an expired share got no ShareExpired event. In the user cache, this also occurred inside one RemoveMany after a transient upload error, because the resync before the next attempt got NotModified. Now a failed upload that is not a CAS conflict drops the cached document from memory, so the next call reads the file again. The entries are not put back: a failed Add leaves an extra entry in memory, and the rollback of Share relies on Remove to delete it. The new test helper NotModifiedStorage returns NotModified for a matching etag, like the CS3 metadata storage. metadata.Disk ignores IfNoneMatch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
ListReceivedShares added a shareid field to the space logger for each share that it found, and listSharesByIDs added the storageid, spaceid and opaqueid fields to the listing logger for each share. A later log line then had the fields of all earlier shares. Log parsers usually keep the first key, so a log search found the wrong share. The logger also got bigger with each share. ListReceivedShares now keeps the space logger unchanged, because each log call adds its own shareid. listSharesByIDs now makes a new logger for each share. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…e manager Bump reva to v2.0.0-20261112224757-ebbb52833e5f, the head of owncloud/reva#813. A stale group share is a share ID in the group cache that has no share. On each listing of the received shares, the jsoncs3 share manager uploaded the received.json file of the user again with the same content, and the stale ID stayed. The new reva: - removes stale group shares from the group cache, only when it read the share file of the space - does not upload a file when a Remove changes nothing - writes a share before the grantee entry, and rolls back after a failure - removes the share in an unshare also when the grantee entry fails - removes many shares with one upload for each document - logs the share cleanup errors after a space delete The bump also brings owncloud/reva#796 into stable-8.0. The previous reva pin did not have it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…stale share fix Bump reva to v2.0.0-20261116153307-8f53b9a6148f, the head of owncloud/reva#813. The new reva commits change only the jsoncs3 share manager: - The stale cleanup reads each share from the same copy of the space file. A parallel listing can no longer make valid shares look stale. - The cleanup removes a stale entry only after a grace period of 5 minutes. During a rolling upgrade, an older version can write the grantee entry before the share. - The rollback of a failed share also removes the grantee entry, and it runs also when the request was cancelled. - A listing removes the expired shares of all spaces together. The changelog describes these fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…g fixes of the stale share cleanup Bump reva to deffdab6d, the head of owncloud/reva#813. The new commits change only the jsoncs3 share manager: - A failed removal upload drops the cached document, so a retry reads the file again and removes the share. - Each log line of a share listing has the fields of one share only. - gofmt formats the jsoncs3 files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…d upload Add in the provider cache, the created and group cache and the user cache puts the entry in memory before the upload. When the upload failed with an error that is not a CAS conflict, the entry stayed in memory. The etag did not change, so a sync got NotModified and kept it. The pod then listed a share or a share state that was not in the file, and the next write for the same document uploaded it. For example, a client got an error when it accepted a share, but this pod showed the share as accepted. Each cache now has dropOnFailedUpload, which RemoveMany and Add use. It drops the cached document after a failed upload that is not a CAS conflict, so the next call reads the file again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…listing that found it staleGracePeriodOver dropped each entry that was older than ten grace periods (50 minutes), counted from the first time that the pod found the share. Each pod counts the grace period itself. With many pods, one pod can find the same stale share less often than every 50 minutes. When other new stale shares arrived in between, the pod dropped the entry and started the grace period again on each listing, so the stale entry stayed. Now each entry holds the first time, for the grace period, and the last time that a listing found the share. The pod forgets an entry only when no listing found it for 24 hours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…ntry at the same time The rollback of a failed Share removed the grantee entry, the share in the provider cache and the entry in the created cache one after the other. Each removal has its own timeout of 30 seconds. A slow metadata storage is a usual cause of the first failure, so a failed Share could block the request for about 90 seconds. The removal of the share and the removal of the created entry do not depend on each other. They now run at the same time, after the removal of the grantee entry. A failed Share now waits for at most two timeouts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…eriod and rollback fixes Bump reva to 6b741a384, the head of owncloud/reva#813. The new commits change only the jsoncs3 share manager: - A failed add upload drops the cached document, so the pod does not show an entry that is not in the file. - A pod remembers a stale share for 24 hours after the last listing that found it, so the grace period also ends on a pod that finds the share only rarely. - The rollback of a failed share removes the share and the created entry at the same time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…e share memory Bump reva to a31eb537b, the head of owncloud/reva#813. The new commit changes only the jsoncs3 share manager: - A pod forgets a stale share 1 hour after the last listing that found it. Before, it was 24 hours. - A pod looks for the stale shares to forget at most once a minute, so a listing that finds many new stale shares does not hold the lock for a long time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…e manager Bump reva to a31eb537b, the head of owncloud/reva#813. A stale group share is a share ID in the group cache that has no share. On each listing of the received shares, the jsoncs3 share manager uploaded the received.json file of the user again with the same content, and the stale ID stayed. The new reva: - removes stale group shares from the group cache, only when it read the space file or a Stat proved that the file is missing - removes a stale entry only after a grace period of 5 minutes, and forgets a stale entry 1 hour after the last listing that found it - does not upload a file when a Remove changes nothing, and reads the file again after a failed Add or Remove upload - does not panic in sharecache Remove when the space entry is missing - writes a share before the grantee entry, and rolls back after a failure - removes the share in an unshare also when the grantee entry fails - removes many shares with one upload for each document, with at most MaxConcurrency removals in parallel - publishes one ShareExpired event for each expired share - logs the share cleanup errors after a space delete The default of SHARING_USER_JSONCS3_MAX_CONCURRENCY changes from 1 to 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
…e updates This commit joins 5 commits. Stale shares: - A pod forgot a stale share 24 hours after the last listing that found it. It now forgets the share after 1 hour. - sweepStale ran a full sweep of staleSince for each new stale ID. It now runs at most once in each staleForgetAfter/60. - The listing forgets a stale share only after all removals of it succeed. A failed removal no longer starts a new grace period. Update under the space lock: - UpdateShare read the share and then released the space lock. A parallel Add or RemoveMany with a failed upload could drop the space from memory. The client got the changed share, but the file kept the old share. - The new Cache.Update reads the share under the space lock. It changes a copy and uploads the space with a CAS retry. It returns NotFound when a parallel unshare removed the share. A retry keeps the expiration. - listCreatedShares reads each share from the ListSpace snapshot and not from the live cache. - The listings collect the expired shares in a slice with a mutex. Cleanup: - CleanupStaleShares removes the orphaned shares in one removeShares call. 20 orphaned shares in one space now do 3 uploads, not 60. - The loops do not change the cache maps that they read. - After a Stat error or a gateway selector error, the cleanup logs the error. It does not stop with a nil pointer panic. Parallel removals: - removeShares used MaxConcurrency as the limit. The ocis default is 1, so the removals ran one after the other. The limit is now at least 3. A higher MaxConcurrency still works. The default for a value of 0 or less is still 10. Sync of the space: - RemoveMany and Update synced the space only when the space was not in memory. A listing with a temporary NotFound stored an empty space with no etag. RemoveMany then found no share, did not upload and reported success. The share stayed in the space file. - RemoveMany and Update now sync first when a share is not in memory. With a known etag, the sync gets NotModified. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
38d3324 to
c600f66
Compare
| delete(us.UserShares[ssid].IDs, shareID) | ||
| // An empty space entry makes each listing read the space file again. | ||
| if len(us.UserShares[ssid].IDs) == 0 { | ||
| delete(us.UserShares, ssid) |
There was a problem hiding this comment.
Question about rolling upgrades:
The old version has a bug, that it panics, when sharecache.Remove is called, but the space entry was already deleted. The new version fixes that bug, but now deletes these space entries more often. That means that when both versions run at the same time, these panics are a lot more likely. Do you think thats acceptable for rolling upgrades? Otherwise, we could release this delete call in a subsequent version.
There was a problem hiding this comment.
We can't fix it in the old binary. It is a slim chance of getting a panic during the rolling upgrades. Here it is safe because before it checked if c.has(..
|
Very nice!
|
PurgeSpace stored a space with a nil Shares map in memory. Add does not sync a space that is in memory. Thus an Add to the purged space on the same pod wrote to the nil map and panicked. PurgeSpace now stores an empty map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
444cdda
…e manager Bump reva to a31eb537b, the head of owncloud/reva#813. A stale group share is a share ID in the group cache that has no share. On each listing of the received shares, the jsoncs3 share manager uploaded the received.json file of the user again with the same content, and the stale ID stayed. The new reva: - removes stale group shares from the group cache, only when it read the space file or a Stat proved that the file is missing - removes a stale entry only after a grace period of 5 minutes, and forgets a stale entry 1 hour after the last listing that found it - does not upload a file when a Remove changes nothing, and reads the file again after a failed Add or Remove upload - does not panic in sharecache Remove when the space entry is missing - writes a share before the grantee entry, and rolls back after a failure - removes the share in an unshare also when the grantee entry fails - removes many shares with one upload for each document, with at most MaxConcurrency removals in parallel - publishes one ShareExpired event for each expired share - logs the share cleanup errors after a space delete The default of SHARING_USER_JSONCS3_MAX_CONCURRENCY changes from 1 to 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Roman Perekhod <2403905@gmail.com>
Description
The jsoncs3 share manager uploaded
received.jsonagain on each listing of the received shares when a user had a stale group share.A stale group share is a share ID in the group cache that has no share in the provider cache.
ListReceivedSharesfound the stale ID and removed it fromreceived.json.The ID is not in that file, so the removal changed nothing, but the file was uploaded anyway.
The group cache kept the ID, so the next listing did the same again.
In production, this causes a large volume of
share not found. cleaning upwarnings and one full document upload per stale share for each listing.Stale group shares come from partial write failures.
ShareandremoveSharewrote the provider cache,created.jsonand the grantee entry at the same time, with no rollback.A failed write also cancelled the shared context, so the group cache update sometimes did not run.
purgeSpaceignored its errors, and it logged them to a disabled logger.A loop that removed many shares, for example in a purge or in a listing that finds expired shares, uploaded each touched document once for each share.
sharecache.Cache.Removepanicked when the user or group had no entry for the space of the share, or when the share ID had no delimiter. The panic ran in an errgroup goroutine and stopped the process. Production saw it inCreatedCache.Removeduring the cleanup of orphaned shares, and the same path runs in each unshare.Changes
Read side:
receivedsharecache.Cache.Removeandsharecache.Cache.Removedo not upload when the share is not in the cache. When the share is not in memory, they sync first and check again.sharecache.Cache.Removedoes not panic when the user or group has no entry for the space of the share, or when the share ID has no delimiter.sharecacheRemoveManydeletes the space entry when it removes the last share of the space. Before, each listing of a group member read the space file of a purged space again.providercache.Cache.Removedoes not upload when the share is not in the space, and it checks the result ofSpaces.Load.ListReceivedSharesrecords which groups give each share ID. When a share is missing, it removes the ID from each group that gave the ID. It sends to the user received states only the IDs that are inreceived.jsonof the user.ListSpacesnapshot. Before, a parallel listing could reset the live cache between two reads, and then the cleanup removed valid shares.Grace period for the stale cleanup:
StaleShareGracePeriod. The default is 5 minutes. Zero removes the entry at the first listing. The rule applies to user entries and to group entries.Write side:
Sharewrites the provider cache andcreated.jsonfirst. It adds the grantee entry only after both writes succeed.Sharerolls back. It removes the grantee entry first, and then the share and thecreated.jsonentry at the same time. Thus a newSharedoes not fail with AlreadyExists, and no stale grantee entry stays.Sharewaits for at most two timeouts.removeShareremoves the grantee entry first. It removes the share also when that fails, because the gateway removes the storage grant in any case. The next listing of a grantee removes a stale grantee entry.removeSharesreturns an error only when it cannot remove the shares.Failed uploads:
AddandRemoveManyin all caches change memory before the upload. Before, a failed upload that is not a CAS conflict kept the changed memory. The etag did not change, so a sync got NotModified and kept that state.dropOnFailedUploaddrops the cached document after such a failure, so the next call reads the file again.Space delete:
ProcessEventsputs its logger into the context.purgeSpacereturns all errors with the share ID, andProcessEventslogs them.removeShareremoves the grantee entry and thecreated.jsonentry at the same time. A failure of one does not stop the other.ProcessEventscontext writes only warnings and errors, because it does not use the configured log level.Batch removal:
RemoveMany. It removes many IDs with one upload, and does no upload when no ID is present.RemovecallsRemoveManywith one ID.removeSharesgroups the shares for each document across spaces and uploads each document once.purgeSpacecalls it once for all shares of the space.removeSharesruns at mostMaxConcurrencyremovals in parallel. WhenMaxConcurrencyis not set, the limit is 10.listSharesByIDs,listCreatedSharesandListReceivedSharescollect the expired shares of all spaces. They remove them together after the listing workers stop. One helper,expireShares, replaces the five copies of the expiry code.The numbers come from the same test on
stable-8.0and on this branch. The 20 expired shares give the same numbers forListShares,ListShareswith a resource filter andListReceivedShares.ShareExpiredevent:providercacheRemoveManyreturns the IDs that its upload removed. The CAS upload makes sure that only one caller removes a share, also across pods.expireSharespublishesShareExpiredonly for these IDs. The event always has theShareID.stable-8.0sent 1–3 events. This branch sends exactly 1 event. A failed removal sends no event.Logs:
Tests:
pkg/share/manager/jsoncs3/jsoncs3_test.go,receivedsharecache/receivedsharecache_test.go,sharecache/sharecache_test.go,providercache/providercache_test.go,purgespace_internal_test.go,batchremove_internal_test.go,rollback_internal_test.go,stalegrace_internal_test.go,expire_internal_test.goandlistsharesbyids_internal_test.go.sharecache.Cache.Removepanic fail with a panic on thesharecache.goofstable-8.0.pkg/share/manager/jsoncs3/testhelpers/fakes.go:CountingUploadStorage,AlwaysAbortedUploadStorageandNotModifiedStorage.NotModifiedStoragereturns NotModified for a matching etag, like the CS3 metadata storage.metadata.DiskignoresIfNoneMatch.receivedsharecachetests are adapted: aRemoveof an unknown share does not upload now, so the bootstrap test usesAdd, and the two resync tests warm the cache first.Changelog:
changelog/unreleased/fix-jsoncs3-stale-group-shares.md.Checks
pkg/share/manager/jsoncs3/...packages pass with-race.go vetandgofmtreport nothing.go build ./...passes../pkg/share/..., the share provider services and the gateway pass.sharing,storage-usersandocispackages build against this branch.Notes for the reviewers
ShareandUnsharenow do two write steps one after the other. This adds one storage round trip of latency.removeSharenow removes the grantee entry before the share, an expired share stays visible a little longer to a parallel listing.StaleShareGracePeriodon each pod. Each pod counts the grace period itself, so the cleanup can need one more listing on each pod.purgeSpaceand the caches below it are now visible. Before, they went to a disabled logger.<storageid>$<spaceid>purges nothing. That is a different bug and is not part of this pull request.🤖 Generated with Claude Code