Conversation
Hello delthas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
895e69e to
c6d7872
Compare
`_isObjectCold`, `_isObjectRestoring` and `_isObjectRestored` compared the archive timestamps directly with `Date.now()`, which returns a Number. A relational operator coerces both sides with ToNumeric, so a `Date` resolves to its epoch milliseconds and compares correctly while an ISO string resolves to `NaN` and every comparison is false. Arsenal types `restoreRequestedAt`, `restoreCompletedAt` and `restoreWillExpireAt` as `Date | string` and its setters validate the value but store it unchanged, so a caller that round-trips the metadata through JSON writes strings into MongoDB. With string timestamps only the never-restored case survived, because `!restoreRequestedAt` short-circuits before any comparison: every object which had ever had a restore requested fell through all three predicates and was accumulated into the plain masterCount/masterData counters as an ordinary hot object. Its bytes were also left out of the destination location while restoring, and out of the cold location once restored. Wrap each timestamp in `new Date()` before comparing, which is what cloudserver and backbeat already do at every equivalent site. Behaviour is unchanged for `Date` values and for objects with no `archive`. The existing tests built their fixtures exclusively with `new Date(...)`, so they never exercised the string form; they are now run over both. Issue: S3UTILS-246
c6d7872 to
e49d7c6
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## development/1.17 #408 +/- ##
====================================================
+ Coverage 45.28% 45.30% +0.02%
====================================================
Files 88 88
Lines 6486 6489 +3
Branches 1360 1363 +3
====================================================
+ Hits 2937 2940 +3
Misses 3503 3503
Partials 46 46 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Requested @SylvainSenechal in place of Edouard Comtet, who is currently away. |
SylvainSenechal
left a comment
There was a problem hiding this comment.
I think you did a few fix recently on s3utils, you may want to do a release now i think we are abiout to do a zenko release
|
/approve |
In the queueThe changeset has received all authorizations and has been added to the The changeset will be merged in:
The following branches will NOT be impacted:
This pull request does not target the following hotfix branch(es) so they
There is no action required on your side. You will be notified here once IMPORTANT Please do not attempt to modify this pull request.
If you need this pull request to be removed from the queue, please contact a The following options are set: approve |
Queue build failedThe corresponding build for the queue failed:
Remove the pull request from the queue
|
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
Please check the status of the associated issue S3UTILS-246. Goodbye delthas. |
|
!release |
|
!done 3h |
|
|
Restore dates are typed
Date | string, so, like cloudserver and backbeat do, we should properly handle string dates, by wrapping the var withnew Date()._isObjectCold,_isObjectRestoringand_isObjectRestoredcompare the archive timestamps directly withDate.now(), which returns a Number. A relational operator coerces both sides withToNumeric:Date <= Number→ epoch milliseconds → compares correctly"2025-01-05T11:42:42.771Z" <= Number→NaN→ always falseArsenal types all three fields
Date | string(lib/models/ObjectMDArchive.ts:9-15) and its setters validate the value but store it unchanged, so a caller that round-trips the metadata through JSON writes ISO strings into MongoDB.Behaviour, measured against the real predicates
Only the never-restored case survives, because
!restoreRequestedAtshort-circuits before any comparison. Every object that has ever had a restore requested is affected.Impact
:274-284uses the flags to pick a counter suffix; all three false means no suffix, so the object is accumulated into plainmasterCount/masterData, indistinguishable from a hot object. Also skipped: the destination-location attribution at:543while restoring, and the cold-location attribution at:554once restored.Fix
Wrap each timestamp in
new Date()before comparing. This is what every equivalent site elsewhere already does — s3utils was the only place comparing the raw value:lib/api/apiUtils/object/coldStorage.js:83new Date(objectMD.archive?.restoreWillExpireAt) < new Date(Date.now())lib/api/objectPut.js:277Date.now() - new Date(objMD.archive.restoreRequestedAt)extensions/lifecycle/tasks/LifecycleRetriggerRestoreTask.js:40new Date(archive.restoreWillExpireAt) < new Date()extensions/lifecycle/LifecycleQueuePopulator.js:477new Date(md.archive.restoreWillExpireAt) < new Date()Inlined at each of the five comparisons rather than behind a helper, since no repository defines one and
new Date(x)at the call site is the established form.new Date(x) <= Date.now()andnew Date(x).getTime() <= Date.now()were checked to agree forDate, ISO string,undefined,nulland unparseable input.The truthiness guards (
!restoreRequestedAt,restoreCompletedAt &&) are untouched, which is what keeps behaviour identical forDatevalues and for objects with noarchive.Scope
Fixed in the consumer rather than in Arsenal, deliberately. The root enabler is that
ObjectMDArchive's setters validate that a timestamp parses but store the caller's type as-is, so normalising toDatethere would address the class everywhere — but that is a wider behavioural change across consumers which have not been surveyed, and cloudserver and backbeat already coerce defensively at their own call sites. Handling it here keeps the change small and matches the convention already in use across the codebase.Tests
The existing fixtures were built exclusively with
new Date(...), which is why this shipped. Now covered at both layers:Dateand ISO-string forms viadescribe.each, matching thetest.eachidiom already used intests/unit/CountItems/, plus a new expired-restore case_processEntryDatatable, which computes the flags through the real predicates and asserts location attributionAll five fail on the parent commit, with exactly the predicted consequences — the restoring row loses
us-east-1(the:543attribution) and the restored row losescold-location(the:554one):The four
Datevariants pass both before and after, confirming the fix changes nothing on that path.Affected range
Introduced by
0f26e06(S3UTILS-155, 2024-03-25), first released in 1.14.6. Unchanged through 1.19.1.Verification
yarn test:unit: 32 suites, 460 tests, all passing.eslintclean.Issue: S3UTILS-246