Repository navigation
fix(judge): let overlapping saves to one judgement take turns - #268
Merged
Merged
Conversation
store.save read the trial's file, merged the new answers and wrote it back through a shared `<file>.tmp`. Two saves in that window lost one answer, and one of them raised FileNotFoundError when the other renamed the shared temporary file away. A save now holds a lock per judgement file for its read, merge and write, and writes through a temporary file of its own. The hygiene readings merge inside the same lock instead of in a separate load in ask_and_store. Saves to different files still run in parallel.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #247
What
store.saveread a trial's judgement file, merged the new answers into it, and wrote it back through<file>.tmp. Two saves in that window lost one answer, and one of them raisedFileNotFoundErrorwhen the other renamed the shared temporary file away. Two batches over the same trials at once (Judge all clicked again, or in a second tab) could do this.tempfile.mkstempin the same folder) and removes it if the write fails.batch.ask_and_storeloaded the record and merged them before it calledsave, outside any lock.savereturns the record as stored, andask_and_storereturns the same record as before.Touches
backend (
nullius/judge/store.py,nullius/judge/batch.py) · testsVerified
app/tests/test_judge_store_race.py. Two saves read the file before either writes, and both answers must be kept. A second test checks that saves to two trials do not wait for each other. On the base, the first test fails:(["FileNotFoundError(2, 'No such file or directory')"], ['q2']) != ([], ['q1', 'q2']).python:3.12container with Node:python3 app/tests/parallel.py: 3111 tests in 160 modules, OK.pip install -r app/requirements.txt, thenpython3 app/tests/parallel.py app/ui_tests: 704 tests in 58 modules, OK.Risk
mkstemp's 0600) where it was 0644 under the usual umask. Workspaces live in the user data directory.