Repository navigation
Conversation
FunctionDefinition located a prototype's NOP stub with list.index, which compares NCSInstruction by value. Every stub is equal, so with two or more prototypes each body was inserted after the first stub and calls looped. Look the stub up by identity instead, and add byte-level tests for two prototypes in declaration order, reverse order, interleaved, with an unprototyped function between, and recursion through two prototypes.
Contributor
|
Closing this for now, Please see: #354 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
A script with two or more function prototypes compiles to code whose calls never return. For example:
should return 32. The compiled script instead runs into the wrong bodies and, in the VM I tested it on, aborts with a call-depth overflow (the same shape with recursion,
Fact/Fib, loops until the call stack is exhausted). A file with a single prototype compiles correctly, so the bug shows up only when a file has at least two prototypes. That is common in game scripts: 21 of the 1,774 vanilla K1.nsssources declare at least two prototypes in the file itself.Cause
FunctionForwardDeclaration.compileemits aNOPas a placeholder (the stub) for each prototype. When the matchingFunctionDefinitionis compiled later,_compile_functionfinds the stub withand inserts the body after it.
list.indexcompares with==, andNCSInstruction.__eq__compares type, arguments and jump target by value. All stubs areNOPs with no arguments and no jump, so they are all equal, andindexreturns the first stub in the file. The second prototype's body is therefore inserted after the first prototype's stub, and the calls to the second function jump to its own (empty) stub, which falls through into whatever follows.The fix
Look the stub up by identity:
One site, one statement. With one prototype the result is unchanged; with several, each body lands at its own stub, in prototype order, whatever order the definitions appear in.
Tests
Five byte-level regression tests in
TestNCSCompiler, in a new "Multiple Forward Declaration Tests" region after the other function tests. Each comparesbytes_ncs(compile_nss(...))with an NCS file assembled in the test from the instruction layout (a small label-resolving assembler in the test class), so they do not useInterpreterand do not depend on the compiler's own output:test_two_prototypes_defined_in_declaration_ordertest_two_prototypes_defined_in_reverse_ordertest_two_prototypes_interleaved_with_definitions(prototype, definition, prototype, definition)test_two_prototypes_with_unprototyped_function_between(a third function with no prototype defined between the other two definitions)test_recursion_through_two_prototypes(two functions calling each other)All five fail on
masterand pass with this change.I also compiled seven small scripts and ran them on an independent NCS VM:
Fact/Fibwith both prototypes first (callingFact), the same with each prototype directly before its definition, one prototype with two definitions, both prototypes first (callingFib), mutual recursion with one prototype, and theA/Bexample above in both definition orders. With this change they return 120, 55120, 2120, 55, 11, 32 and 32, as expected. Onmasterfive of them abort with a call-depth overflow; the two scripts with a single prototype already gave the right value.Test results
Scoped command from
CONTRIBUTING.md(Windows, Python 3.12,Libraries/PyKotor/tests,K1_PATHunset), compared test by test from the JUnit XML:master(ca61ce8)The only difference is the five new tests. The same five tests in
tests/common/test_decode_fallbacks.pyfail on both (test_8bit_encoding_only,test_bom,test_no_valid_encoding_found_strict_errors,test_strict_error_handling_decoding_failure,test_with_BOM_included); they are unrelated to this change. (The skip count is higher than on a full checkout because the native_gl_accelextension was not built in my worktree.)ruff format --checkis clean for both touched files, andruff checkreports no finding on the changed lines.Other notes
bleeding-edgehas the same line (classes.py,ncs.instructions.index(root.function_map[name].instruction)), so it needs the same one-line change there.masterwith scripts that make no engine calls.