Skip to content

Do not freeze an unresolved type variable default into an instance - #21873

Open
luantaraschi wants to merge 1 commit into
python:masterfrom
luantaraschi:fix/typevar-default-placeholder-import-cycle
Open

Do not freeze an unresolved type variable default into an instance#21873
luantaraschi wants to merge 1 commit into
python:masterfrom
luantaraschi:fix/typevar-default-placeholder-import-cycle

Conversation

@luantaraschi

Copy link
Copy Markdown

Fixes #21741.

AssertionError: Must not defer during final iteration
  File "mypy/semanal.py", line 1036, in analyze_func_def
    self.defer(defn)

Bisected to 47ca4c2 (#21491). Before it, a class whose type variable default held a placeholder was marked incomplete and its body was not analysed; now it defers and carries on, so a leftover placeholder reaches the method signatures.

Where the placeholder comes from

fix_instance() fills a missing type argument from tv.default without checking whether that default is ready:

if tv.has_default():
    arg = tv.default

During a pass where the default is still a forward reference, process_typevar_parameters() has it as PlaceholderType(None, ...), and that is what gets substituted. In the reproducer, ParamMeasT = ParameterBase | Callable[[], None] names the bare generic, so the alias target becomes ParameterBase[<placeholder None>].

That freezes. A placeholder at the top of an alias target marks the alias incomplete, but a nested one deliberately does not, so that recursive aliases like A = Sequence[str | A] stay legal:

incomplete_target = isinstance(res, ProperType) and isinstance(res, PlaceholderType)

So the alias is committed with the placeholder inside it, no further pass is asked for, and nothing ever resolves it: visit_placeholder_type() gives up on a placeholder with no fullname. From there it travels into the TypedDict item that names the alias, and then into the signature of the method that returns that TypedDict, where analyze_func_def() sees has_placeholder(result) on the final iteration and calls defer().

The reported Cannot resolve TypedDict item (possible cyclic definition) is the same placeholder arriving one step earlier. It is not a real cycle in the user's code.

The change

fix_instance() takes the analyzer API, and when the default it is about to substitute still has a placeholder it either asks for another pass or, on the final iteration, falls back to Any. The second half matters as much as the first: defer() documents that it must not be called once there is no pass left, and this is the one place that knows the default will never arrive.

Output for the reproducer now matches 2.1.0 exactly, both the crash and the spurious TypedDict error being gone:

m_param.py:24: error: Missing return statement  [empty-body]
m_param.py:28: error: Missing return statement  [empty-body]
m_instrument.py:12: error: Missing return statement  [empty-body]
Found 3 errors in 2 files (checked 4 source files)

The third caller of fix_instance(), the one in semanal.py that repairs an alias target, is left as it was: it has no test pulling on it here and I did not want to widen the change without one.

Tests

testTypeVarDefaultPlaceholderInImportCycleTypedDict in check-typevar-defaults.test, a four-module cycle reduced from the reporter's repository. On master it is an INTERNAL ERROR. It also pins the answer rather than just the absence of a crash: the default resolves to c.Inst and the TypedDict item comes out as tuple[a.Base[c.Inst] | (def ()), ...].

Ran locally on Windows: testcheck 8226 passed, plus testsemanal, testtransform, testdeps, testmerge, testfinegrained and testtypegen, 1944 passed. Self check clean.

@github-actions

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[2.2 regression] Internal mypy error related to circular imports and TypedDict

1 participant