Skip to content

fix(sync): a Python import alias links its module when the module appears later - #2501

Merged
colbymchenry merged 1 commit into
mainfrom
claude/kind-heisenberg-fd6f7b
Oct 11, 2026
Merged

colbymchenry merged 1 commit into
mainfrom
claude/kind-heisenberg-fd6f7b

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Problem

codegraph sync (and the watcher) never relinked a reference through a Python import alias, a binding whose local name differs from the name its module declares, when the module was added after the importing file was indexed, came back after a delete, or gained the name in a later edit. A fresh index links it.

  • from .models import User as UserModel, from app.models import Group as GroupModel
  • from . import helpers as h (a submodule), from .pkg import sub as subpkg (a subpackage), from .. import helpers as hh
  • from settings import Config as Cfg (a new top-level module)

Every reference through the alias stayed failed: the import itself, a call (UserModel(), mk()), a base class, and the class or function passed as a value (isinstance(u, UserModel), return mk).

Repro (from #2450's residue list): app/__init__.py empty, app/main.py = from .models import User as UserModel / from . import helpers as h / def run(): return UserModel(), h; index, add app/models.py and app/helpers.py, sync. A fresh index has app/main.py -imports-> class User (refName UserModel), -imports-> file app/helpers.py (refName h) and run -instantiates-> User; the synced graph kept those three refs failed with tails UserModel / h.

Cause

Sync's #1240 retry finds a parked failed ref by name_tail, looked up by the changed files' node names and their moduleReferenceKeys. Python's binding refs carry the local alias, which no changed file declares. #2450 fixed this for ES modules (importBindingTail parks such a ref under module:<stem> and the retry also finds module-parked rows by whole name); its first line returned early for every non-ESM language.

Fix

  • importBindingTail gets a Python branch. A whole-name ref through a binding whose local name differs from the module's name for it (from X import A as B, import X.Y as Z) is parked under module:<last segment of the module>. The module comes from Python's dotted path, counted from the importing file's own package when it starts with dots: .models in app/main.py → module:models, . → module:app, ..pkg from app/sub/ → module:pkg, app.models → module:models; dots that climb above the project park nothing new. An unaliased import keeps its tail (the module, or the one it re-exports from, declares that name), as does a standard-library import (isExternalImport's list).
  • moduleReferenceKeys gives a package's __init__.py the key of the package above it, as a module file beside it already has through its folder: app/pkg/sub/__init__.py is what from app.pkg import sub as s (parked as module:pkg) imports.
  • fix(sync): a reference through an import binding links its module when the module appears later #2450's by-name lookup still finds every such row by its alias, so the triggers are main's plus the module's: a binding its module never declares is linked by name in a fresh index (from .lib import FormSchemaBase as FormSchema → another file's FormSchema), and that relinks as before.

An installed package's import can't be told apart from a module the project doesn't have yet (from settings import Config as Cfg must relink when settings.py appears), so it is parked under its module too. That is netbox's from django.utils.translation import gettext_lazy as _: 8,611 _() calls now wait under module:translation instead of _. Both keys are over the 500 ceiling, so nothing retries them, as before.

No extraction change (so no kernel change) and no schema change. Rows parked by an older version keep their alias tail (still found by name); a re-index parks them under their modules, hence the CHANGELOG's re-index note.

Validation

Tests: __tests__/sync-python-import-binding-retry.test.ts (5). Added / restored / edited-module scenarios over relative, package-relative (., ..), absolute and top-level imports, a submodule file, a subpackage, a re-export through a package __init__.py, and a function alias; each is compared edge for edge with a full index of the final tree in a second folder. Plus the by-name case and the parking keys. On main (fc0da38) 4 of the 5 fail: 0 of 19 links after add, 0 of 19 after restore, 5 of 19 after the edit, and the parking test. The by-name case passes on main by design and fails when #2450's by-name lookup is removed (0 of 2 edges), which is the path it guards. All 5 pass with the kernel and with CODEGRAPH_KERNEL=0. Related suites (all sync* retry suites, sync.test.ts, convergence, the Python resolution suites; 235 tests) pass, as does tsc. Full suite: 6,927 passed, 0 failed (82 skipped).

A/B: main = fc0da38 and fix = this branch, with the same kernel. Python repos: django, netbox (Django), pydantic, fastapi, flask, requests, httpx, microblog (Flask), fastapi-template and django-realworld. Non-Python controls: ts-express, takenote, vue3-element-admin, gin, serilog and kong.

  • Fresh index:
    • Byte-identical (nodes, edges, refs, files): the six non-Python controls (ts-express, takenote, vue3-element-admin, gin, serilog, kong) and three Python repos with no failing alias (django-realworld, httpx, fastapi-template).
    • Everywhere else, nodes, edges and files are byte-identical. refs differ only in the name_tail of refs that already failed (now module:*): requests 32, microblog 26, flask 22, fastapi 37, pydantic 55, django 171, netbox 8,699 (8,611 of them netbox's _ calls).
  • Late scenarios: every 7th source file held out, then restored and synced ("hold"), or every 7th Python file emptied, then restored and synced ("gut", the module-gains-the-name case). Each arm's synced graph is compared with its own fresh index; the numbers are differing dump lines, main → fix:
repo hold gut
requests 62 → 0 50 → 0
django 18,258 → 18,202 8,568 → 8,470
pydantic 7,770 → 7,758 2,248 → 2,248
netbox 1,990 → 1,988 1,812 → 1,812
fastapi 4,106 → 4,106 48 → 48
flask / httpx 206 / 172, unchanged 206 / 172, unchanged
fastapi-template 66 → 66 0 → 0
microblog, django-realworld 0 → 0 0 → 0
controls: ts-express, takenote, gin / serilog / kong / vue3-element-admin 0 / 8 / 126 / 182, unchanged —

Every residue bucket that changed went down: requests 10 (hold) and 9 (gut), django 20 and 17, pydantic 5, netbox 1. Examples are DjangoTestCase, MakeMigrationsCommand, PythonSerializer, DjangoWarning and RequestsSSLError. None went up. The one apparent rise, pydantic's module-parked bucket going from 0 to 1, is a ref that fails in both arms and only changed tail (see Residues). For the controls, the synced graphs themselves are byte-identical between arms.

Resolver pool: the parking decision also runs in pool workers. requests and pydantic indexed with the pool forced on (CODEGRAPH_PARALLEL_RESOLVE_MIN=0) and off (CODEGRAPH_NO_PARALLEL_RESOLVE=1) give the same module: tails. pydantic differs by one pre-existing function_ref, the same on main: foo = validate_call(foo) in tests/test_validate_call.py links to the nested def foo only when pooled.

Cost: the decision runs once per failed ref, with the file's mappings already cached by the attempt: ~0.1 s per full index on django (137K failed Python refs) and netbox (104K), ~15 ms on pydantic. That's about 0.1–0.3% of an index. The workspace map isExternalImport reads is already loaded by resolveImportPath before any ref is parked. Sync adds one lookup key per nested __init__.py.

Residues (pre-existing, unchanged)

  • Unaliased import through a package __init__.py that appears later: from fastapi import FastAPI, where fastapi/__init__.py re-exports from .applications import FastAPI as FastAPI. The ref waits under FastAPI, and the restored __init__.py declares no such node. fastapi holds FastAPI 731, APIRouter 165 and HTTPException 75 failed calls; netbox has the same for its utilities/forms/widgets classes. This is the Python side of the barrel case. Parking these under their module could crowd keys like module:models past the 500 ceiling in Django apps, so it needs its own measurement; filed as a follow-up.
  • Same edge, other metadata: a ref resolved by name while its module was missing keeps that metadata (fastapi 963, netbox 214, flask 83).
  • Member calls through an imported module or value: httpx httpx.MockTransport 61, pydantic m.model_dump 216.
  • Alias re-exported from a module in another folder: pydantic's from ._internal._schema_generation_shared import GetJsonSchemaHandler as _GetJsonSchemaHandler waits under module:_schema_generation_shared. The class lives in annotated_handlers.py, whose keys don't name that module. It fails on main too, under its alias.
  • pydantic's 3,423 retargeted instance-method guesses (v.validate_python) are the same in both arms.

Overlap

🤖 Generated with Claude Code

…ears later

`from .models import User as UserModel` and `from . import helpers as h`
bind a name their module declares under another: `User`, `helpers`. Every
reference through the alias (the import, a call, a base class, the class
or function passed as a value) was parked under the alias, which no file
a sync adds or changes carries. So a module added after its importers were
indexed, restored after a delete, or given the name in a later edit never
relinked them, while a fresh index did. #2450 fixed the same gap for ES
modules.

Such a reference is now parked under its module's key. A Python import
names its module by a dotted path, counted from the importing file's own
package when it starts with dots: `.models` in app/main.py is
'module:models', `.` is 'module:app', `..pkg` the package above's
'module:pkg'. #2450's lookup still finds it by its own name, which is how
a fresh index links an alias its module never declares. A package's
`__init__.py` also carries the key of the package above it, as a module
file beside it does through its folder, so `from app.pkg import sub as s`
relinks when the subpackage appears.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@colbymchenry
colbymchenry merged commit 5171a6a into main Oct 11, 2026
@colbymchenry
colbymchenry deleted the claude/kind-heisenberg-fd6f7b branch October 11, 2026 19:22
colbymchenry added a commit that referenced this pull request Oct 11, 2026
…the package appears later (#2507)

`from fastapi import FastAPI` reaches `fastapi/applications.py` through
`fastapi/__init__.py`, which re-exports it and declares no node named
`FastAPI`. While that `__init__.py` was missing, or before it gained the
re-export, the calls through the name failed and were parked under the
name, which the package file that then appears or changes never carries.
A fresh index linked them; a sync never did.

An unaliased from-import of a module in the project's own folders is now
parked under the module's key, as an aliased one is (#2501): a relative
import, or an absolute one whose dotted path, or the package holding it,
is a project folder at any depth. An outside package's names (`from
django.db.models import Q`) keep their tails, since on netbox they would
crowd 'module:models', 'module:urls' and others past the retry ceiling.
#2450's by-name lookup still finds every parked ref by its own name.

Two limits made the restored package's importers wait anyway:

- A module key gets a retry ceiling of its own, 5,000 rather than the 500
  a name tail has. A large population under a module key is every
  importer of a package that was missing, not noise: fastapi's 1,069
  `from fastapi import ...` refs, Django's 2,257 under 'module:urls'.
- The CG-33 rebind's per-name ceiling counts only the edges that existed
  before the retry. Relinking every `from django.urls import path` pushed
  `path` over it, so the `path` edges the sync's new definitions moved
  were never rebound; the retry's own edges are still rebound under a
  name that stays below it, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant