fix(sync): a Python import of a name a package re-exports links when the package appears later

[colbymchenry]

Problem

codegraph sync (and the watcher) never relinked an unaliased Python import of a name a package re-exports when the package's __init__.py was added after the importing files were indexed, or gained the re-export in a later edit. A fresh index links it.

from fastapi import FastAPI reaches fastapi/applications.py through fastapi/__init__.py (from .applications import FastAPI as FastAPI). With __init__.py missing, the calls through the name (FastAPI(), APIRouter(), HTTPException(...)) fail and stay failed after it comes back. In a hold-out run on fastapi (every 7th source file held out, restored and synced), that left FastAPI 735, APIRouter 166 and HTTPException 75 calls unlinked. netbox had the same for its utilities/forms/widgets classes, and Django for everything imported from django.urls and django.test. Follow-up to #2501; residue noted there.

Cause

Three things, each of which alone kept the importers unlinked:

1. Tail. The failed refs were parked under the name (FastAPI). Sync's Incremental sync misses cross-file relations after a new export is added #1240 retry looks failed refs up by the changed files' node names and module keys, and a re-exporting __init__.py declares no node of that name (a re-export only makes an import node). fix(sync): a Python import alias links its module when the module appears later #2501 parked aliased Python bindings under their module and left unaliased ones on their names.
2. Retry ceiling. Once parked under the module, fastapi's refs all wait under module:fastapi: 1,069 of them, over the 500-per-key ceiling getRetryableFailedReferences applies, so they were skipped anyway. Django: 2,257 under module:urls, 1,867 under module:test.
3. Rebind ceiling. With 1 and 2 fixed, django's late sync relinked thousands of refs but rebound some other edges worse than main: 107 imports by name instead of through the package, and 2 calls to a different same-named method. The retry runs before the CG-33 rebind (resurrectStaleResolutionEdges), whose per-name ceiling counts every edge whose target has the name. Relinking every from django.urls import path pushed path over 500, so the rebind skipped path and left the path edges this sync's new definitions moved unrebound. That was traced with an instrumented sync: on main all three refs are re-opened by the rebind and re-resolved in the sweep; on the first fix arm none was.

Fix

• Park unaliased project imports under their module (pythonImportBindingTail): an unaliased from X import Y is parked under module:<last segment of X>, as an alias is, when X is in the project's own folders. That means a relative import, or an absolute one whose dotted path, or the package holding it, is a project folder at any depth (isProjectPythonModule, a per-pass memo of folder-path suffixes). It's a folder, not a module, because fastapi/__init__.py is what goes missing while fastapi/ keeps its other files. An outside package's names keep their tails: from django.db.models import Q, from django.urls import path in netbox. Parking those would crowd netbox's module:urls (1,575), module:models (1,140), module:exceptions (876) past the ceiling, and the project's own refs under the same keys with them. An unaliased import x keeps its tail, as before. fix(sync): a reference through an import binding links its module when the module appears later #2450's by-name lookup still finds every parked row by its own name.
• A retry ceiling of its own for module keys: 5,000 instead of 500. A name tail's 500 guards against generic names (get, map). A module key's population is every importer of one module, which the file that just appeared resolves at once. The ceiling now only bounds the work. This also covers JS/TS modules parked by fix(sync): a reference through an import binding links its module when the module appears later #2450/fix(sync): a route links the module it lazily loads when the module appears later #2422 with more than 500 waiting refs. fix(sync): a reference through an import binding links its module when the module appears later #2450 measured one on vscode, module:sessionState with 546 refs that fail in a fresh index too; it would now be retried when such a file changes, which is wasted work but no change in the graph. Not re-measured here.
• The rebind's per-name ceiling counts only edges from before the retry (getResolutionEdgesByTargetName(…, countedUpToEdgeId), getMaxEdgeId; edges.id is AUTOINCREMENT). The retry's own edges were resolved against this sync's graph a moment earlier. They are still re-opened under a name that stays below the ceiling, exactly as on main, so the rebind's choice of names equals main's whenever the retry adds no edges.

No schema change, no extraction change (no kernel change).

Validation

Tests: __tests__/sync-python-import-binding-retry.test.ts, now 10 tests:

• The package added after its importers, and an edit that gives it the re-export. Each is compared edge for edge with a full index in a second folder. They cover an absolute import (from shop import Order), one from the package above (from .. import Order), and a renamed re-export, so nothing named Order exists for a name lookup.
• fastapi's same-name shape (Kit as Kit), compared link for link.
• The parking keys: project imports go to their module; from django.db.models import Q and stdlib imports keep their tails.
• Both ceilings, at query level: 600 refs under a module key are retried; 600 under a name are not, nor by name; the rebind keeps a name whose 600 edges came after the mark.
• On main (5171a6a), the 6 new or changed tests fail: 2 of 8 links after adding the package, 2 of 8 after the edit, 2 of 3 in the same-name shape, the parking test, the module ceiling (0 of 600), and getMaxEdgeId missing. The 4 alias tests from fix(sync): a Python import alias links its module when the module appears later #2501 pass on both.
• All 10 pass with the kernel and with CODEGRAPH_KERNEL=0.
• Related suites (every sync* and python* suite, 238 tests) pass. Full suite: 6,932 passed, 0 failed (82 skipped). tsc clean.

A "restored after delete" scenario isn't included. Deleting a re-export-only __init__.py doesn't unlink anything: the edges that resolved through it target nodes in files that still exist. That's a separate convergence gap in the removal direction, the same on main, and filed as a follow-up.

A/B: main = 5171a6a, fix = this branch, same kernel.

• Python repos: django, netbox, pydantic, fastapi, flask, requests, httpx, microblog, fastapi-template, django-realworld.

• Non-Python controls: ts-express, takenote, vue3-element-admin, gin, serilog, kong.

• Fresh index: the six controls, httpx and django-realworld are byte-identical (nodes, edges, refs, files). Everywhere else, nodes, edges and files are byte-identical, and only the name_tail of refs that already failed changes: requests 116, microblog 1, flask 34, fastapi-template 12, netbox 237, django 1,421, fastapi 2,172, pydantic 3,046.

• Late scenarios: every 7th source file held out and then restored ("hold"), or every 7th Python file emptied and then restored ("gut"), then synced. Each arm's synced graph is compared with its own fresh index; the numbers are differing dump lines, main → fix.

repo	hold	gut
django	18,202 → 7,966	8,470 → 6,854
netbox	9,956 → 7,166	3,612 → 3,612
fastapi	4,122 → 2,140	48 → 48
pydantic	7,728 → 7,718	2,250 → 2,250
flask / httpx / fastapi-template	206 / 172 / 66, unchanged	206 / 172 / 0, unchanged
requests, microblog, django-realworld	0 → 0	0 → 0
controls: ts-express, takenote, gin / serilog / kong / vue3-element-admin	0 / 8 / 126 / 182, unchanged	—

Every residue bucket that changed went down: django 80 (hold) and 12 (gut), netbox 25, fastapi 8, pydantic 2. None went up, and the 107 + 2 rebind mismatches the first fix arm had are gone.

Cost:

• Parking decision: about 50–60 ms per full index on django (137K failed Python refs) and netbox (105K), the same as main within noise.
• Sync time, interleaved main/fix/main/fix on a quiet box: vue3-element-admin 2.38/2.42 vs 2.44/2.38 s, requests 1.57/1.33 vs 1.40/1.35 s, fastapi 5.23/5.25 vs 5.30/5.53 s. fastapi relinks about 1,000 more refs for that.
• Over the module ceiling after a hold-out sync: only keys whose refs fail in a fresh index too. Those are fastapi module:testclient (1,484, re-exported from starlette), pydantic module:pydantic_core (2,432, compiled-extension names) and netbox module:translation (8,612, gettext_lazy as _, from fix(sync): a Python import alias links its module when the module appears later #2501). Each sits over the 5,000 module ceiling or is never resolvable, so it costs nothing. A key over the ceiling also drops the module trigger for anything else parked under it; on pydantic that is 5 aliased refs into the same unresolvable package.

Resolver pool: the decision also runs in pool workers. fastapi and django indexed with the pool forced on (CODEGRAPH_PARALLEL_RESOLVE_MIN=0) and off (CODEGRAPH_NO_PARALLEL_RESOLVE=1) park the same module: tails (2,209/2,209 and 1,593/1,593). fastapi is byte-identical. django differs by one pre-existing nested-function function_ref (async_view), which is the same on main, like pydantic's foo noted in #2501.

Residues (pre-existing, the same on main)

• Same edge, other metadata: an import matched by name while its package was missing keeps exact-match where a fresh index says import (fastapi 961, django 3,065, netbox 261). This includes fastapi's from fastapi import FastAPI imports themselves; their calls are what this PR relinks.
• Import-retry ceiling (fix(sync): an import links the file it names when that file appears later #2392): django's 678 django.test module-path imports wait under the tail test, which is over 500.
• Popular names over the name ceiling: netbox self.assertHttpStatus 801, add_permissions 576, and extends TestCase 417, which imports Django's TestCase that a fresh index links by name to netbox's own.
• Stale links after a re-exporting __init__.py is deleted (see Validation); filed separately.

Overlap

• fix(sync): a Python import alias links its module when the module appears later #2501 (merged): this extends its pythonImportBindingTail; the alias behavior is unchanged.
• fix(sync): a reference through an import binding links its module when the module appears later #2450 / fix(sync): a route links the module it lazily loads when the module appears later #2422 (merged): their module keys share the new module ceiling; their suites pass.
• fix(python): resolve imports from the package roots the project has #1921 (open, Python import resolver rewrite) touches import-resolver.ts, queries.ts and sync, but none of these functions. Whichever lands second may need a small rebase.

🤖 Generated with Claude Code