Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- In Rust, a variable that holds what a method call returns, like `let server = builder.port(80).build();`, `let settings = config.load()?;` or `let conn = client.connect().await;`, now has the type that method is declared to return, so calls on the variable link to that type's methods. A generic method's type comes from the call's turbofish, like `arg.get::<ValueHint>()`, and a call on a type that implements one trait twice, like `Encoder<Bytes>` and `Encoder<&[u8]>`, links to the implementation that takes the argument given. Before, such calls were linked by name and could reach another type's method of the same name. Re-index Rust projects after upgrading.
- In Rust, a `for` loop's variable now has the type of an item of what the loop goes over, like `for item in &self.items` over a `Vec<Item>`, `for (key, value) in &map` over a `HashMap`, or `for sc in cmd.get_subcommands()` over the `impl Iterator` a method returns, and each name in a tuple pattern like `let (tx, rx) = mpsc::channel(8);` has the type of its part of the tuple, so calls on them link to that type's methods. Before, such calls were linked by name: tokio's `tx.send(…)` calls on channel senders went to an internal type's `send`. Re-index Rust projects after upgrading.
- In Rust, a call on a field of `self`, like `self.store.save()` or `self.inner.lock()`, now links to the method of the type the struct declares for that field, also when the field's type is a parameter the `impl` bounds by a trait (`impl<S: Store> Worker<S>`), a trait object, or held in an `Arc` or a `Box`. A field whose type comes from the standard library or a dependency, like `jar: cookie::CookieJar` or `inner: std::sync::Mutex<…>`, no longer links to the project's own type of the same name, which often made a wrapper's method look like it called itself. Re-index Rust projects after upgrading.
- In Rust, a call written with a turbofish, like `matches.get_one::<String>("name")`, `parse_config::<Settings>()` or `Pool::<u8>::new()`, now links to the function it calls. Before, the type arguments stayed in the call's name, so no such call was linked. Re-index Rust projects after upgrading.
- In C++, every name a declaration lists now has the declaration's type, like `strings_found` in google/breakpad's `ByteBuffer symbols_found, strings_found;` or `data_` in a class's `Table *index_, *data_;`, so calls on it link to that class's methods. Before, only the first name had its type read, and a call on another name linked to nothing or to the class of an earlier variable with the same name.
- In C++, a call on a parameter, a local lambda or functor, a lambda's parameter or a range-`for` variable no longer links to a function or method elsewhere that only shares its name, like protocolbuffers/protobuf's code generators calling `format(…)` on their `Formatter& format` parameter, which linked to the conformance suite's `TestResult::format`, or facebook/rocksdb's tests calling a local `auto key = [](int i) {…};`. C++ finds the local first, so that is what the call calls.
- In TypeScript and JavaScript, an import of a package your project declares itself with `declare module '…'` in a `.d.ts` file, like a `typings/mammoth/index.d.ts` written for an untyped package or VS Code's own `vscode.d.ts`, now links to that declaration, so `codegraph callers`, impact and `codegraph affected` show the code that uses your typings instead of nothing or a project symbol that only shares a name. Named, default and namespace imports, `export =` and renaming export lists are read the way TypeScript reads them, a function such a file declares without a body (`export function convertToHtml(…): Promise<Result>;`) is now indexed so calls to it have something to link to, and only an import of its package reaches it. A `declare module` that augments a package's own types, a wildcard like `declare module '*.svg'`, and copies of typings kept in test fixtures are left alone. Re-index TypeScript projects after upgrading.
Expand Down
21 changes: 21 additions & 0 deletions __tests__/fixtures/kernel-parity/torture.rs
Original file line number Diff line number Diff line change
Expand Up @@ -312,3 +312,24 @@ pub union Reg {
}

impl Base for Reg {}

// Turbofish calls name the function they wrap: `s.get::<u32>()` is `s.get`,
// and a path's type arguments drop (`TfPool::<u8>::new` is `TfPool::new`),
// nested ones and a `->` inside them included.
fn turbofish_calls(s: &TfStore, items: &[u8]) {
let _a = s.get::<u32>();
let _b = tf_free::<Vec<u8>>();
let _c = TfPool::<u8>::new();
let _d = TfPool::<Vec<u8>>::with_capacity::<u8>(4);
let _e = TfPool::<u8>::new().len();
let _f = TfPool::create::<u8>().len();
let _g = tf_run::<fn() -> u8>(tf_free);
let _h = items.iter().collect::<Vec<_>>();
}

impl TfStore {
fn turbofish_self(&self) {
self.get::<u16>();
self.cache.load::<u8>();
}
}
125 changes: 125 additions & 0 deletions __tests__/rust-turbofish-calls.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
/**
* A Rust call written with a turbofish — `s.get::<u32>()`, `helper::<u8>()`,
* `Pool::<u8>::new()` — calls the function the turbofish wraps. Both engines
* kept the type arguments in the callee's name (`s.get::<u32>`), which no
* resolution strategy reads, so every such call was left unresolved.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { CodeGraph } from '../src';

let root = '';
let cg: CodeGraph;

beforeAll(async () => {
root = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-rust-turbofish-'));
fs.mkdirSync(path.join(root, 'src'));
fs.writeFileSync(path.join(root, 'Cargo.toml'), '[package]\nname = "app"\nversion = "0.1.0"\n');
fs.writeFileSync(path.join(root, 'src', 'main.rs'), [
'pub struct Store {',
' cache: Cache,',
'}',
'',
'impl Store {',
' pub fn get<T>(&self) -> Option<T> { None }',
'',
' pub fn own(&self) {',
' let _ = self.get::<u16>();',
' let _ = self.cache.load::<u8>();',
' }',
'}',
'',
'pub struct Cache;',
'',
'impl Cache {',
' pub fn load<T>(&self) -> Option<T> { None }',
'}',
'',
'/// A distractor: another `get` the typed receiver must win over.',
'pub struct Other;',
'',
'impl Other {',
' pub fn get<T>(&self) -> Option<T> { None }',
'}',
'',
'pub struct Pool<T> {',
' items: Vec<T>,',
'}',
'',
'impl<T> Pool<T> {',
' pub fn new() -> Pool<T> { Pool { items: Vec::new() } }',
' pub fn create<U>() -> Pool<T> { Pool { items: Vec::new() } }',
' pub fn len(&self) -> usize { self.items.len() }',
'}',
'',
'pub fn helper<T>() -> u8 { 0 }',
'',
'pub fn run<F>(f: F) -> u8 { 0 }',
'',
'pub fn calls(s: &Store) {',
' let _ = s.get::<u32>();',
' let _ = helper::<Vec<u8>>();',
' let _ = Pool::<u8>::new();',
' let _ = Pool::<u8>::create::<u16>();',
' let _ = run::<fn() -> u8>(helper::<u8>);',
'}',
'',
'pub fn chains() -> usize {',
' Pool::<u8>::new().len() + Pool::<u8>::create::<u16>().len()',
'}',
'',
'fn main() {}',
'',
].join('\n'));
cg = await CodeGraph.init(root, { index: true });
});

afterAll(() => {
cg?.destroy();
fs.rmSync(root, { recursive: true, force: true });
});

/** `qualifiedName` of every call target of the function or method `name`. */
function callees(name: string): string[] {
const fn = cg.getNodesByName(name).find((n) => n.kind === 'function' || n.kind === 'method');
expect(fn).toBeDefined();
return cg.getOutgoingEdges(fn!.id)
.filter((e) => e.kind === 'calls')
.map((e) => cg.getNode(e.target)!.qualifiedName)
.sort();
}

describe('Rust turbofish calls', () => {
it('a method call on a typed local reaches its type\'s method', () => {
expect(callees('calls')).toContain('Store::get');
expect(callees('calls')).not.toContain('Other::get');
});

it('a free function and a path call reach what they name, type arguments and all', () => {
const c = callees('calls');
expect(c).toContain('helper');
expect(c).toContain('run');
expect(c).toContain('Pool::new');
expect(c).toContain('Pool::create');
});

it('a call on self or on a field of self reaches the method', () => {
expect(callees('own')).toEqual(['Cache::load', 'Store::get']);
});

it('a chain through a turbofish path reaches the method of what it returns', () => {
// Two `.len()` calls, one per chain.
expect(callees('chains')).toEqual(['Pool::create', 'Pool::len', 'Pool::len', 'Pool::new']);
});

it('keeps no type arguments in a call reference', () => {
const fn = cg.getNodesByName('calls').find((n) => n.kind === 'function')!;
const names = cg.getOutgoingEdges(fn.id)
.filter((e) => e.kind === 'calls')
.map((e) => String((e.metadata as Record<string, unknown> | undefined)?.refName ?? ''));
expect(names.length).toBeGreaterThan(0);
expect(names.filter((n) => n.includes('::<'))).toEqual([]);
});
});
61 changes: 57 additions & 4 deletions codegraph-kernel/src/rustlang.rs
Original file line number Diff line number Diff line change
Expand Up @@ -882,6 +882,11 @@ impl<'t> Walker<'t> {
let func = node
.child_by_field_name("function")
.or_else(|| node.named_child(0));
// A turbofish — `s.get::<u32>()`, `helper::<u8>()`,
// `Store::with_capacity::<u8>(1)` — calls the function it wraps: its
// type arguments are no part of the callee's name. Mirrors
// TreeSitterExtractor.extractCall.
let func = func.map(unwrap_generic_function);
let mut callee_name = String::new();

if let Some(func) = func {
Expand Down Expand Up @@ -920,7 +925,7 @@ impl<'t> Walker<'t> {
// function chain (`Foo::new().bar()`, inner
// callee a scoped_identifier). Instance chains
// keep the bare method name.
let inner_fn = r.child_by_field_name("function");
let inner_fn = r.child_by_field_name("function").map(unwrap_generic_function);
let reencode =
inner_fn.map(|f| f.kind() == "scoped_identifier").unwrap_or(false);
if reencode {
Expand All @@ -930,6 +935,7 @@ impl<'t> Walker<'t> {
.chars()
.filter(|c| !c.is_whitespace())
.collect();
let inner = strip_turbofish(&inner);
callee_name = format!("{inner}().{method_name}");
} else {
callee_name = method_name.to_string();
Expand Down Expand Up @@ -974,10 +980,11 @@ impl<'t> Walker<'t> {
}
}
} else if matches!(func.kind(), "scoped_identifier" | "scoped_call_expression") {
callee_name = self.text(func).to_string();
// A path's turbofish is no part of the name either
// (`Vec::<u8>::with_capacity` → `Vec::with_capacity`).
callee_name = strip_turbofish(self.text(func));
} else {
// identifier; generic_function keeps the raw turbofish text
// (`helper::<T>` — unresolvable downstream, preserved).
// identifier (a turbofish was unwrapped above).
callee_name = self.text(func).to_string();
}
}
Expand Down Expand Up @@ -1601,6 +1608,52 @@ fn is_stoplisted(name: &str) -> bool {
)
}

/// The function a turbofish wraps: `s.get::<u32>` → `s.get`,
/// `helper::<u8>` → `helper`; any other callee as is.
fn unwrap_generic_function(func: Node<'_>) -> Node<'_> {
if func.kind() == "generic_function" {
func.child_by_field_name("function").unwrap_or(func)
} else {
func
}
}

/// A callee path without its turbofish type arguments: `Vec::<u8>::new` →
/// `Vec::new`. The brackets nest (`::<Vec<u8>>`), and the `>` of a `->` is
/// no closing one (`::<fn() -> u8>`). Mirrors stripRustTurbofish in
/// TreeSitterExtractor: both emit the same bytes.
fn strip_turbofish(text: &str) -> String {
if !text.contains("::<") {
return text.to_string();
}
let chars: Vec<char> = text.chars().collect();
let mut out = String::with_capacity(text.len());
let mut i = 0;
while i < chars.len() {
if chars[i] == ':' && i + 2 < chars.len() && chars[i + 1] == ':' && chars[i + 2] == '<' {
let mut depth = 0i32;
let mut j = i + 2;
while j < chars.len() {
let c = chars[j];
if c == '<' {
depth += 1;
} else if c == '>' && chars[j - 1] != '-' {
depth -= 1;
if depth == 0 {
break;
}
}
j += 1;
}
i = j + 1;
continue;
}
out.push(chars[i]);
i += 1;
}
out
}

/// LITERAL_RECEIVER_TYPES (shared table).
fn is_literal_receiver(kind: &str) -> bool {
matches!(
Expand Down
49 changes: 46 additions & 3 deletions src/extraction/tree-sitter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,33 @@ function peelSwiftCallWrappers(node: SyntaxNode, source: string): SyntaxNode {
return cur;
}

/**
* A Rust callee path without its turbofish type arguments: `Vec::<u8>::new`
* calls `Vec::new`, `Store::with_capacity::<u8>` calls `Store::with_capacity`.
* The arguments' brackets nest (`::<Vec<u8>>`), and the `>` of a `->` is no
* closing one (`::<fn() -> u8>`). Mirrored in the kernel's strip_turbofish
* (rustlang.rs): both emit the same bytes.
*/
function stripRustTurbofish(text: string): string {
if (!text.includes('::<')) return text;
let out = '';
for (let i = 0; i < text.length; i++) {
if (text.startsWith('::<', i)) {
let depth = 0;
let j = i + 2;
for (; j < text.length; j++) {
const c = text[j];
if (c === '<') depth++;
else if (c === '>' && text[j - 1] !== '-' && --depth === 0) break;
}
i = j;
continue;
}
out += text[i];
}
return out;
}

/**
* A Swift receiver's text from its innermost wrapped operand on: the wrappers
* sit on the chain's left edge, so `!Settings.Defaults` reads
Expand Down Expand Up @@ -5465,6 +5492,15 @@ export class TreeSitterExtractor {
func = peelSwiftCallWrappers(func, this.source);
}

// A Rust turbofish — `s.get::<u32>()`, `helper::<u8>()`,
// `Store::with_capacity::<u8>(1)` — calls the function it wraps: its type
// arguments are no part of the callee's name, which stayed in the ref
// (`s.get::<u32>`) and resolved through no strategy. Mirrored in the
// kernel's extract_call (rustlang.rs).
if (func && this.language === 'rust' && func.type === 'generic_function') {
func = getChildByField(func, 'function') ?? func;
}

// C++ explicit operator call `a.operator+(b)` / `p->operator+(b)` (#1247):
// tree-sitter-cpp can't parse an operator_name in field position, so the
// callee is NOT a field_expression — the call_expression carries
Expand Down Expand Up @@ -5640,10 +5676,16 @@ export class TreeSitterExtractor {
// fluent codebases. Leave those to the bare-name path.
reencode = /^[A-Z]/.test(innerCallee);
} else {
const innerFn = getChildByField(receiver, 'function');
let innerFn = getChildByField(receiver, 'function');
// Rust: the inner call's turbofish is no part of its name either
// (`Foo::create::<T>().run()`, `Vec::<u8>::new().len()`).
if (this.language === 'rust' && innerFn?.type === 'generic_function') {
innerFn = getChildByField(innerFn, 'function') ?? innerFn;
}
innerCallee = innerFn
? getNodeText(innerFn, this.source).replace(/->/g, '.').replace(/\s+/g, '')
: '';
if (this.language === 'rust') innerCallee = stripRustTurbofish(innerCallee);
// Rust: only re-encode an associated-function chain
// (`Foo::new().bar()`), whose inner callee is a path/`scoped_identifier`.
// Go: only a bare package-level factory chain (`New().Method()`),
Expand Down Expand Up @@ -5805,8 +5847,9 @@ export class TreeSitterExtractor {
}
}
} else if (func.type === 'scoped_identifier' || func.type === 'scoped_call_expression') {
// Scoped call: Module::function()
calleeName = getNodeText(func, this.source);
// Scoped call: Module::function(); a Rust path's turbofish is no
// part of the name (`Vec::<u8>::with_capacity` → `Vec::with_capacity`).
calleeName = this.language === 'rust' ? stripRustTurbofish(getNodeText(func, this.source)) : getNodeText(func, this.source);
} else if (this.language === 'csharp' && func.type === 'member_access_expression') {
// C# member call `recv.Method(...)`. When the receiver is itself a call
// — a chained factory `Foo.Create(args).Bar()` — encode `inner().Bar`
Expand Down