Skip to content

Reduce the overhead of ncon - #314

Open
lkdvos wants to merge 7 commits into
mainfrom
ncon-overhead
Open

lkdvos wants to merge 7 commits into
mainfrom
ncon-overhead

Conversation

@lkdvos

@lkdvos lkdvos commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

ncon spent most of its time on bookkeeping rather than on contractions: runtime type inference (promote_op) at every node of the contraction tree, abstractly typed containers, repeated keyword argument handling, and unspecialized network analysis. This PR makes the label handling type stable, so that every pairwise contraction goes through a single dynamic dispatch. It also includes the fix for #273, previously #315.

  • Type-stable BLAS contraction. makeblascontractable received the scalar type as an unconstrained argument, which Julia does not specialize on, so the returned tensor was inferred as Any and the rest of the BLAS contraction path was dynamically dispatched. It also returned either its input or a permuted copy, whose types differ when the copy comes from an allocator such as Bumper's, and that union was heap allocated on every contraction. makeblascontractable is removed: _blas_contract! now makes the inputs BLAS-contractable one at a time and passes them on within each branch, so the union is never materialized. Each input is reshaped into a matrix before it is passed on, so the later steps do not depend on its number of indices, which keeps the number of compiled methods down. This also speeds up @tensor (see the reference row below).
  • Specialize the network analysis. ncontree.jl and indexordertree.jl were included in the module-level @nospecialize block meant for the macros, but are also used at runtime by ncon.
  • Faster validation and tree construction. Labels are counted rather than repeatedly searched and deleted, and the contraction tree is built in a type-stable manner as a list of pairwise contractions. The nested trees returned by ncontree and indexordertree are unchanged, except that disconnected parts of the network are now combined from left to right.
  • Type-stable ncon. Tensors, labels, output and conjugation flags are canonicalized once, all label manipulations happen on Vector{Int}, and keyword arguments are processed once.
  • contract_indices for labels stored in vectors. This avoids the type-unstable tuple implementation, which is compiled for every combination of numbers of indices. It also applies to tensorcontract(IC, A, IA, B, IB) with vector labels.

As a side effect, traces can now be specified with row matrices as index lists.

Fixes #273, together with QuantumKitHub/Strided.jl#94. Allocations in its reproducer (StridedBLAS, Bumper allocator, latest Strided and StridedViews main):

main this PR this PR + Strided.jl#94
Float64 112 B 48 B 0 B
ComplexF64 112 B 48 B 0 B
ComplexF64, conj 48 B 48 B 0 B

Benchmarks

Exclusive node, Julia 1.13.1, a single thread, every measurement in a fresh process, registered Strided. Tensors have dimension 2 per index unless noted, so the overhead dominates. Minimum times; three interleaved rounds agree to within 2%.

main this PR
5 tensors, mixed arity 58.8 μs / 386 allocs 6.3 μs / 149 allocs
4 tensors, mixed arity, Tuple input 44.2 μs / 290 allocs 4.8 μs / 126 allocs
matrix chain (4) 28.5 μs / 228 allocs 3.0 μs / 115 allocs
matrix chain, order keyword 33.5 μs / 361 allocs 2.9 μs / 109 allocs
with trace 9.0 μs / 85 allocs 3.0 μs / 68 allocs
10-tensor MPS ladder, ComplexF64 137.9 μs / 842 allocs 19.8 μs / 321 allocs
MPS transfer matrix, D = 32 108.0 μs / 311 allocs 58.4 μs / 126 allocs
@tensor reference (first row) 3.47 μs / 46 allocs 2.52 μs / 22 allocs
#273 reproducer, Bumper allocator 0.57 μs / 112 B 0.57 μs / 48 B
same, default allocator 2.35 μs / 16 allocs 0.61 μs / 4 allocs
8⁴ tensors, permuted copies needed 18.2 μs / 25 allocs 14.6 μs / 7 allocs
first ncon call / with new types 2.46 s / 1.18 s 1.74 s / 0.69 s
first calls of 12 networks with different numbers of indices 7.1 s 4.4 s

The last row is from an earlier run with the same setup (the previous version of this PR, without the #273 fix); the final version matches it within noise in local comparisons (median 4.32 s without and 4.39 s with the fix).

🤖 Generated with Claude Code

@lkdvos
lkdvos marked this pull request as draft October 13, 2026 19:54
@lkdvos
lkdvos force-pushed the ncon-overhead branch 2 times, most recently from cc48af7 to 9bde54c Compare October 14, 2026 15:52
@lkdvos
lkdvos marked this pull request as ready for review October 14, 2026 18:09
@lkdvos
lkdvos requested a review from Jutho October 14, 2026 18:52
@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.13260% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/indexnotation/ncontree.jl 90.47% 6 Missing ⚠️
src/implementation/indices.jl 96.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
ext/TensorOperationsChainRulesCoreExt.jl 83.52% <ø> (ø)
src/TensorOperations.jl 100.00% <ø> (ø)
src/implementation/blascontract.jl 100.00% <100.00%> (ø)
src/implementation/ncon.jl 100.00% <100.00%> (ø)
src/indexnotation/indexordertree.jl 100.00% <100.00%> (+14.00%) ⬆️
src/implementation/indices.jl 98.09% <96.00%> (-0.72%) ⬇️
src/indexnotation/ncontree.jl 90.21% <90.47%> (-2.21%) ⬇️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/indexnotation/indexordertree.jl Outdated
Comment thread src/indexnotation/indexordertree.jl Outdated
Comment thread src/indexnotation/indexordertree.jl Outdated
Comment thread src/indexnotation/ncontree.jl Outdated
Comment thread src/indexnotation/ncontree.jl Outdated
Comment thread src/indexnotation/ncontree.jl
Comment thread src/indexnotation/ncontree.jl Outdated
Comment thread src/indexnotation/ncontree.jl Outdated

# note: avoiding `collect(Any, xs)` and mutation for compatibility with Zygote
_anyvector(xs::AbstractVector) = Vector{Any}(xs)
_anyvector(xs::Tuple) = Any[xs...]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we still need this. I also don't understand the Zygote comment.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main point is that various constructions are not compatible with Zygote because internally they do some mutation and have no rules. The comment wasn't very clear and the functions weren't either, so I think I now have a new implementation that is better and bypasses most of this?

Comment thread src/indexnotation/ncontree.jl Outdated
Comment thread src/indexnotation/ncontree.jl
Comment thread src/implementation/ncon.jl

@Jutho Jutho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR. Definitely contains several good improvements, but quite a big PR with also some changes that I don't (yet?) understand the benefit of and seem to make the code more complicated to understand.

lkdvos and others added 7 commits October 22, 2026 17:11
`makeblascontractable` received the scalar type of the destination as an unconstrained
argument, which Julia does not specialize on. As a result, the allocation and permutation of
inputs that are not directly BLAS-contractable were dynamically dispatched, which also
affects the `@tensor` macro.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The index notation files are included in a module-level `@nospecialize` block to reduce the
compilation time of the macros. However, `ncontree.jl` and `indexordertree.jl` contain
functions that are also called at runtime by `ncon`, such as `check_nconstyle`, `ncontree`
and `indexordertree`, which therefore ran without specialization on their arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `check_nconstyle` and `_verify_indexorder` count label occurrences instead of repeatedly
  searching and deleting from an abstractly typed vector.
- `ncontree` and `indexordertree` share a single implementation that merges partial trees
  in a type-stable manner, and represent the contraction tree as a list of pairwise
  contractions. The nested trees that are returned are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The network and the intermediate results of `ncon` have non-uniform types, which previously
caused type instabilities and runtime type inference throughout the evaluation of the
contraction tree. Now:

- `resolve_traces` canonicalizes the tensors and labels to a `Vector{Any}` and
  `Vector{Vector{Int}}`, and the output and conjugation flags are canonicalized as well,
  such that the remainder of `ncon` is type stable and compiled only once.
- The contraction tree is traversed as a list of pairwise contractions with integer ids.
- The keyword arguments are processed once, instead of for every pairwise contraction.
- All label manipulations happen on `Vector{Int}`, and every pairwise contraction goes
  through a single function barrier, in which the scalar type is inferred statically
  instead of through `promote_op` at runtime.

As a side effect, `ncon` can now be differentiated with Zygote, and networks with traces
can be specified with row matrices as index lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tuple implementation of `contract_indices` manipulates the labels in a type-unstable
manner, and is compiled for every combination of numbers of indices. For labels stored in
vectors, as in `ncon`, the permutations are now computed as vectors in a type-stable manner,
and only converted to tuples at the end.

`ncon` passes these tuples to its function barrier as separate arguments rather than as
`Index2Tuple`s, which avoids boxing them. This reduces the compilation time for networks with
different numbers of indices, at a small cost in runtime for small networks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`makeblascontractable` returned either its input or a permuted copy, whose types differ when
the copy is allocated by an allocator such as Bumper's. The resulting union is not an
`isbits` type and was therefore heap allocated on every contraction, even when no copy was
required. Its specialization on the scalar type also made Enzyme fail ("could not determine
how to keep a pointer phi rooted").

`_blas_contract!` now makes the inputs BLAS-contractable one at a time, passing them on to
the next step in each branch rather than returning them, such that the union is never
materialized. Each input is reshaped into a matrix before it is passed on, so that the
subsequent steps no longer depend on the number of indices of the inputs, which keeps the
number of compiled methods (and the compilation time) at the level of before this change.
`makeblascontractable` and `_unsafe_blas_contract!` are no longer used and are removed,
together with the tests of the former.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `_ncontree_steps` merges the two occurrences of every contraction index directly from the
  sorted list, rather than searching for the second occurrence.
- `_check_nconstyle` compares neighbours in the sorted list of labels instead of counting.
- `_indexordertree_steps` asserts that the second occurrence of a contraction index exists,
  as before, and uses `∋(c)` rather than anonymous functions.
- Disconnected parts of the network are combined from left to right.
- `resolve_traces` uses comprehensions, without the helpers `_ncon_network` and
  `_anyvector`. As a consequence, `ncon` can no longer be differentiated with Zygote.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lkdvos

lkdvos commented Oct 23, 2026

Copy link
Copy Markdown
Member Author

Alright, I think the comments have been addressed here, thanks for the careful review, I agree I should have gone through the tree manipulation implementations a bit more carefully but I am not too familiar with that piece of the code so I definitely missed some obvious simplifications. Let me know what you think about this version, happy to still adapt some things!

@lkdvos
lkdvos requested a review from Jutho October 23, 2026 14:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StridedBLAS contraction path allocates 64 B on Julia 1.12.1; inlining appears to eliminate the allocation

2 participants