Skip to content

Python 3.14 refcount semantics and problems with resize #35

Description

@bmcfee

I'm just flagging this for you so you're aware, though it's likely to be fixed upstream of you (ie in numpy) anyway.

I've been seeing test failures in librosa on 3.14 environments lately, and after discussing with numpy devs, traced it back to the following issue numpy/numpy#30991 wherein the resize operation causes problems with memory reference counting on python 3.14 (but not earlier).

Note: this is a rather sneaky bug, as it only pops up for me when bundling resample calls within a vectorization like np.apply_along_axis (as we do in librosa to handle multichannel).

Looking into the python-samplerate code, it appears that resize is generally used to make a truncated view of an over-allocated output array:

// create a shorter view of the array
if ((size_t)src_data.output_frames_gen < new_size) {
out_shape[0] = src_data.output_frames_gen;
output.resize(out_shape);
}

It's probably more hassle than it's worth to rewrite these bindings to avoid view resize. (Unless libsamplerate exposes a function to pre-compute output lengths?) It may also work (and is probably safe) to force refcheck=False into the resize calls, but that seems a little hacky IMO.

Activity

  1. mattip commented on Mar 16, 2026

    @mattip

    It's probably more hassle than it's worth to rewrite these bindings to avoid view resize

    NumPy dev piling on here. After a naive look at the code it seems like newsize could be adjusted before allocating output, rather than after. But there are probably things I don't understand.

    In general, np.resize in my opinion is something to be avoided. It can lead to memory fragmentation.

  2. bmcfee commented on Mar 16, 2026

    @bmcfee
    Author

    In general, np.resize in my opinion is something to be avoided. It can lead to memory fragmentation.

    👍 to that, though I think in this case it's not actually that bad. I didn't write this code or the underlying library that it's wrapping, but I have implemented these algorithms independently in the past. The code in question is doing a specific kind of interpolation over the input array, and due to the vagaries of floating point non-associativity, it's easy to end up in off-by-one errors when computing output lengths by rounding. As such, it's safer to over-allocate and trim (view-trim), and the fragmentation is relatively small (ie one sample, or 4-8 bytes typically).

    At least, I'm reasonably confident that's how it plays out in practice, though I could be missing some details.

  3. mattip commented on Mar 16, 2026

    @mattip

    Ahh, then it might work to return a view output[:newsize], if it only overallocates by a row or two, rather than trying to resize it.

  4. ngoldbaum commented on Mar 17, 2026

    @ngoldbaum

    FWIW, I think the use here is unsafe, and the error being raised on 3.14 is correctly pointing out an unsafe pattern.

    Unfortuantely on 3.13 and older, it's impossible to tell the difference between an array created by an extension with one reference and a uniquely referenced array created by a Python function. The old check was not stringent enough to catch issues in C extensions. But on 3.14 due to changes in reference counting semantics for function locals, we now check for truly uniquely referenced arrays, and that catches the code in python-samplerate.

    I think in practice the chance of issues here is small, but I think the error is probably pointing out a real possible source of unsafety that's been undetected until now.

  5. tuxu commented on Mar 18, 2026

    @tuxu
    Owner

    Thanks for raising the issue and discussing here. I think it's best to return a view, which is also how v0.1.0 worked.

  6. fakufaku commented on Mar 21, 2026

    @fakufaku
    Collaborator

    Thanks for raising the issue and chiming in here.
    I've created a fix that changes the resize for a view in #36 .
    @tuxu Please review and approve the change if you have a chance.

  7. added a commit that references this issue on Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions