GH-1391: Cap the token positions of one SentenceVectorsDL.embedAll inference

[krickert]

Draft until #1462 is merged

TextEmbedder.embedAll and the manual say the implementation forms its own batches, so a caller may pass a list of any size. SentenceVectorsDL.embedAll groups texts by tokenized length and splits each group so that no inference takes more than 16384 token positions (rows times row length), which is 32 rows at the BERT length of 512. An input longer than 16384 tokens runs in a batch of its own.

Vectors do not change: batches are not padded, so every row is computed from the same tensors as its single-input call.

The grouping is in a package-private batches(Tokens[]) that returns the index lists embedAll runs, so the tests can check the inference plan directly. MAX_BATCH_TOKEN_POSITIONS is a package-private constant, not an option.

AI assisted test description:

Tests, in SentenceVectorsDLBatchLimitTest: row count boundaries computed from the cap (exact fit, one over, one under, lengths that do and do not divide it), a length at and over the cap, an empty call, groups in call order, a mix where only one group splits (checked through the session at the batch boundaries), an input at the cap next to short inputs, and every pooling path through split groups of alternating lengths. The GPU parity test of #1462 also runs a split group. The tiny model texts and expected vectors are shared in TinyVectorModels.

Fixes #1391

[rzo1]

Hi @krickert, thanks for the PR. The splitting in batches(...) looks correct to me. Some remarks inline, and two on the process:

• This PR contains the commit of #1462, which currently has changes requested. Please mark it as draft until #1462 is merged and rebase it afterwards; the review points on the shared test fixtures of #1462 apply here as well.
• The description mentions new tests in SentenceVectorsDLEmbedderTest, but the commit only adds SentenceVectorsDLBatchLimitTest. Please also set the milestone and labels.

No eval build is needed, since no model and no default behaviour changes.

[krickert]

Thanks. We'll keep in draft until #1462 happens, and rebased on the updated #1462 with its fixture changes. The description now lists only the tests the commit adds.

This helps the performance a lot, highly recommend keeping it in if we have the throughput.