Repository navigation
Conversation
…story_chars) Every generation's input carried the full history of all earlier turns, so memory grew with steps x session size and long sessions hit a V8 heap OOM in the Stop hook, which aborts before turns are marked uploaded and fails on every later Stop. Keep the most recent earlier-turn messages within a character budget (default 200k, LANGFUSE_CODEX_MAX_HISTORY_CHARS), replace the rest with an omission marker, and record the count in codex.history_omitted_messages. Fixes #119 Co-authored-by: Hassieb Pakzad <hassiebp@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #119 (Linear LFE-17701, langfuse/langfuse#18995).
Problem
Every generation's input was built from the entire history of all earlier turns, and each input is serialized onto its own span. Memory therefore grew with steps × session size. On long sessions, the Stop hook crashed with a V8 heap OOM. That crash is a hard
abort()that skips the fail-open try/catch, and it happens beforemarkTurnUploaded, so every later Stop hook had an even bigger backlog to process and crashed too.Fix
max_history_chars(default200000), set inlangfuse.jsonor withLANGFUSE_CODEX_MAX_HISTORY_CHARS. For each turn, the history from earlier turns is built once. It walks back from the most recent message until the budget is used, and replaces everything older with a single[N earlier message(s) omitted …]system marker.codex.history_omitted_messagesmetadata, so the loss of detail is visible in the trace.[...seen]). That is replaced by per-turn message arrays and a bounded history computed lazily, only for turns that are actually emitted.Verification
pnpm test: 125 tests pass, including new ones for the truncation and the marker/metadata and for config parsing.pnpm lintpasses.mainmainNot in this PR
Some follow-ups from the issue are left out: flushing between pending turns, and capping individual tool outputs. Memory within a single very long turn still grows with that turn's own steps.
Linear Issue: LFE-17701