Repository navigation
OpenAI Responses GPT-5.6+ cache the stable prefix for 12 hours - #1698
Merged
Merged
Conversation
Put ttl 12h on the existing explicit prefix breakpoint and keep the request default at 30m. Chat Completions, Codex, and older models stay byte-identical. GRAFF_OPENAI_CACHE_TTL selects 12h (default), 30m, or off. Usage parses cached_tokens and the 12h/30m write split. The retention probe builds those bodies through Agent.buildBody and records later cached_tokens in bench/openai-cache-retention.json. Co-authored-by: Rach <justrach@users.noreply.github.com>
…R 0312) Renumber the ADR to 0312 (0298 is taken on main). The default stays 30m, ADR 0009's bytes, until a live probe records that OpenAI accepts prompt_cache_breakpoint.ttl "12h" and keeps the entry: no write was ever sent, and an unsupported value is a 400 on every request. GRAFF_OPENAI_CACHE_TTL=12h opts in.
justrach
marked this pull request as ready for review
November 2, 2026 02:23
justrach
added a commit
that referenced
this pull request
Nov 2, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OpenAI's prompt-caching guide (2026-10-20) added a 12-hour minimum lifetime on the Responses API for GPT-5.6 and later:
Unsupported values return
400 invalid_request_error.prompt_cache_retentionstays the older-model knob. Cache reads stay 0.1× (0.05× on GPT-6.1 Sol).The Responses API reference still documents
ttlas"30m"only. The guide is the contract this change follows.Target
main. The newest existingrelease/v*branch isrelease/v0.302.42and is behind tags that already landed onmain(through v0.302.48).Change
prompt_cache_breakpoint.ttl = "12h"on the stable prefix (system/developer + tools). The request keepsprompt_cache_options.ttl = "30m"for session context. The current user turn is never marked.GRAFF_OPENAI_CACHE_TTL/openai_cache_ttlselect12h(default),30m(previous bytes), oroff.prompt_cache_keyis unchanged.cached_tokens,cache_write_tokens, andcache_write_tokens_details(ttl_12h_tokens/ttl_30m_tokens). 12-hour writes bill the extra 1.25× on top of the existing 1.25× write price.ADR 0298. ADR 0009 updated.
Retention probe
scripts/openai_cache_retention_probebuilds both prefixes throughAgent.buildBody(the OpenAI provider path), then POSTs them.write— unique-nonce 12h prefix + 30m control, then an immediate resend--check— resend the saved prefixes later (expect a 12h hit at 1h / 3h / 6h / ~11.5h; a 30m hit after ~1h is recorded, not a failure)Pinned key:
graff-openai-cache-retention-v1State (committed, no secrets):
bench/openai-cache-retention-state.jsonNonces:
graff-12h-20261021T100354Z/graff-30m-20261021T100354ZLog:
bench/openai-cache-retention.jsonEnv-gated Zig tests:
GRAFF_OPENAI_CACHE_RETENTION=materialize|write|check.Tests
scripts/eval-tier1.sh --only fmt,lines,spec,reach,build,testsgreenLive smoke
OPENAI_API_KEYwas unset in this environment. The probe materialized both request bodies only (status: not_run). No Responses POST was sent. Re-runwritewith a key, then--checkat 1h / 3h / 6h / ~11.5h against the committed state file.Do not merge until a live write + later
--checkis recorded.