Skip to content

OpenAI Responses GPT-5.6+ cache the stable prefix for 12 hours - #1698

Merged
justrach merged 2 commits into
mainfrom
cursor/openai-12h-prompt-cache-f7da
Nov 2, 2026
Merged

justrach merged 2 commits into
mainfrom
cursor/openai-12h-prompt-cache-f7da

Conversation

@justrach

Copy link
Copy Markdown
Owner

OpenAI's prompt-caching guide (2026-10-20) added a 12-hour minimum lifetime on the Responses API for GPT-5.6 and later:

In the Responses API, set prompt_cache_options.ttl to choose the minimum cache lifetime:
"30m" (default) — 30 minutes — 1.25× the uncached input-token rate
"12h" — 12 hours — 2.5× the uncached input-token rate

The request-level ttl applies to the implicit breakpoint and to explicit breakpoints without their own ttl. Set prompt_cache_breakpoint.ttl to override it for an individual explicit breakpoint.

OpenAI may retain entries longer than the requested minimum.

The minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later.

Unsupported values return 400 invalid_request_error. prompt_cache_retention stays the older-model knob. Cache reads stay 0.1× (0.05× on GPT-6.1 Sol).

The Responses API reference still documents ttl as "30m" only. The guide is the contract this change follows.

Target

main. The newest existing release/v* branch is release/v0.302.42 and is behind tags that already landed on main (through v0.302.48).

Change

  • Direct OpenAI Responses on GPT-5.6+ writes prompt_cache_breakpoint.ttl = "12h" on the stable prefix (system/developer + tools). The request keeps prompt_cache_options.ttl = "30m" for session context. The current user turn is never marked.
  • GRAFF_OPENAI_CACHE_TTL / openai_cache_ttl select 12h (default), 30m (previous bytes), or off.
  • Chat Completions, Codex, ChatGPT-new, xAI, and models before GPT-5.6 receive no new fields.
  • prompt_cache_key is unchanged.
  • Usage parses cached_tokens, cache_write_tokens, and cache_write_tokens_details (ttl_12h_tokens / ttl_30m_tokens). 12-hour writes bill the extra 1.25× on top of the existing 1.25× write price.

ADR 0298. ADR 0009 updated.

Retention probe

scripts/openai_cache_retention_probe builds both prefixes through Agent.buildBody (the OpenAI provider path), then POSTs them.

  • write — unique-nonce 12h prefix + 30m control, then an immediate resend
  • --check — resend the saved prefixes later (expect a 12h hit at 1h / 3h / 6h / ~11.5h; a 30m hit after ~1h is recorded, not a failure)

Pinned key: graff-openai-cache-retention-v1
State (committed, no secrets): bench/openai-cache-retention-state.json
Nonces: graff-12h-20261021T100354Z / graff-30m-20261021T100354Z
Log: bench/openai-cache-retention.json

OPENAI_API_KEY=... scripts/openai_cache_retention_probe write
OPENAI_API_KEY=... scripts/openai_cache_retention_probe --check

Env-gated Zig tests: GRAFF_OPENAI_CACHE_RETENTION=materialize|write|check.

Tests

  • scripts/eval-tier1.sh --only fmt,lines,spec,reach,build,tests green
  • Focused unit filters for the new cache/TTL/retention tests pass
  • Request-body matrix: model × API × setting (12h / 30m / off)

Live smoke

OPENAI_API_KEY was unset in this environment. The probe materialized both request bodies only (status: not_run). No Responses POST was sent. Re-run write with a key, then --check at 1h / 3h / 6h / ~11.5h against the committed state file.

Do not merge until a live write + later --check is recorded.

Open in Web Open in Cursor 

cursoragent and others added 2 commits October 21, 2026 10:05
Put ttl 12h on the existing explicit prefix breakpoint and keep the
request default at 30m. Chat Completions, Codex, and older models stay
byte-identical. GRAFF_OPENAI_CACHE_TTL selects 12h (default), 30m, or
off. Usage parses cached_tokens and the 12h/30m write split.

The retention probe builds those bodies through Agent.buildBody and
records later cached_tokens in bench/openai-cache-retention.json.

Co-authored-by: Rach <justrach@users.noreply.github.com>
…R 0312)

Renumber the ADR to 0312 (0298 is taken on main). The default stays 30m, ADR 0009's bytes, until a live probe records that OpenAI accepts prompt_cache_breakpoint.ttl "12h" and keeps the entry: no write was ever sent, and an unsupported value is a 400 on every request. GRAFF_OPENAI_CACHE_TTL=12h opts in.
@justrach
justrach marked this pull request as ready for review November 2, 2026 02:23
@justrach justrach mentioned this pull request Nov 2, 2026
@justrach
justrach merged commit 6b60df4 into main Nov 2, 2026
6 checks passed
@justrach
justrach deleted the cursor/openai-12h-prompt-cache-f7da branch November 2, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants