Portable provider transcript items for language-model clients.
The library defines the item vocabulary shared by stateless provider
transcripts: type predicates over decoded Responses and Chat
Completions items, family privacy for encrypted reasoning, native
compaction checkpoints and server-executed calls, canonicalization of
legacy compaction types, and constructors for user messages and
function-call outputs. Items are ordinary EQUAL hash tables as
decoded from provider JSON.
The name is a portmanteau of Common Lisp and clanker.
- ASDF system:
clinker-transcript - Test system:
clinker-transcript/tests - Package:
CLINKER-TRANSCRIPT - Nickname:
TRANSCRIPT
(transcript:family-private-item-p item)
(transcript:native-compaction-item-canonicalize item)
(transcript:item-assistant-text item)
(transcript:user-message-item "hello" leading-content-items)
(transcript:function-call-output-item "call-1" "ok")
(transcript:input-image-item "data:image/png;base64,..." :detail "high")
(transcript:input-text-item "caption")FAMILY-PRIVATE-ITEM-P is true for reasoning items, Chat Completions
thinking content, native compaction checkpoints, and server-executed
search and tool-search calls: items that only the producing model
family can replay. Transcript stores use it to filter projections when
a conversation moves between families.
Use MAKE-PROJECTION with :items to initialize an ordered projection from a
finite proper list or vector. Append with PROJECTION-APPEND and obtain a fresh
chronological list with PROJECTION-ITEMS. Item objects are shared, but collection
storage is independent. The implementation uses Structlisp deques.
Use PROJECTION-METADATA-TABLE with an application-defined key to obtain an EQ
item-metadata table. Whole-projection replacement with PROJECTION-REPLACE
retains metadata only for retained item identities, including explicit NIL
values. Obtain fresh metadata tables after replacement. Prepare replacements
before publication and provide caller-side exclusion for concurrent access.
Use ITEMS-FOR-FAMILY with :item-families and :handoff-families EQ tables to
filter private content and omit a portable handoff for its native family.
Before publishing a live function call, use VALIDATE-FUNCTION-CALL with
:preceding-items to reject missing or duplicate call identifiers.
(let ((plan (transcript:reconcile-items items
:repaired-output-p #'repair-output-p)))
(transcript:reconciliation-repairs plan)
(transcript:reconciliation-items plan :repair-output #'publish-repair-output))RECONCILE-ITEMS validates the entire input before returning a plan. Keep each
contiguous call group together, followed by outputs in call order. Correlated
outputs are moved beside their calls; orphan outputs retain their positions.
Duplicate calls and outputs signal RECONCILIATION-ERROR with a reason, offending
item and optional call identifier. With :repaired-output-p, tolerate exactly one
late output after a recognized repair only when the repair follows its call.
Keep that first output because later transcript items may depend on it.
Inspect RECONCILIATION-REPAIRS for ordered MISSING-OUTPUT-REPAIR intents. Each
intent exposes its call and call identifier. Resolve intents using the
:repair-output callback to RECONCILIATION-ITEMS. Supply wording and durably
publish a matching output in that callback before returning it. With no callback,
handle the missing-output condition with the USE-OUTPUT restart.
Install the returned item list only after materialization succeeds. A failed callback leaves the plan unchanged. After partial publication, read the durable state and build a new plan to avoid publishing an already completed repair twice. The library neither executes tools nor publishes records; record schemas, paths, leases, ownership and publication policy belong to the host.
Capture a COMPACTION-PLAN before requesting a summary or native checkpoint.
CUTOFF is an exclusive item count, defaulting to the captured projection length.
Inspect calls without outputs at that cutoff using COMPACTION-PLAN-UNRESOLVED-CALLS.
A later output does not change this captured fact. COMPACTION-PLAN-ITEMS returns
an independent chronological list of shared retained items: the earliest
unresolved call group with adjacent private context, its necessary call/output
context, and every post-cutoff item. Fully resolved prefixes can be discarded.
(let* ((plan (transcript:make-compaction-plan
projection :cutoff captured-count :repaired-output-p #'repair-output-p))
(replacement
(transcript:compaction-plan-projection
plan :replacement-items summary-or-checkpoint-and-handoff
:additional-items arrivals-since-capture
:repair-output #'interrupted-output
:family target-family
:item-families producing-families
:handoff-families native-handoff-families)))
(transcript:projection-items replacement))Supply chronological arrivals since capture, including outputs and steering, as
:additional-items. Validate them against all captured correlations before
filtering private content or calling repair policy. An actual arriving output
resolves its call without invoking :repair-output. Retain additional earlier
context when a late correlated output requires it. Use the same durable-repair
predicate as reconciliation to recognize one late result following a repair.
Supply a summary or native checkpoint and portable handoff as :replacement-items.
When :family is supplied, select private content and handoffs using the existing
ITEMS-FOR-FAMILY policy, then reconcile the selected transcript. Without
:family, retain private items for later host-controlled family projection.
The :repaired-output-p materialization argument defaults to the captured predicate.
COMPACTION-PLAN-PROJECTION returns a separate projection only after validating
the replacement and every supplied repair. Retained item identities keep copied
metadata tables, including explicit NIL values. The source projection and plan
are unchanged on success or failure. Provide exclusion while capturing and
publishing; item references and metadata values are shared and must not be
mutated by callbacks. The library performs no I/O, publishes no records and
executes no tools. Choose truthful interrupted/unknown output content in the
caller, and publish the replacement only after all host-side checks succeed.
./script/bootstrap
./script/check
./script/check --fix./script/check runs the tests, then checks the source with
cheque as cheque.sexp
configures it. --fix skips the tests and repairs what cheque can repair.
cheque and its tools are development dependencies: checks/qlfile lists them,
and the check installs them on first use into ~/.cache/cheque/tools/.
COLL-Attribution. See LICENSE.lisp.