Repository navigation
feat(workflow): publish and resume workflow runs across machines (#1084) - #1280
Conversation
|
I couldn’t complete the exact-scope review: the checkout is shallow, and the requested three-dot diff fails with I did not substitute a two-commit diff, which would include unrelated changes, or run, build, or install PR code. The supplied PR description documents sufficient testing, including representative real-CLI verification; no missing-e2e finding is warranted. Please provide the exact diff or a checkout with the base history available. No code-review verdict is issued. |
cd5ae1d to
5c3860e
Compare
The PR description documents sufficient testing, including representative real-CLI verification. No missing-e2e finding applies. The earlier review’s shallow-history blocker is resolved: the requested three-dot diff is readable. I did not run, build, or install PR code. |
A member can publish a run to teamai-reports and continue it on another machine, or hand it to a teammate, with its workspaces and, when asked, its agent sessions. The session commands now write teamai-reports under its lock with exit codes that say when nothing was written, and the local driver stops a command step when it alone gets SIGTERM.
…ss origin A claim checked the run's branches on the business origin, and the local driver reconciled the copies once after it, whatever the run's status. After the owner finished a run and a merge deleted its teamai/<run>/* branches, another member's first resume failed with commit_missing instead of reporting how the run ended. Only a running run needs its workspaces. A claim checks them only then, and the driver reconciles a claimed run's copies once a retry reopens it.
622a112 to
988af80
Compare
|
No new findings I’m confident are real in the current diff.
|
Keep this branch's reports publish contract over Tencent#1291's: a write that returns null pushes nothing, an empty list retries the unpushed commit, and a refused session push fails the command. Tencent#1291's session retry test now expects that contract (exit 1 while origin refuses, then Pushed: <file>).
|
No new findings I’m confident are real in the current diff.
|
Tencent#1084 promises that a published run snapshot leaves out "prompts, secrets, unsafe outputs, and absolute machine paths", and that "snapshots contain no secret, prompt, private session content, or absolute machine path". The snapshot dropped the checkout and workspace owners, but copied inputs, outputs, results, and HITL notes as they were, so an input such as /Users/alice/private/data.csv reached teamai-reports. The check that refuses a secret now also refuses a value or key holding an absolute path of this machine, with local_path_detected naming the step and field. It counts a Windows drive or share path, a ~/ path, a file:// URL, and a / path whose first directory exists at this machine's root. A URL, a relative path, a route such as /api/v1, and text such as and/or pass. The value is not rewritten, since expressions read it.
The snapshot named its owner by the OS login, while the run records, labels, and stats on teamai-reports name the member by the configured TeamAI username. A run published by alice on a laptop whose login is a1234 showed a1234 in the snapshot and in every not_owner message, and alice in the run records. The owner now carries the username the run records use.
The snapshot check masked an environment value only when its name looked secret, such as GITHUB_TOKEN. A secret the team declares in env/secrets.yaml can have any name, so a run whose output held the value of WAREHOUSE_DSN was published as it was. collectTeamSecrets, next to resolveTeamEnv, returns the value of each declared secret as this member resolves it, keyed by name, and publish checks the snapshot against those values too. Plain env.yaml variables are not secrets and still publish.
Run records follow the usage reporting setting, while workflow publish writes the run's state with it on or off. That state is what another machine needs to continue the run, not a usage report. The guide now says so, and that a run stays off teamai-reports until it is published.
|
No new findings I’m confident are real in the current diff.
|
The failure-code test keeps both sides: main's namespace conflict diagnostics and this branch's run handover answers.
Resolved: The earlier completed-run claim bug is fixed; finished runs skip origin checks and copy reconciliation. Testing sufficient: The description includes representative real-CLI verification. Older verification commits are not blocking, and additional providers and agents are deferred to CI. Reviewed the requested three-dot diff using the trusted base rules. I did not run, build, or install PR code. |
publish dropped a value declared in env/secrets.yaml when it was shorter than the 8 characters that mark an environment value as secret, so a declared ACCESS_CODE=493827 in a run's inputs was published as is. A declared value is now checked whatever its length. One shorter than 8 characters matches only as a whole word, so 4938 does not match inside build-49381. One shorter than 4 cannot be told from ordinary text: it is not checked, and publish warns with its name instead of dropping it silently.
Resolved: The completed-run claim bug is fixed. The earlier six-character secret example is also fixed when represented as a string. Testing sufficient: The description documents representative real-CLI verification. Older verification commits are not blocking; additional providers and agents are deferred to CI. Reviewed the requested three-dot diff using the trusted base review rules. I did not run, build, or install PR code. |
publish checked only strings and keys, so a step output {"code":493827}
with a declared ACCESS_CODE=493827 was published as is, while "493827"
was refused. A number or boolean is now checked as its JSON text, by the
same secret and path checks as a string.
|
No new findings I’m confident are real in the current diff.
Reviewed the requested three-dot diff using the trusted base review rules. I did not run, build, or install PR code. |
Keeps main's migration results and archive lock ownership, with session migrate --push archiving through teamai-reports on top: the archive phase returns its failures as migration results. The local driver test awaits saveSession, which main made asynchronous.
|
No new findings I’m confident are real in the current diff.
Reviewed the requested three-dot diff using the trusted base review rules. I did not run, build, or install PR code. |
# Conflicts: # docs/guide/dashboard.md # docs/guide/zh-CN/dashboard.md # src/session-flow/session-cmd.ts # src/session-flow/session-migrate-cmd.ts
|
No new findings I’m confident are real in the current diff.
Reviewed the requested three-dot diff using the trusted base review rules. I did not run, build, or install PR code. |
|
No new findings I’m confident are real in the current diff.
Reviewed only the requested three-dot diff against the trusted base review rules. I did not run, build, or install PR code. |
# Conflicts: # src/workflow-commands.ts
|
No new findings I’m confident are real in the current diff.
Reviewed the requested three-dot diff against the trusted base review rules. I did not run, build, or install PR code. |
Summary
Lets a member hand a running workflow to another machine or teammate, for example when their tool runs out of tokens, or pick up a failed CI run. Part of #818.
teamai workflow publish <run> [--sessions]writes the run toworkflows/runs/<id>/state.jsononteamai-reports. The first publish makes this install the owner: a random id in<dataHome>/workflows/owner.json, plus the member'susernamefrom the team config for messages, the name run records and labels use. Publishing an unchanged run writes nothing. Another install's run answersnot_ownerbefore anything is pushed, drained or archived.snapshot: 1, parsed strictly. Every run and step field is classifiedpublish,droporrewritein a table the compiler checks, so a new field does not build until someone decides whether it may leave the machine. It carries what steps did and what expressions read, never driver identities, paths, workflow definitions or failure text. Commit ids are full ids, and branch names stay underteamai/<run>/. A value that looks like a secret, or equals a secret the team declares inenv/secrets.yaml, refuses the publish withsecret_detected, naming the step, field and label, never the value. A declared secret of 8 or more characters matches anywhere, one of 4 to 7 only as a whole word, and a shorter one is not checked: publish warns and names it. Numbers and booleans are checked as text. An absolute path of this machine (/Users/…,C:\…,\\server\…,~/…,file://…) in an input, output or note refuses it withlocal_path_detected, naming the field. URLs, repo-relative paths and route-like text such as/api/v1/userspublish.writeid on its own line, and the write lands only over the state it read (compare-and-swap inside the reports lock). A write whose push failed stays pending, and the next publish or claim adopts it. A run that changed under a publish answersstate_moved.teamai workflow resume <run>takes a run this machine does not have fromteamai-reports.--claimtakes it from another owner. Other drivers sendclaim: truein thenextpayload. A claim releases every lease and moves an in-flight lease that opened its own copy to the next pass. It rebuilds the workflow definitions at the run's team revision, and refuses withdefinition_mismatchwhen they differ. The resume command claims inside the driver lock, so a running driver answersdriver_runningwith nothing claimed.ls-remoteand pins the base asteamai/<run>/base. It never pushes workspace branches. A driver pushes every commit it reports, failed steps included, and fetches a lease commit it lacks. After a claim, the local driver reconciles its copies once: a clean copy that is behind fast-forwards, and a dirty, ahead, diverged or off-branch copy failsbranch_divergedwith nothing reset. Origin behind a recorded commit answerscommit_missingwith the push to run. A claim checks the branches, and the driver reconciles the copies, only while the run isrunning: asucceeded,failedorcancelledrun leases nothing until a retry reopens it, so another member can claim it after a merge deleted its branches, and the firstresumeafter the retry fetches them.publishbut without sessions, and a laternextor run control retries it after a crash. Once origin holds the end, nothing is checked or pushed again, so deleting the run's branches after a merge changes nothing. A claim of a finished run runs nothing again, also when its branches are gone. A member whose run was claimed meanwhile is told so, and neither the snapshot nor the end record changes.--sessionsputs the sessions the unsettled steps resume into the team session archive, scrubbed. The driver restores a session from the archive when this machine has no copy. On a git team it reads a temporary clone fetched from origin, and leaves the member's team clone as it is. A step that resumes a session in a copy reportsresumed, and the run records that the copy continues it, so the next step gets every turn on any machine. The lease also lists earlier sessions to fall back to, and a report'sresumedmust name one the lease offered. When this machine cannot read the newest session of a chain,--sessionsarchives the newest one it can and lists the ones it skipped.session push,session migrate --pushandsession pullwriteteamai-reportsunder its lock, and answer busy with exit 1. They first take the archive publication lock from #1334, so a second session command on the same machine waits for the first instead of failing busy. Session commands work on a user-scope install. A session from a workflow step's worktree is filed under the checkout it came from. A retriedsession save --pushpushes the summary a failed push left.max_parallel: 1, a SIGTERM or SIGHUP sent to the driver alone stops the running command and its children, and the run can be resumed.*_PATHvalues and plain words. Run record stats count per member. Reports commits use a resolvable identity. A failed agent-config reset or merge fails the step and names the files. Nested Copilot paths copy into a fresh worktree.Decisions
state.jsononteamai-reports, notstate.yamlas the issue sketchedwriteline when git rebases one onto the other.resumeis extended with--claim;publishis the only new commandpublish --sessions{ tool, id }references.teamai/<run>/baseteamai/<run>/*, so a base that only sits on a movedmainor in a shallow CI clone would otherwise be missing on the claiming machine.resumeneeds, not usage telemetry. A run stays offteamai-reportsas long as nobody publishes it.resolveRepoRootno longer prints and returns undefined itself.session migrate --jsonstill answersarchive_failed.Evidence
Head
e70a3339:mainmerged in again at6467c3d7(with #1265). The one conflict was insrc/workflow-commands.ts. #1265 passesteamConfig.toolPathsto the local driver instead ofscopedToolPaths(...), and this branch'sworkflow startandworkflow resume(under the driver lock) now do the same.tsc --noEmit, lint and build pass.vitest run: 515 files, 14999 tests pass (21 skipped). The six e2e files listed below pass again, 12 tests.At
3c335f2f:mainmerged in again at4b9e0000. The two conflicts were in tests only:workflow-engine.test.tsandworkflow-workspace.test.tskeep this branch'sresumableSessionsimport andsaveRunhelper next to main's_setDriverPollMs(20).tsc --noEmit, lint and build pass.vitest run: 505 files, 12556 tests pass (21 skipped). The six e2e files listed below pass again, 12 tests. The real CLI run below was not repeated at this head.At
ac82dde5:mainmerged in at12b80e7f(with #1334). Theteamai-reportsarchive step ofsession push,session migrate --pushandsession pullnow runs inside #1334's publication lock, and its saves get #1334's rollback. A new test holds that lock in a single-repo team and checks thatsession pushwaits for it. It fails on the branch before the merge, where the push finished while the lock was held.tsc --noEmit, lint and build pass.vitest run: 505 files, 12533 tests pass (21 skipped). e2esession-archive-lock,session-migrate-opencode,session-publication,workflow-publish-resume,workflow-records-reports-branch-1083andreports-writer-sync-561: 6 files, 12 tests pass. Real CLI atac82dde5(dist/index.js,teamai init .against a bare business origin, its own HOME, a Claude Code session file, no model runs), 13 checks passed:Before that, at
060f4483: short declared secrets and numeric values (a743d72a,16481355), andmainmerged in at6c57c3b7(with #1302:session migrate --pusharchives its copy onteamai-reports).vitest run: 499 files, 12478 tests pass (21 skipped). Real CLI: a declaredACCESS_CODE=493827ininputs.targetrefuses the publish withsecret_detectedand does not print the value.DOOR_PIN(42) is named in a warning as too short to protect.build-4938271publishes. A numeric output493827refuses withfield outputs.code. The previous head published both.migrate --pushin a single-repo team archives onteamai-reportsand leavesmainwithoutsessions/.Before that, head
412ea45a, four commits on0ee0c67c: absolute paths and the team's declared secrets refuse a publish, the owner is named by the teamusername, and the guide says publishing is not usage reporting.tsc --noEmitand lint pass at each commit.vitest run: 496 files, 12430 tests pass (21 skipped). e2eworkflow-publish-resume: 5 of 5. Real CLI at412ea45a(dist/index.js, two members with their own HOME, bare team and business origins, no model runs):Before that, at
0ee0c67cwithmainmerged in (with #1274, #1282, #1285 and #1291). #1291 fixed the same retry onmainwith a weaker contract; this branch's is kept: a write that returns null pushes nothing, an empty list retries the unpushed commit, and a refused session push fails the command. #1291's session retry e2e now expects that (exit 1 while origin refuses, thenPushed: sessions/alice/2026-09.md).tsc --noEmitand lint pass.vitest run: 496 files, 12420 tests pass (21 skipped). e2e:cli-error-exit,reports-writer-sync-561,workflow-publish-resumeandworkflow-conformancepass (4 files, 53 tests).Before that, at
988af80f, rebased ontomain(with #1251, #1252, #1268 and #1281):tsc --noEmit, lint and build pass.vitest run: 495 files, 12397 tests pass (21 skipped). e2e, the full suite (the branch touches modules 146 of 147 files reference): 169 files and 1430 tests pass (30 skipped).workflow-publish-resumenow has alice take back the run bob finished with her interrupted copy still dirty:resume --claimanswersdoneand leaves the copy as it was. The two terminal Ctrl-C cases ofworkflow-signalsfail here at the same setup wait as onmain; CI runs them.The rebase onto #1268 dropped the options
update()passed to the worktree open, so a claim from a team with noteamai-reportscreated the branch on origin;update()passes them again, andworkflow-handoffcovers it. The last commit carries the Windows test-list fixmainneeds forcodex-team-hooks.test.ts.Real CLI, before the rebase (the first row at
622a1121, the rest at5c3860eb; not rerun at988af80f;dist/index.js,init --self --provider gitwith a forge URL mapped to local bare origins, two or three members each with their own HOME and data home, a bare team origin and a bare business origin, a stubclaudethat records the session it resumes):succeeded, a merge deletesteamai/<run>/*on the business origin, then bob, who never had the run, runsworkflow resume <run>622a1121:donewithsucceeded, exit 0, bob owns the run at state revision 2. At5c3860eb:commit_missingonteamai/<run>/base, exit 1workspace: newcopy, stops it mid-run, publishes; bob runsworkflow resume <run>succeeded; the copy branch on the business origin holds both members' commits; alice'spublishthen answersnot_ownernaming bobresumeanswersbranch_divergednaming the worktree; the file never reaches the business originsucceeded; bob'sworkflow resume <run>runs nothing; the end record stays alice'sdonecarriesnotPublishedwithcommit_missingand the push to run; origin keeps the earlier snapshotworkflow resumepublishes itteamai/<run>/*branches were deleteddone, nothing published or pushed againsession: { from: plan }, alternating alice and bob withpublish --sessionsfinalresumes a session holding the plan, build and check turns; on one machine every step resumes the same session id--sessions; carol claims and publishes--sessionsskippedstate_movedkill -TERMto the driver pid with a running command step,max_parallel: 1workflow resumefinishes the runsession save --pushrefused by the team origin, then retriedPushed: sessions/alice/<month>.mdworkflow resume --claimwhile this run's driver is runningdriver_running;teamai-reportson origin unchangedLeft to CI: Codex, CodeBuddy, OpenCode, and the
gitlab/githubproviders. Windows signal handling was not run.Merge Danger
Door: two-way. Run files gain optional fields (
publication,sessionContinuations, and the session a step's lease offers); old run files read as before. The snapshot format is new.Blast Radius: workflows and session commands. Runs that are never published behave as before, apart from the driver signal fix and the session command locking and exit codes.
Related: #1267, #1270, #1277 (found during this work, outside its scope). #1268 is the separate reports git timeout fix.
Closes #1084