Skip to content

Add TrueFoundryAgentStore for agents - #560

Open
bhaveshpatel640 wants to merge 11 commits into
mainfrom
AGE-2064
Open

Add TrueFoundryAgentStore for agents#560
bhaveshpatel640 wants to merge 11 commits into
mainfrom
AGE-2064

Conversation

@bhaveshpatel640

@bhaveshpatel640 bhaveshpatel640 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Wire agent create/update/delete to ServiceFoundry in TrueFoundry mode via TrueFoundryAgentStore (SF-first, remote id in external_id). Standalone stays DB-only.

Linear: AGE-2064

Changes

  • TrueFoundryAgentStore + putRemoteAgent / deleteRemoteAgent / toPutRemoteAgentPayload
  • Shared SF client; resolveAgentStore (same pattern as models/MCP)
  • Unit tests; patch changeset; map* / to* convention in src/truefoundry/AGENTS.md

How was this tested?

  • Unit: tests/unit/truefoundry/TrueFoundryAgentStore.test.ts (create/update/delete + failure paths)
  • Related router/deps tests updated for resolveAgentStore
  • tsc --noEmit for @truefoundry/trueforge

Checklist

  • I have read the contributing guidelines
  • pnpm build, pnpm test, pnpm typecheck, pnpm lint:ci, and pnpm format:check pass locally
  • Tests added/updated where it makes sense
  • No hand-edits to generated code (packages/trueforge-sdk, .github/fern/openapi/openapi.json, docs/openapi.json) — fork PRs omit SDK regen; maintainers regenerate after merge
  • Docs / .env.example updated if configuration or behavior changed (N/A — no new env; AGENTS.md mapper convention only)

Note

Medium Risk
Dual-write orchestration between the database and ServiceFoundry on every mutating agent path can leave partial state if cleanup fails; Postgres locking mitigates concurrency but distributed failures remain possible.

Overview
In TrueFoundry mode, agent CRUD goes through a new TrueFoundryAgentStore decorator: local DB first on create, then PUT /internal/tfg/agents, persisting the returned id in external_id; manifest updates and deletes sync remotely with rollback/cleanup on failure. Postgres uses a transaction-scoped advisory lock so concurrent update/delete cannot desync ServiceFoundry and the DB.

Wiring matches models/MCP: resolveAgentStore (request token → decorator; scheduler/no context → DB only), a shared ServiceFoundry client, and 10s HTTP timeouts on all ServiceFoundry calls (refactored via a shared JSON request helper).

listAgents now accepts optional external_ids filtering in SQLite/Postgres stores. tfg and trueforge are reserved agent names on create (AgentNameReservedError → HTTP 400). Routers, tests, and OpenAPI bootstrap are updated for resolveAgentStore.

Reviewed by Cursor Bugbot for commit 2b721e6. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 2b721e6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@truefoundry/trueforge Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@bhaveshpatel640
bhaveshpatel640 marked this pull request as draft September 2, 2026 19:22
Comment thread packages/trueforge/src/truefoundry/TrueFoundryAgentStore.ts
@bhaveshpatel640
bhaveshpatel640 marked this pull request as ready for review September 2, 2026 19:52
@bhaveshpatel640
bhaveshpatel640 marked this pull request as draft September 2, 2026 19:52
Base automatically changed from AGE-2083 to main September 3, 2026 05:50
@bhaveshpatel640
bhaveshpatel640 marked this pull request as ready for review September 3, 2026 06:06
@bhaveshpatel640
bhaveshpatel640 marked this pull request as draft September 3, 2026 06:06
@bhaveshpatel640 bhaveshpatel640 changed the title Add TrueFoundryAgentStore for SF remote agent sync Add TrueFoundryAgentStore for agents Sep 3, 2026
@bhaveshpatel640
bhaveshpatel640 marked this pull request as ready for review September 3, 2026 08:33
Comment thread packages/trueforge/src/apis/agents.ts
Comment thread docs/openapi.json Outdated
cursor[bot]

This comment was marked as outdated.

Comment thread packages/trueforge/src/truefoundry/toPutRemoteAgentPayload.ts Outdated
import { toPutRemoteAgentPayload } from './toPutRemoteAgentPayload';
import { TrueFoundryServiceFoundryServerClient } from './TrueFoundryServiceFoundryServerClient';

function asError(value: unknown): Error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you ask Cursor to check if this can be a common pattern across TrueForge?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a common pattern

constructor(input: {
inner: IAgentStore<TTransaction>;
client: TrueFoundryServiceFoundryServerClient;
accessToken: string;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chiragjn I feel this should always be a callback.
Later we can implement refresh logic here.

throw new AgentNameConflictError({ tenant_id: input.tenant_id, name: input.name });
}

const { remoteAgentId } = await this.#client.putRemoteAgent({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code, it is not clear to me whether all this is happening within the transaction or not.

Comment thread packages/trueforge/src/truefoundry/TrueFoundryAgentStore.ts Outdated
Comment thread packages/trueforge/src/truefoundry/TrueFoundryAgentStore.ts Outdated
// Race: peer create won the name and owns this remote (1:1) — do not delete it.
if (!(error instanceof AgentNameConflictError)) {
try {
await this.#client.deleteRemoteAgent({ accessToken: this.#accessToken, remoteAgentId });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this worth it? How do you plan to roll back if we have an issue at commit time?

throw new AgentNameConflictError({ tenant_id: input.tenant_id, name: input.name });
}

const { remoteAgentId } = await this.#client.putRemoteAgent({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit worried about what happens in a race situation. How do we know which put call won here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible,

  1. Agent A (mcp: a, b) and Agent A (mcp: c) arrive. Note that the name is same.
  2. In TrueFoundry, we save (mcp: c)
  3. In TrueForge, we save (mcp: a, b).

assertAgentNameNotReserved(input.name);

// SF PUT upserts by name — skip if local name exists (avoids overwrite/delete of e.g. research→sf-1).
const existing = await this.#inner.getAgent({ tenant_id: input.tenant_id, name: input.name }, transaction);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you not need a lock here to protect the system properly?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out advisory locks in Postgres once and see if you can come up with a solution. Think about contention as well.

This is a non-issue in standalone mode.

@bhaveshpatel640 bhaveshpatel640 Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for Create:

Approach A — Insert first

  • create: createDB(external_id=null) → putRemote → updateDB(external_id) | on put/update fail → deleteDB (+ deleteRemote if put ok) | cleanup+fail → AggregateError
  • Pros:
    • Fixes same-name MCP desync (loser never calls SF)
    • No DB lock while waiting on SF
    • Works in Postgres and SQLite
    • No name-check read needed — unique insert picks the winner (0 getAgent reads on the happy path)
  • Cons:
    • Two DB writes per create
    • Short window where external_id is null
    • Rollback means deleting the local row (and remote if put already ran)

Approach B — Advisory lock

  • create: lock(name) → check name → putRemote → createDB(external_id) → unlock | on DB fail → deleteRemote | both fail → AggregateError
  • Pros:
    • Fixes same-name desync if the lock covers the SF call
    • One DB insert; external_id set immediately
    • Clear “one create at a time” per name
    • Only 1 SF put for a same-name race (waiter re-checks under lock and skips put)
  • Cons:
    • Lock held across SF HTTP (can wait seconds) → contention on same name
    • More wiring (txn + lock; SQLite needs a no-op path)
    • Extra name-check read under the lock (winner and waiter each do 1 getAgent)

Preferring A for create same race fixed, no lock across the network, Two DB writes per create agents & Short window where external_id is null

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for Update:

Approach A — DB first

  • update: get → updateDB(manifest) → putRemote → updateDB(external_id if changed) | on put fail → updateDB(previous) | cleanup+fail → AggregateError
  • Pros:
    • DB is updated before SF; SF is not called if the local write fails
    • No DB lock while waiting on SF
    • Works in Postgres and SQLite
    • Same local-first shape as create
  • Cons:
    • Does not fix concurrent update MCP desync (both updates can succeed on the same row, then both call SF)
    • Often two DB writes when external_id changes (manifest + id); restore needs another write on put failure
    • Short window where DB is ahead of SF
    • Rollback means rewriting the previous manifest (row is kept)

Approach B — Advisory lock

  • update: lock(name|id) → get → putRemote → updateDB → unlock | on DB fail → putRemote(old) | both fail → AggregateError
    (or lock → get → updateDB → putRemote → unlock)
  • Pros:
    • Fixes concurrent update desync if the lock covers the SF call
    • Clear “one update at a time” per agent
    • Only 1 SF put for a contended update (waiter runs after unlock with a fresh get)
  • Cons:
    • Lock held across SF HTTP (can wait seconds) → contention on hot agents
    • More wiring (txn + lock; SQLite needs a no-op path)
    • Extra get under the lock for waiters

Prefer B here

@bhaveshpatel640 bhaveshpatel640 Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative using Redis:

  • Use Redis as the mutex for (tenant_id, agent_id). Postgres is only used for short reads/writes. HTTP never runs under a PG txn or advisory lock.
  • Redis — update / delete (by agent id)
    • Mutex key: tf:agent-update:{tenant_id}:{id}
SET key <token> NX EX 30          ← acquire (retry until 30s)
  getAgent                        ← short PG read (auto-commit / caller txn only if passed)
  putRemote / deleteRemote        ← SF HTTP; no open PG txn, no pool pin
  updateAgent / deleteAgent       ← short PG write
EVAL compare-and-del(token)       ← release only if we still own the key

Flow with redis lock:

  • create: createDB(null) → lock → putRemote → updateDB(external_id) | on put/update fail → deleteDB (+ deleteRemote if put ok)
  • update: lock → get → putRemote(new) → updateDB | on DB fail → putRemote(old) | both fail → AggregateError
  • delete: lock → get → deleteRemote(404 ok) → deleteDB

@bhaveshpatel640
bhaveshpatel640 marked this pull request as draft September 3, 2026 09:17
@bhaveshpatel640
bhaveshpatel640 marked this pull request as ready for review September 3, 2026 10:58

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 67459f4. Configure here.

const key = `tf:agent:${input.tenant_id}:${input.id}`;
await sql`SELECT pg_advisory_xact_lock(hashtext(${key}))`.execute(trx);
return fn(trx);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lock holds txn during HTTP

Medium Severity

createPostgresAgentUpdateLock opens a Postgres transaction and updateAgent awaits putRemoteAgent (and restore) inside that callback, so ServiceFoundry HTTP runs while the txn and pg_advisory_xact_lock stay held. That violates the store rule that transaction callbacks do only local DB work. A pool connection can sit idle for the 10s SF timeout, and a commit/rollback after a successful put leaves ServiceFoundry and the DB out of sync.

Additional Locations (1)
Fix in Cursor Fix in Web

Triggered by project rule: @truefoundry/trueforge review rules

Reviewed by Cursor Bugbot for commit 67459f4. Configure here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread packages/trueforge/src/truefoundry/TrueFoundryAgentStore.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants