feat(core): published definitions are integration types like code; code can embed one; workspaces own their own

docs/architecture-decisions/063-integration-definitions-as-data.md

@@ -56,6 +56,9 @@ Frigg supports **two authoring forms behind one runtime contract**:
 
 Code is the escape hatch and data is the composition. A data definition never contains code. It names building blocks that exist as code, and the app has registered and versioned them.
 
+> §1–§6 are the proposal as first written. Where they differ from what was built (step and expression vocabulary,
+> the document's shape, `name` vs `id`, routes), §7 onward is what runs.
+
 ### 1. What is data and what stays code
 
 | Concern | Data-defined | Stays code (referenced by name + version) |
@@ -421,6 +424,26 @@ does not exist at runtime, so "disabled but still firing" cannot happen.
 Shape B is what actually makes §4's "constant infrastructure" true. Shape A still grows the stack per definition
 id, just not per version.
 
+**Neither shape is visible from outside.** Rule: nobody can tell how an integration was made, not a user, an API
+caller or a log reader. Shape B's class is a *provider of integration types* (`static IntegrationProvider`,
+`integrations/integration-types.js`): wherever Frigg turns `config.type` into a class (the integration use cases,
+the queue worker, `GET /api/integrations` options), a published definition id resolves to a generated subclass named
+by that id, with the definition's label and modules. So an integration is installed with `{ type: 'phone-crm' }`,
+exactly as code is, and its logs, telemetry, schedule job ids and webhook URLs carry `phone-crm`. Registered code
+classes win over a definition with the same name (logged once per process as `frigg.integrations.definition.shadowed_by_code`). Only the deployment is shared: `IntegrationBase.getDeploymentName()`
+is the host's name for a generated class, so it queues and books one-time jobs on the host's queue, and its
+triggers run on the host's tick (§15a); the host's webhook function
+also serves `/api/{integrationType}/webhooks[/{integrationId}]` (HTTP API cannot match part of a path segment, and
+literal routes of registered integrations win); app-level webhooks, one URL per vendor app, use the host's URL and fan
+out to installations of every type it provides.
+
+**Code uses the same runtime.** `extendIntegrationClass(CodeClass, { definition, manifests })` is the third shape: a
+hand-written class that embeds a definition keeps its name, modules and handlers and gains workflows (with pauses),
+syncs, UI surfaces, the generated form and run history. Its code steps may call the class's own methods (`#method`),
+its own queued events are recorded as runs beside its workflows, and whatever the definition does not declare
+(webhooks, the form, schedules) stays the code's. So the capabilities that once belonged to data-defined integrations
+belong to integrations; how one was written is not something its users, API callers or logs can tell. `{ type: 'data-defined', definition: { id } }` still installs.
+
 ### 14. Runtime: how an event loads and runs
 
 **Cold start (unchanged).** `loadAppDefinition()` registers the generated classes like any others. No definition is
@@ -510,25 +533,36 @@ claimed once (§15a). Steps such as
 | `GET …/actions`, `POST …/actions/:id/options` | `GET_USER_ACTIONS`, `GET_USER_ACTION_OPTIONS` | Button workflows and backfills; a button's inputs are its options form |
 | `POST …/actions/:id` | any registered event | Runs a button workflow (`WF_*`), a backfill (`SYNC_*_BACKFILL`) or renders a UI surface (`UI_*`, body `{ record }`) |
 
-Data-defined apps add a few routes of their own (`router.js`, mounted by `createIntegrationRouter` only when a
-registered class is data-defined). These are the only new routes, and every one is authenticated:
+Apps that register the runtime add a few routes of their own (`router.js`, mounted by `createIntegrationRouter` when
+a registered class carries `DataDefined`). Every one is authenticated, and the installation routes answer for every
+integration however it was made: one with no run history, syncs or other versions gets `[]`, or a plan with nothing
+to move to, never a 404 for being code.
 
 | Route | Who | What |
 |---|---|---|
-| `GET /api/data-defined/definitions` | signed-in user | The install catalogue: the latest published version of each definition |
+| `GET /api/data-defined/definitions` | signed-in user | The published ones with their counts (`GET /api/integrations` already lists them as options beside code) |
 | `GET /api/integrations/:id/upgrade?to=` | signed-in user, own installation | The upgrade plan (`planUpgrade`); changes nothing |
 | `POST /api/integrations/:id/upgrade` `{ to? }` | signed-in user, own installation | Applies the plan: uninstall steps of the old version, then install steps of the new one if it asks for a reinstall, then the new config and a reload. Answers 409 with the plan when a person must supply something first |
 | `GET /api/integrations/:id/runs` | signed-in user, own installation | `WorkflowRun` history, newest first (`workflowId`, `status`, `before`, `limit`) |
-| `POST /admin/integrations/definitions` `{ document, notes?, publish?, strict? }` | `x-frigg-admin-api-key` | Validate and publish (or save a draft). 201 created, 200 for an identical republish, 400 invalid, 409 refused |
-| `GET /admin/integrations/definitions/:id` | admin key | Every version, without documents |
-| `POST /admin/integrations/definitions/:id/versions/:version/deprecate` | admin key | `PUBLISHED` → `DEPRECATED` |
+| `GET /api/integrations/:id/syncs` | signed-in user, own installation | Each sync's backfill progress and poll marks (§15b) |
+| `POST /api/integrations/:id/runs/:runId/decision` `{ decision, note? }` | the run's approvers, or the owner when it names none | Decide a paused approval (§15c). Answers 202 and carries the run on through the queue. An approver need not own the installation; anyone else gets a 404 |
+| `POST /api/integration-definitions` `{ document, notes?, publish? }` | signed-in user | Publish (or draft) an integration of the user's own workspace: listed and installable there, invisible elsewhere. 409 `ID_TAKEN` for an id owned by the app's catalogue or a code integration (both public); an id owned by another workspace answers 404 `NOT_FOUND`, identical to reading it, so ids cannot be probed (ADR-052 §3) |
+| `GET /api/integration-definitions/:id` | signed-in user, own workspace | Every version (drafts too) and the newest document, for the editor; 404 for anyone else |
+| `POST /admin/integrations/versions` `{ document, notes?, publish?, strict? }` | `x-frigg-admin-api-key` | Validate and publish (or save a draft). 201 created, 200 for an identical republish, 400 invalid, 409 refused |
+| `GET /admin/integrations/:id/versions` | admin key | Every version, without documents |
+| `POST /admin/integrations/:id/versions/:version/deprecate` | admin key | `PUBLISHED` → `DEPRECATED` |
 
 The admin routes live in core's existing `/admin/*` namespace, beside `/admin/scripts` and `/admin/db-migrate`, behind the same
-`validateAdminApiKey` middleware (`handlers/middleware/admin-auth`). They are deliberately not under `/api/admin`, which would be a
-third namespace. The user routes stay where they are until the `/api/v2` route registry (PR #671) lands; they move then.
+`validateAdminApiKey` middleware (`handlers/middleware/admin-auth`); `/api/admin` would have been a third namespace. The user routes
+stay where they are until the `/api/v2` route registry (PR #671) lands. Every route the router serves is listed once, with method,
+path and auth, in the exported `DEFINITION_ROUTES`, so that move is a data change.
+
+**Follow-up: these routes move to `/api/v2`.** The paths above are v0.1's. [ADR-053](./053-management-api-versioning.md) puts every new Management API resource under `/api/v2/`, and the route registry in #671 is how they get there. Moving them is a separate change, and the paths in this table will change with it.
+
+**What "workspace" means in this table.** The workspace routes (`/api/integration-definitions`) scope a definition to the authenticated user: the workspace id is `user.getId()`. That is a stand-in, not an ADR-056 owner and organization. Aligning it with ADR-056 (owner plus `organizationId`, `visible(context)`, binding scope) is the follow-up in §5.
 
-Installing needs no new route. `POST /api/integrations` with `config.definition: { id }` pins the latest published
-version, and its checksum, in `onCreate`. An app that bundles its definitions (`memoryDefinitions`) has no store to
+Installing needs no new route. `POST /api/integrations` with `config: { type: '<definition id>' }` pins the latest
+published version, and its checksum, in `onCreate`. An app that bundles its definitions (`memoryDefinitions`) has no store to
 publish to, so the admin routes answer 501.
 
 A host renderer (the HubSpot card) therefore calls Frigg through an authenticated path. Proving the host request is
@@ -574,6 +608,11 @@ or on a polling sync (`ongoing: { "mode": "poll", "every", "configurable" }`). `
 trigger entries (`compiled.triggers`), and the data-defined class resolves them per installation: it declares
 `Definition.instanceTriggers: { tick: '5m' }`, and its static `triggersFor({ config })` compiles the installation's
 pinned definition with its config at each tick. Each entry's handler runs its compiled event.
+The shared class's tick also covers the integration types it provides (`static triggerTypes()`): a published
+definition's installations carry their own `config.type`, so the tick pages each type in turn, and a continuation
+cursor names the type it stopped in. A code class that embeds a definition (`extendIntegrationClass`) keeps its own
+`Definition.triggers` beside the compiled ones, and declares `instanceTriggers` only when its definition has
+schedule-triggered workflows or polling syncs.
 
 | Layer | What | File |
 |---|---|---|

packages/core/handlers/integration-event-dispatcher.js

@@ -19,9 +19,13 @@ class IntegrationEventDispatcher {
     }
 
     async dispatchJob({ event, data, context, delivery }) {
-        return this._dispatch(event, (instance, handler) =>
-            handler.call(instance, { data, context, delivery })
-        );
+        return this._dispatch(event, (instance, handler) => {
+            const invoke = () => handler.call(instance, { data, context, delivery });
+            // An integration that keeps run history records each queued event as a run (data-defined runtime).
+            return typeof instance.runEvent === 'function'
+                ? instance.runEvent(event, invoke)
+                : invoke();
+        });
     }
 
     /**

packages/core/handlers/routers/integration-webhook-routers.js

@@ -24,55 +24,43 @@
     }
 
     const router = Router();
+    const receive = async (req, res, next, integrationId) => {
+        try {
+            const dispatcher = new IntegrationEventDispatcher(new IntegrationClass());
+            const dispatch = () => dispatcher.dispatchHttp({ event: 'WEBHOOK_RECEIVED', req, res, next });
+            // Logs only: the id comes from an unauthenticated URL, so it must not reach the telemetry context that
+            // drives usage attribution.
+            await (integrationId ? runInContext({ [LOGGER_SCOPE_KEY]: { integrationId } }, dispatch) : dispatch());
+        } catch (error) {
+            next(error);
+        }
+    };
     const basePath = `/api/${IntegrationClass.Definition.name}-integration/webhooks`;
 
     console.log(
         `\n│ Configuring webhook routes for ${IntegrationClass.Definition.name}:`
     );
 
     // General webhook route (no integration ID)
-    router.post(basePath, async (req, res, next) => {
-        try {
-            const integrationInstance = new IntegrationClass();
-            const dispatcher = new IntegrationEventDispatcher(
-                integrationInstance
-            );
-            await dispatcher.dispatchHttp({
-                event: 'WEBHOOK_RECEIVED',
-                req,
-                res,
-                next,
-            });
-        } catch (error) {
-            next(error);
-        }
-    });
+    router.post(basePath, (req, res, next) => receive(req, res, next));
     console.log(`│ POST ${basePath}`);
 
     // Integration-specific webhook route (with integration ID)
-    router.post(`${basePath}/:integrationId`, async (req, res, next) => {
-        try {
-            const integrationInstance = new IntegrationClass();
-            const dispatcher = new IntegrationEventDispatcher(
-                integrationInstance
-            );
-            // Logs only: the id comes from an unauthenticated URL, so it must
-            // not reach the telemetry context that drives usage attribution.
-            await runInContext(
-                { [LOGGER_SCOPE_KEY]: { integrationId: req.params.integrationId } },
-                () =>
-                    dispatcher.dispatchHttp({
-                        event: 'WEBHOOK_RECEIVED',
-                        req,
-                        res,
-                        next,
-                    })
-            );
-        } catch (error) {
-            next(error);
-        }
-    });
+    router.post(`${basePath}/:integrationId`, (req, res, next) => receive(req, res, next, req.params.integrationId));
     console.log(`│ POST ${basePath}/:integrationId`);
+
+    // A provider (the shared runtime for published integration definitions) also receives the webhooks of every
+    // type it provides, at each type's own URL. Receipt has no database, so the type in the URL is not looked up
+    // here: the delivery is queued on this deployment and the worker hydrates the installation by its id.
+    if (IntegrationClass.IntegrationProvider) {
+        const providedPath = '/api/:integrationType/webhooks';
+        // Registered integrations keep their own routes (a local server mounts every router on one app).
+        const registered = new Set(integrationClasses.map((c) => `${c.Definition.name}-integration`));
+        const isProvidedPath = (req) => /^[A-Za-z0-9_.-]+-integration$/.test(req.params.integrationType || '') && !registered.has(req.params.integrationType);
+        router.post(providedPath, (req, res, next) => (isProvidedPath(req) ? receive(req, res, next) : next()));
+        router.post(`${providedPath}/:integrationId`, (req, res, next) => (isProvidedPath(req) ? receive(req, res, next, req.params.integrationId) : next()));
+        console.log(`│ POST /api/<provided type>-integration/webhooks[/:integrationId]`);
+    }
     console.log('│');
 
     handlers[`${IntegrationClass.Definition.name}Webhook`] = {

packages/core/handlers/routers/integration-webhook-routers.test.js

@@ -86,13 +86,28 @@ jest.mock('../app-definition-loader', () => {
         };
     }
 
+    // A provider: the shared runtime for published integration definitions.
+    class ProviderIntegration extends IntegrationBase {
+        static Definition = { name: 'runtime-host', version: '1.0.0', modules: {}, webhooks: { enabled: true } };
+        static IntegrationProvider = { resolve: async () => null, list: async () => [] };
+        constructor(params) {
+            super(params);
+            this.events = {
+                WEBHOOK_RECEIVED: {
+                    handler: async ({ req, res }) => res.status(200).json({ receivedBy: 'runtime-host', integrationId: req.params.integrationId ?? null, type: req.params.integrationType ?? null }),
+                },
+            };
+        }
+    }
+
     return {
         loadAppDefinition: () => ({
             integrations: [
                 WebhookEnabledIntegration,
                 AdvancedWebhookIntegration,
                 LoggingWebhookIntegration,
                 NoWebhookIntegration,
+                ProviderIntegration,
             ],
         }),
     };
@@ -151,6 +166,37 @@ describe('Integration Webhook Routers', () => {
         });
     });
 
+    describe('types a provider offers', () => {
+        const post = (path) =>
+            handlers['runtime-hostWebhook'].handler(
+                {
+                    version: '2.0',
+                    routeKey: 'POST /api/{integrationType}/webhooks/{integrationId}',
+                    rawPath: path,
+                    rawQueryString: '',
+                    headers: { 'content-type': 'application/json' },
+                    requestContext: { http: { method: 'POST', path }, requestId: 'r' },
+                    body: '{}',
+                    isBase64Encoded: false,
+                },
+                { awsRequestId: 'req-provided' }
+            );
+
+        it("receives a provided type's webhooks at that type's own URL", async () => {
+            const withId = await post('/api/phone-hubspot-integration/webhooks/int-5');
+            expect(withId.statusCode).toBe(200);
+            expect(JSON.parse(withId.body)).toEqual({ receivedBy: 'runtime-host', integrationId: 'int-5', type: 'phone-hubspot-integration' });
+            const appLevel = await post('/api/phone-hubspot-integration/webhooks');
+            expect(JSON.parse(appLevel.body)).toMatchObject({ receivedBy: 'runtime-host', integrationId: null });
+        });
+
+        it('leaves registered integrations and other paths alone', async () => {
+            expect((await post('/api/webhook-enabled-integration/webhooks/int-5')).statusCode).toBe(404);
+            expect((await post('/api/users/webhooks/int-5')).statusCode).toBe(404);
+            expect(handlers['webhook-enabledWebhook']).toBeDefined();
+        });
+    });
+
     describe('logger scope (ADR-048)', () => {
         const invoke = (path) =>
             handlers['logging-webhookWebhook'].handler(