One source of truth for API keys in the orkait stack. Round-robin rotation, per-feature rate limits, monthly usage budgets, scoped consumers, bound secrets, and opt-in encryption at rest.
Docs: platform-docs.orkait.com (sign-in required)
Other services call keypooler to obtain a usable key for a feature. They never store, rotate, or decrypt credentials themselves.
caller ──► GET /key?feature=firecrawl_scrape ──► keypooler picks a key ──► { value, secrets, metadata }
│
├─ scope-filters to the caller's allowed tiers
├─ round-robins across the pool
├─ enforces per-feature rate + usage budgets
└─ audits the serve
| Capability | Detail |
|---|---|
| 🔄 Rotation | Round-robin across all keys in a tier; exhausted keys are skipped, the next is tried. |
| ⏱️ Rate limits | Per-feature, windowed (e.g. 10 calls / 60s). Defined on the tier, applied per key. |
| 📅 Usage budgets | Per-key usage_limit with an optional usage_window_seconds (e.g. 2592000 = monthly auto-reset). nil window = lifetime cap. |
| 💰 Spend budgets | Per-key budget in usd or credits, refilled each month on its reset_day (1-28) or lifetime without one. Whoever used a key reports what it cost (POST /key/{id}/spend, idempotent on request_id); a key at its budget is not served until the next reset. |
| 🔎 Provider balances | Every 15 minutes keypooler asks Firecrawl (/v1/team/credit-usage), Tavily (/usage) and Apify (/v2/users/me/limits) what each of their keys has used of its limit. The balance shows on GET /admin/keys; a key at its limit is skipped until the provider's own cycle end (Tavily gives none, so it is checked again next round). |
| 👤 Scoped consumers | Each client gets a bearer token scoped to specific tiers. The admin token is a superuser. |
| 🔗 Bound secrets | Extra named secrets travel with a key (e.g. a Firecrawl webhook_secret), returned at serve time. |
| 🔐 Opt-in encryption | Plaintext at rest by default; set ENCRYPTION_KEY to encrypt new writes. Self-tagged, so both coexist. |
| 📜 Audit | Every serve appends a usage_events row (key, consumer, feature, time). |
| ⏳ Expiry | Optional expires_at per key; expired keys leave the pool automatically. |
Keys live in tiers. A tier names the features it covers and the rate limit for each. Consumers are scoped clients that may only draw from the tiers they are granted. The admin token bypasses scoping.
flowchart LR
C[caller] -->|Bearer token| K{/key?feature=X}
K -->|admin token| ALL[all tiers]
K -->|consumer token| SC[scoped tiers only]
ALL & SC --> F[filter pool:<br/>active, unexpired, under usage limit,<br/>supports feature, has rate budget]
F --> RR[round-robin pick]
RR --> G[TryRate + TryConsumeUsage]
G -->|ok| R[return value + secrets + metadata]
G -->|none left| E[403 out-of-scope / 429 exhausted]
R -.async.-> A[(usage_events audit)]
Auth model
| Caller | /key access |
/admin/* |
|---|---|---|
| Admin token | every tier (superuser) | full |
| Consumer token | only granted tiers (401 unknown, 403 out-of-scope) |
denied |
🗄️ Data model (8 tables)
| Table | Holds |
|---|---|
tiers |
feature group + description |
tier_features |
per-feature rate_limit + window_seconds |
keys |
the key value (plaintext or enc:gcm:…), tier, expires_at, usage_limit, usage_window_seconds/start, metadata_json |
key_secrets |
named secrets bound to a key (e.g. webhook_secret) |
consumers |
scoped clients, bearer token stored sha256-hashed (shown once) |
consumer_scopes |
which tiers a consumer may draw from |
usage_events |
append-only audit, one row per serve |
spend_events |
one row per spend report, unique on request_id; keys carries budget_*, spent and spent_period_start |
Migrations are migrations/001_init.sql and 002_budgets.sql, each idempotent (IF NOT EXISTS). Deletes also clean up children explicitly, in one transaction, so a reused id can never re-grant an old scope.
⚡ The serve path: no database round trip
On every /key |
Where it lives |
|---|---|
| Keys, tiers, rate windows, usage gates | in memory, loaded at boot and on admin writes (internal/keypool) |
| Consumer token to its scopes | in memory for a minute; any admin write clears it, so a revocation holds on the next call (internal/api/authcache.go) |
| Usage counts and audit events | queued, then written every second and on shutdown: counts coalesced per key, events in one COPY (internal/writeback). A crash loses at most a second of them; a database outage keeps up to 10,000 events for the next flush |
Measured locally against Postgres 17: 2,000 consumer draws over one keep-alive connection at p50 0.028 ms, p99 0.081 ms; the 2,001 serves landed as usage_count = 2001 and 2,001 audit rows.
Storage is self-describing. Each value carries a scheme tag, so plaintext and encrypted rows coexist in one database and the mode can be switched on later with no migration.
ENCRYPTION_KEY unset (default) ──► key_value = "fc-2c7b07f…" (plaintext)
ENCRYPTION_KEY set (32-byte hex) ──► key_value = "enc:gcm:9f3a…" (AES-256-GCM, hex)
read: decrypt iff the value starts with "enc:gcm:" · a tagged value with no key is a hard error
The decision lives in one place (crypto.Sealer). Callers always receive the plaintext value from the API response and never need the key.
export ADMIN_TOKEN=$(openssl rand -hex 32)
# optional: export ENCRYPTION_KEY=$(openssl rand -hex 32) # omit for plaintext at rest
docker compose up -d postgres # local Postgres on 127.0.0.1:5432
export DATABASE_URL=postgres://keypooler:keypooler@127.0.0.1:5432/keypooler
go build -o keypooler ./cmd/keypooler # pure Go, no CGO
./keypooler📦 Register a tier, add a key, create a scoped consumer
A="Authorization: Bearer $ADMIN_TOKEN"
# 1. create a tier with two rate-limited features
curl -X POST localhost:8080/admin/tiers -H "$A" -H 'Content-Type: application/json' -d '{
"name": "firecrawl",
"description": "Firecrawl pooled free-tier accounts",
"features": { "firecrawl_scrape": {"rate_limit":10,"window_seconds":60},
"firecrawl_search": {"rate_limit":5,"window_seconds":60} }
}'
# 2. add a key with a monthly 1000-credit budget and a bound webhook secret
curl -X POST localhost:8080/admin/keys -H "$A" -H 'Content-Type: application/json' -d '{
"name": "firecrawl-01", "key": "fc-xxxx", "tier": "firecrawl",
"usage_limit": 1000, "usage_window_seconds": 2592000,
"secrets": { "webhook_secret": "whsec_xxxx" }
}'
# 3. create a consumer and scope it to the tier (token is shown ONCE)
CID=$(curl -s -X POST localhost:8080/admin/consumers -H "$A" -H 'Content-Type: application/json' \
-d '{"name":"my-service","description":"firecrawl via rotation"}' | jq -r .id)
curl -X POST localhost:8080/admin/consumers/$CID/scopes -H "$A" -H 'Content-Type: application/json' \
-d '{"tier":"firecrawl"}'
# 4. draw a rotated key (consumer token)
curl localhost:8080/key?feature=firecrawl_scrape -H "Authorization: Bearer <consumer-token>"Endpoints
| Method | Path | Auth | Purpose |
|---|---|---|---|
GET |
/health |
public | liveness |
GET |
/key?feature=X |
admin or consumer | draw a rotated key for a feature |
GET |
/key/{id}?feature=X |
admin or consumer | serve one named key for a feature, for a caller that must stay on the key an account was set up with; scope, rate limit and usage apply as on a draw; 404 unknown id, 403 out of scope or feature not on its tier, 429 exhausted |
POST |
/key/{id}/exhausted |
admin or consumer | report a key the provider refused; body {"until": RFC3339}, the key is skipped until then and serves again on its own |
POST |
/key/{id}/spend |
admin or consumer | report what a call cost; body {"amount": 0.0123, "unit": "usd", "request_id": "..."}, answers spent, remaining, resets_at; a repeated request_id is not counted twice, a unit other than the budget's is 409 |
GET POST PATCH |
/admin/tiers |
admin | list / create / update tier features |
DELETE |
/admin/tiers/{name} |
admin | remove a tier with its features and scopes; 409 while any key still uses it |
GET POST |
/admin/keys |
admin | list / add keys |
PATCH |
/admin/keys/{id} |
admin | set a key's budget, {"budget": {"amount": 200, "unit": "usd", "reset_day": 1}}, or clear it with {"budget": null} |
DELETE |
/admin/keys/{id} |
admin | remove a key (+ its secrets) |
GET POST |
/admin/consumers |
admin | list / create consumers |
DELETE |
/admin/consumers/{id} |
admin | remove a consumer (+ its scopes) |
POST |
/admin/consumers/{id}/scopes |
admin | grant a tier scope |
GET |
/admin/usage?limit=N |
admin | audit log |
GET |
/admin/health |
admin | pool size |
Environment variables
| Var | Default | Notes |
|---|---|---|
ADMIN_TOKEN |
(required) | superuser bearer token |
ENCRYPTION_KEY |
(empty = plaintext) | 32-byte hex; when set, new writes are encrypted |
DATABASE_URL |
(required) | postgres://user:pass@host:5432/db |
Everything else is a constant in internal/config: listen on :8080, 4 database connections (the serve path uses none), HTTP timeouts 30/30/120 s and a 30 s shutdown, JSON logs at info.
cmd/keypooler/main.go wires DB, sealer, key pool, HTTP server
internal/
api/ handlers, router, middleware, consumer auth
balance/ provider balance checks (Firecrawl, Tavily, Apify) and the 15-minute sync
config/ env config + validation
crypto/ AES-256-GCM + Sealer (opt-in, self-tagged)
db/ Postgres adapter (pgx pool), migrations
keypool/ round-robin pool, rate + usage budgets
writeback/ usage counts and audit events, flushed behind the serve
migrations/ 001_init.sql schema, 002_budgets.sql spend budgets
Runs on Railway as service keypooler in the platform project (Singapore), built from main, beside ai-gateway and the shared Postgres it uses (database keypooler, reached over the private network). Infrastructure is orkait/infra terraform/platform.
A schema change is a new migrations/NNN_name.sql; an applied version is never re-run.
Tests need a throwaway Postgres (the db tests drop their own tables):
docker compose up -d postgres
KEYPOOLER_TEST_DATABASE_URL=postgres://keypooler:keypooler@127.0.0.1:5432/keypooler go test -race ./...Go 1.25 (CGO-free, fully static binary) · Postgres via pgx (native protocol, cached prepared statements) · AES-256-GCM · zerolog · stdlib net/http · distroless runtime image.