Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -475,7 +475,9 @@ the others checks or relies on Reply's own state rather than repeating an effect

**Retries.** Reads are retried on errors. A write is retried only when it can't have reached Reply:
the connection was never made, or a `429` asked to wait 3 seconds or less. A longer `429` comes back
as `rate_limited` with `retry_after`.
as `rate_limited` with `retry_after`. A `429` that names a `reference` or says a daily or weekly limit
was reached is never resent, whatever it asks to wait: Reply may have recorded the attempt, and a repeat
would record another. Each comes back at once as the refusal it is.

## Raw API access

Expand Down Expand Up @@ -508,7 +510,8 @@ A write through `reply api` (POST, PUT, PATCH, DELETE) is never resent on a 5xx
connection that broke after sending. That last case exits 3: the write may have happened. `reply api`
goes by the method alone, so this includes a POST that only reads, such as the `…/filter` calls; run
it again yourself if it failed. A `429` is waited out only when `Retry-After` asks for 3 seconds or
less; a longer one is printed as the `429` it is.
less; a longer one is printed as the `429` it is, and so is one whose body names a `reference` or a
daily or weekly limit.

### Windows: Git Bash rewrites the path

Expand Down
11 changes: 8 additions & 3 deletions skills/reply-adapter/reply-api/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: >
Use before calling any non-trivial v3 endpoint, when choosing an endpoint for a task,
or when an API call fails and needs interpretation.
metadata:
version: 1.5.0
version: 1.5.1
pack: reply-adapter
category: execution
maturity: reviewed
Expand Down Expand Up @@ -73,7 +73,9 @@ per-page `.md` fetches; the spec is large.
has no feed yet answers a bare 404 — page through the inbox threads instead.
- **Rate limits:** 100 req/min and 3000 req/hour per user; reporting and sequence-stats endpoints
are stricter. On 429 the CLI already retried honoring `Retry-After` — if it still fails, wait
and reduce concurrency (sequential calls for bulk work).
and reduce concurrency (sequential calls for bulk work). The exception is a 429 that names a
`reference` or says a daily or weekly limit was reached: it is never resent and comes back at
once as the refusal it is.
- **Direct sends** (from reply-cli 0.17.0): `--send-after <moment>` takes a moment with its offset
and sends it as UTC, and Reply holds the send until then. `--idempotency-key <key>` goes as the
`Idempotency-Key` header: the same request repeated under it answers the send Reply took rather
Expand Down Expand Up @@ -192,7 +194,7 @@ Record endpoint paths used, entity IDs created/modified, background-job IDs and
| 403 `insufficient_scope` | key lacks the endpoint's scope | tell the user which scope is missing; don't retry |
| 403/401 `TEAM_REQUIRED`·`TEAM_NOT_ACCESSIBLE`·`USER_REQUIRED`·`USER_NOT_FOUND` | team/user resolution | see reply-auth skill |
| 404 | wrong/stale ID | re-list the entity, re-resolve the ID — don't invent IDs |
| 429 | rate limit | respect `Retry-After`; go sequential |
| 429 | rate limit | respect `Retry-After`; go sequential; one that names a `reference` or says a daily or weekly limit was reached is never resent and comes back at once as the refusal it is |
| 5xx | transient | already retried; wait, retry once, then report the API may be down |

## Safety
Expand All @@ -207,6 +209,9 @@ skill's confirmation rules — especially anything that sends messages or starts

## Changelog

- 1.5.1 (2026-11-05): *Conventions* and the error table say that a 429 naming a `reference` or
saying a daily or weekly limit was reached is never resent and comes back at once as the refusal
it is.
- 1.5.0 (2026-10-30): *Conventions* says how a direct send's text is filled in as a template:
on an email, a LinkedIn message and an InMail; a send that goes now refused by reply-cli 0.17.2's
three codes, with the names Reply lists; a held send filled in when it goes and failed then; the
Expand Down
8 changes: 6 additions & 2 deletions skills/reply-adapter/reply-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: >
Use when executing any Reply.io operation from a shell, setting up authentication,
or when another Reply skill needs commands executed.
metadata:
version: 1.3.0
version: 1.3.1
pack: reply-adapter
category: execution
maturity: reviewed
Expand Down Expand Up @@ -116,7 +116,9 @@ reply api /v3/whoami --verbose # full request/response trace on s
Rules: the path is used **exactly as written in the API docs** (starts with `/v3`; query string
goes in the path). `--method GET|POST|PUT|PATCH|DELETE` overrides the default. Output is always
`{"code": <http status>, "data": <body>}` on stdout — check `code` yourself; stdout stays clean
for piping. Transient failures (429/5xx) retry automatically honoring `Retry-After`.
for piping. Transient failures (429/5xx) retry automatically honoring `Retry-After`, except a 429
that names a `reference` or says a daily or weekly limit was reached, which is never resent and
comes back at once as the refusal it is.

### Exit codes & errors

Expand Down Expand Up @@ -163,6 +165,8 @@ explicit confirmation its Safety section requires.

## Changelog

- 1.3.1 (2026-11-05): `reply api` says that a 429 naming a `reference` or saying a daily or weekly
limit was reached is never resent and comes back at once as the refusal it is.
- 1.3.0 (2026-08-10): said plainly that reaching the whole API is not reaching the whole job — the
per-operation answer is in `reply-operations-mapping` — because "exposes the entire API v3" was
the sentence an agent was reading as "anything the user asks for is reachable". The read-back in
Expand Down
65 changes: 63 additions & 2 deletions src/__tests__/operations/direct-send.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import fs from 'fs';
import os from 'os';
import path from 'path';
import {describe, it, expect, beforeEach} from 'vitest';
import {mock_fetch, json, ctx, sent, answer, capture} from './helpers';
import {describe, it, expect, beforeEach, afterEach} from 'vitest';
import {mock_fetch, json, ctx, sent, answer, capture, instant_timers, real_timers} from './helpers';
import {handle_direct_send, type Direct_send_name, type Direct_send_opts} from '../../operations/direct-send';

const SUBJECT = 'Quick question, Marta';
Expand Down Expand Up @@ -230,6 +230,67 @@ describe('the LinkedIn sends', ()=>{
}
});

describe('a limit is one attempt at the send', ()=>{
beforeEach(()=>{ instant_timers(); });
afterEach(()=>{ real_timers(); });

// Every call gets the same refusal, so a retry shows up in the call count rather than in a request left unanswered.
const always = (body: Record<string, unknown>, status = 429, headers: Record<string, string> = {})=>{
mock_fetch.mockImplementation(async()=>json(body, status, headers));
};
const LIMITED = {title: 'Too Many Requests', status: 429};
const details_of = (e: unknown)=>(e as {details?: unknown}).details;

it('a provider limit that recorded a failed send is one call, and names that send', async()=>{
always({
...LIMITED, code: 'directLinkedIn.weeklyLimitReached', detail: 'The send failed.',
reference: 'linkedin-1042', state: 'failed', sendAfter: '2026-11-03T14:00:00Z',
});
const e = await failure('linkedin-connect', {note: NOTE}, LINKEDIN_OPTS);
expect(e).toMatchObject({code: 'linkedin.limit_reached', exit_code: 1, status: 429, reply_code: 'directLinkedIn.weeklyLimitReached'});
expect(details_of(e)).toStrictEqual({limit: 'weekly', reference: 'linkedin-1042', state: 'failed', send_after: '2026-11-03T14:00:00Z'});
expect(mock_fetch).toHaveBeenCalledTimes(1);
});

it('each allowance is one call, with no reference and no Retry-After to go by', async()=>{
for (const [reply_code, limit] of [['directLinkedIn.dailyLimitReached', 'daily'], ['directLinkedIn.weeklyLimitReached', 'weekly']])
{
mock_fetch.mockReset();
always({...LIMITED, code: reply_code});
const e = await failure('linkedin-message', {body: TEXT}, LINKEDIN_OPTS);
expect(e, reply_code).toMatchObject({code: 'linkedin.limit_reached', details: {limit}});
expect(mock_fetch, reply_code).toHaveBeenCalledTimes(1);
}
});

it('passes on a short Retry-After rather than waiting it out', async()=>{
always({...LIMITED, code: 'directLinkedIn.dailyLimitReached'}, 429, {'Retry-After': '2'});
expect(await failure('linkedin-message', {body: TEXT}, LINKEDIN_OPTS)).toMatchObject({code: 'linkedin.limit_reached', retry_after: 2});
expect(mock_fetch).toHaveBeenCalledTimes(1);
});

it('a 429 naming a send under a code that is no limit is one call too, and stays rate_limited', async()=>{
always({...LIMITED, code: 'directLinkedIn.somethingNew', reference: 'linkedin-1042', state: 'failed'});
const e = await failure('linkedin-message', {body: TEXT}, LINKEDIN_OPTS);
expect(e).toMatchObject({code: 'rate_limited', details: {reference: 'linkedin-1042', state: 'failed'}});
expect(mock_fetch).toHaveBeenCalledTimes(1);
});

it('a plain rate limit is still resent, and the send goes through', async()=>{
answer(json(LIMITED, 429), activity('LinkedInConnect'));
expect(await run('linkedin-connect', {note: NOTE}, LINKEDIN_OPTS)).toMatchObject({status: 'sent', activity_id: 1042});
expect(mock_fetch).toHaveBeenCalledTimes(2);
});

it('at any other status an allowance code stays reply.refused, and still names the send', async()=>{
always({title: 'Bad Request', status: 400, code: 'directLinkedIn.weeklyLimitReached', reference: 'linkedin-1042', state: 'failed'}, 400);
const e = await failure('linkedin-connect', {note: NOTE}, LINKEDIN_OPTS);
expect(e).toMatchObject({code: 'reply.refused', reply_code: 'directLinkedIn.weeklyLimitReached'});
expect(details_of(e)).toStrictEqual({reference: 'linkedin-1042', state: 'failed'});
expect(mock_fetch).toHaveBeenCalledTimes(1);
});
});

it('holds a text to the length Reply publishes, counting characters as its document does, before any call', async()=>{
const cases: [Direct_send_name, Record<string, string>, string, number][] = [
['linkedin-message', {}, 'body', 8000],
Expand Down
84 changes: 84 additions & 0 deletions src/__tests__/utils/client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,90 @@ describe('utils/client', ()=>{
expect(mock_fetch).toHaveBeenCalledTimes(2);
});

describe('a 429 that settles the request is one attempt', ()=>{
const LIMIT_WITH_RECORD = {
title: 'Too Many Requests', status: 429, code: 'directLinkedIn.weeklyLimitReached', detail: 'The send failed.',
reference: 'linkedin-1042', state: 'failed', sendAfter: '2026-11-03T14:00:00Z',
};
const once_only = async(body: unknown, headers: Record<string, string> = {})=>{
instant_timers();
mock_fetch.mockImplementation(async()=>err_res(429, body, headers));
const {request_raw} = await import('../../utils/client');
const r = await request_raw(BASE, 'tok', 'POST', '/x', {a: 1});
expect(r.status).toBe(429);
expect(mock_fetch).toHaveBeenCalledTimes(1);
return r;
};

it.each([
['a reference and a state', {reference: 'linkedin-1042', state: 'failed'}],
['a reference under a code that names no allowance', {code: 'directLinkedIn.unknown', reference: 'email-7'}],
['the weekly allowance', {code: 'directLinkedIn.weeklyLimitReached'}],
['the daily allowance', {code: 'directLinkedIn.dailyLimitReached'}],
['the weekly allowance in another family', {code: 'directEmail.weeklyLimitReached'}],
['the daily allowance in another family', {code: 'supportRequest.dailyLimitReached'}],
['the weekly allowance with no family', {code: 'weeklyLimitReached'}],
['the daily allowance with no family', {code: 'dailyLimitReached'}],
])('%s is not resent', async(_name, body)=>{
await once_only({title: 'Too Many Requests', ...body});
});

it('hands the caller the whole answer of that one attempt, with a short Retry-After not waited out', async()=>{
const r = await once_only(LIMIT_WITH_RECORD, {'Retry-After': '2'});
expect(r.data).toEqual(LIMIT_WITH_RECORD);
expect(r.response_headers['retry-after']).toBe('2');
});

it('holds for a read too: it goes by the answer, not by the call', async()=>{
instant_timers();
mock_fetch.mockImplementation(async()=>err_res(429, {code: 'directLinkedIn.dailyLimitReached'}));
const {request_raw} = await import('../../utils/client');
expect((await request_raw(BASE, 'tok', 'GET', '/x')).status).toBe(429);
expect(mock_fetch).toHaveBeenCalledTimes(1);
});

it('throws from request as any other refusal, after that one attempt', async()=>{
instant_timers();
mock_fetch.mockImplementation(async()=>err_res(429, LIMIT_WITH_RECORD));
const err = await get(BASE, 'tok', '/x').catch(e=>e);
expect(err).toBeInstanceOf(Api_error);
expect(err.status).toBe(429);
expect(err.code).toBe('directLinkedIn.weeklyLimitReached');
expect(mock_fetch).toHaveBeenCalledTimes(1);
});
});

describe('an ordinary 429 keeps its retry', ()=>{
it.each([
['no body', ''],
['a title alone', {title: 'Too Many Requests', status: 429}],
['a code that names no allowance', {code: 'rateLimit.exceeded'}],
['a code that only resembles an allowance', {code: 'linkedInCommentPreview.dailySampleLimitReached'}],
['a null reference', {code: 'rateLimit.exceeded', reference: null, state: null}],
['an empty reference', {reference: ''}],
['a reference that is not text', {reference: 1042}],
['a body of JSON null', 'null'],
['a body of a JSON list', '[]'],
['a body that is not JSON', 'API calls quota exceeded!'],
])('%s is resent up to the limit', async(_name, body)=>{
instant_timers();
mock_fetch.mockImplementation(async()=>err_res(429, body));
const {request_raw} = await import('../../utils/client');
expect((await request_raw(BASE, 'tok', 'POST', '/x', {a: 1})).status).toBe(429);
expect(mock_fetch).toHaveBeenCalledTimes(4); // initial + 3 retries
});

it('waits out a short Retry-After as before, and takes the answer after it', async()=>{
instant_timers();
mock_fetch.mockReset();
mock_fetch.mockResolvedValueOnce(err_res(429, {title: 'Too Many Requests'}, {'Retry-After': '2'}))
.mockResolvedValueOnce(json_res({ok: true}));
const {request_raw} = await import('../../utils/client');
expect((await request_raw(BASE, 'tok', 'POST', '/x', {a: 1})).status).toBe(200);
expect(mock_fetch).toHaveBeenCalledTimes(2);
});
});

const broken_body = ()=>new Response(new ReadableStream({
start(c){ c.error(new TypeError('terminated')); },
}), {status: 200});
Expand Down
20 changes: 19 additions & 1 deletion src/utils/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ const RETRY_BASE_MS = 500;
// The longest Retry-After waited out here. A longer one goes back to the caller as the 429 it
// is: a wait inside one call spends a time budget the caller may not have.
const MAX_WAIT_SECONDS = 3;
// The allowances a 429 can say have run out, as opposed to a pace that was too fast.
const ALLOWANCE_CODES = ['weeklyLimitReached', 'dailyLimitReached'];
// Node's fetch reports every transport failure as TypeError('fetch failed') with the system
// error on `.cause`. These are failures to connect at all, so the request never left: no route,
// no answer to the connect, a URL that can't be parsed, or a TLS handshake that failed.
Expand Down Expand Up @@ -127,6 +129,22 @@ const parse_retry_after = (raw: string | null | undefined): number | null=>{

const backoff_ms = (attempt: number): number=>RETRY_BASE_MS * 2 ** attempt;

// Whether a 429 answers the request instead of asking for it later, read from the body alone so
// a route that answers in the same shape is covered too. A reference means Reply recorded this
// attempt, so sending it again records another. An allowance that has run out won't come back
// within any wait made here. A code is matched on its last part: the part before the dot is only
// the family of the route that answered.
const settles_request = (text: string): boolean=>{
const body = parse_body(text) as {reference?: unknown; code?: unknown} | string | null;
if (typeof body !== 'object' || body === null)
{
return false;
}
const {reference, code} = body;
return (typeof reference === 'string' && reference !== '')
|| (typeof code === 'string' && ALLOWANCE_CODES.includes(code.slice(code.lastIndexOf('.') + 1)));
};

// One request under the retry policy, body included: an answer that breaks off mid-body is a
// failure after sending like any other. It returns the final Response and its text for any HTTP
// status. It throws RuntimeError('network') when Reply could not be reached and nothing was sent,
Expand Down Expand Up @@ -160,7 +178,7 @@ const send = async(url: string, init: RequestInit, kind: Call_kind): Promise<{re
await sleep(backoff_ms(attempt));
continue;
}
if (res.status === 429 && can_retry)
if (res.status === 429 && can_retry && !settles_request(text))
{
const wait = parse_retry_after(res.headers.get('Retry-After'));
if (wait !== null && wait > MAX_WAIT_SECONDS)
Expand Down
Loading