[russellwheatley]

Replaces the FirebaseUI Web getting-started agent skill with the platform-agnostic v5 version. In my evals it cuts tool calls by about 75% and removes node_modules lookups entirely on React, Angular and shadcn/ui, with no drop in pass rate.

Summary

• SKILL.md is one platform-agnostic router (5.5 KB). It detects React, shadcn/ui or Angular, points the agent at that stack's two reference files plus core.md, and carries the search rules: glob with targetDirectory set to src, never **/* or src/**/* from the project root, never look in node_modules.
• References are split per platform and every file is under 10 KB: core.md, react.md, react-recipes.md, angular.md, angular-recipes.md, shadcn.md, shadcn-recipes.md. They hold every prop, selector and signature the agent needs, so it has no reason to read the installed packages.
• Fixes found while testing:
  • core.md: signatures corrected against @firebase-oss/ui-core 7.1.0.
  • shadcn: removed the ui-styles step and fixed the registry type issue.
  • Angular: provideAuth wiring, the styles path, and the .fui-form__action reset.
  • React: auth-state hook and RequireAuth recipes.
• Why: with the skill on main, an agent still spends most of its tool calls globbing the whole project and grepping node_modules to work out prop names. Putting the signatures in the skill and telling it where not to look removes that work.

Only .agents/skills/firebaseui-web-getting-started/ changes. Nothing else in the repo references the skill's files.

How it was tested

Evals live in invertase/firebase-ui-evals (branch russell/skill-v5-all-platforms, recorded runs under evals/.history/). They run on reeval with Cursor composer-2.5 only, which is deliberately the weaker model. Each attempt gets a fresh Docker sandbox, builds a full auth app from the skill, and is scored by:

• check scripts (typecheck, build, route and screenshot capture),
• flow judges (typesafe-ai/jev-1.13.0, score threshold 0.8),
• a visual judge and a pairwise compare judge (google/gemini-3.8-flash).

A node_modules call is a tool call whose input mentions node_modules, counted from the attempt transcripts. All numbers below are per-attempt means with n=4 attempts per arm. Rounds are groups of consecutive tool calls with no text between them.

Study 1 (Oct 22, reeval 0.12.0, React only, 12 attempts, 13.9 min wall)

Arm	Tool calls	node_modules calls	Input tokens	Agent time
no-skill	61	41	1.20M	160 s
shipped skill (main at 55500ba)	53	36	0.99M	141 s
colleague's skill (v2, previously new-skill)	34	14	0.42M	107 s

Input tokens in study 1 follow the 0.12.0 counting, which included cache reads and differs from the Cursor dashboard. reeval fixed this in 0.13.0 (invertase/reeval#487), so tokens are only comparable within a study.

No-skill was the slowest and used the most tokens and tool calls, so I dropped it from every later run. The data shows the skill matters.

Study 2 (Oct 23, reeval 0.13.0, React + Angular + shadcn, 28 attempts, 22m20s wall at concurrency 8)

Total tokens = uncached input + cache read + output.

Eval	Arm	Tool calls	node_modules calls	Total tokens	Agent time
React	v4	15.0	0	157k	63 s
React	v5	15.5	0	184k	67 s
React	shipped	56.5	39.5	1,033k	134 s
Angular	v5	15.0	0	126k	64 s
Angular	shipped	56.8	31.2	1,105k	145 s
shadcn	v5	16.8	0	168k	62 s
shadcn	shipped	73.0	25.5	1,154k	223 s

v5 vs shipped, change in per-attempt mean:

Scope	Tool calls	node_modules calls	Agent time	Rounds	Uncached input	Cache read
React	-73%	-100%	-50%	-80%	-50%	-84%
Angular	-74%	-100%	-56%	-83%	-63%	-91%
shadcn	-77%	-100%	-72%	-78%	-47%	-87%
Pooled (n=12 vs 12)	-75%	-100%	-62%	-80%	-55%	-87%

Pooled absolute values, shipped to v5: tool calls 62.1 to 15.8, node_modules calls 32.1 to 0, agent time 167 s to 64 s, rounds 27.5 to 5.5.

Pass counts: 26 of 28 attempts passed. v5 passed 12 of 12, shipped 11 of 12 (one Angular email-link flow score of 0.72 against the 0.8 threshold), and React v4 3 of 4 (an /anonymous screenshot timeout caused by the agent's page signing in on mount). Moving from v4 to v5 did not regress React: tool calls +3%, agent time +7%, rounds unchanged. v5 won the v4 vs v5 compare judge 6-0 on fit and 5-1 on adjustability.

Caveats

• composer-2.5 only, and n=4 per arm. Small differences (for example React v4 vs v5) are within the noise.
• v5 loses the compare judge on "fit" to the shipped skill for Angular (6-2) and narrowly for React (5-3 in the direct pair). The recipes show bare pages, so agents don't wrap them in the app's existing page container or add a Sign in link on Home. On shadcn v5 won fit 7-1, but one loss came from the shipped output offering "add email" to anonymous users. v5 won "adjust" on every platform. Follow-up: page container reuse in the recipes, a Sign in link on Home, and the shadcn add-email link.
• Every arm, including v5, makes one broad root glob in round 1 (about 119 KB of output). No skill can prevent that. A workspace rule, .cursorignore or a Cursor-side fix would.
• Angular SMS MFA enrolment is not confirmed in the emulator, and the phone input defaults to +93 in ui-angular 7.1.0.
• The shadcn registry copies have 3 TypeScript errors (UserCredential | void) against the 7.1.0 packages. Worth an upstream fix.
• Earlier smoke runs are left out on purpose.

Reviewer checklist

• SKILL.md and every file under references/ is under 10,000 bytes (wc -c).
• The router in SKILL.md names exactly the reference files that exist, and the old files that no longer exist are gone.
• Signatures in core.md match @firebase-oss/ui-core 7.1.0.
• Search rules read correctly: targetDirectory src, no root **/*, no node_modules.
• Nothing else in the repo links to the removed skill files (I found no references).
• Results can be reproduced from the evals branch: npm run eval:study in invertase/firebase-ui-evals.

[gemini-code-assist]

Code Review

This pull request restructures the FirebaseUI Web getting started skill documentation, introducing comprehensive reference and recipe guides for Angular, React, shadcn/ui, and the core library. The feedback highlights two issues in the Angular reference file: an incorrect reference to the unreleased Angular 20 and @angular/fire@20 instead of version 19, and an invalid template binding syntax [variant]="'primary' | 'secondary'" which evaluates as a bitwise OR operation at runtime and fails strict template type checking.

[gemini-code-assist]

The reference mentions @angular/fire@20 and Angular 20, but Angular 19 is the current major version (Angular 20 is not yet released). Please update these references to @angular/fire@19 and Angular 19 (also on line 17) to match the target environment.

[gemini-code-assist]

In Angular templates, writing [variant]="'primary' | 'secondary'" will be evaluated as a bitwise OR operation at runtime (resulting in 0), which will fail strict template type checking. It should be documented as variant="primary" or variant="secondary" (or explained that it accepts 'primary' | 'secondary').