pkg-assistant

A deterministic-first, LLM-assisted assistant for configuring and executing scientific Python packages on the available hardware. The first supported package is TRED (3D LArTPC detector response simulator); the architecture is package-agnostic and new packages are added as Skills — schema-validated directories of YAML + Jinja2 + small helper scripts.

You can drive the assistant in two ways:

• Deterministic CLI — pkg-assistant plan/test/run/check, no LLM in the loop.
• Conversational agent — pkg-assistant chat / pkg-assistant ask "...", which uses an OpenAI model (default gpt-5.4-nano) to translate natural language into the same deterministic tools.

---

Design principles

• Python is the source of truth. Hardware probing, resource estimation, config generation, log parsing, retry decisions, and completion checking are deterministic Python. Adding the LLM does not move authority into the model — it adds a UX layer on top.
• Skills are the unit of extension. A package = a directory under skills/<name>/ with a manifest validated against agent_interface/skill_schema.json. No per-package Python class lives in the core engine. The same skill is portable to Codex, Claude Code, or any schema-aware agent.
• Three layers of validation. (1) Tool-call enums on every skill parameter; (2) pydantic + JSON-schema validation when a skill loads; (3) a per-skill allowed_fields validator that rejects forbidden config edits before they reach disk.
• LLM is advisory only. It never executes shell commands, never writes files directly, never bypasses validation. All edits flow through apply_suggestion which requires explicit user approval.

---

Install

Requires Python 3.12+ and uv.

uv venv --python 3.12
source .venv/bin/activate
uv pip install -e ".[dev]"        # add ",gpu" if pynvml/CUDA is available

For the agent commands, set your API key:

export OPENAI_API_KEY=sk-...
# optional: pin a different model or proxy
export OPENAI_MODEL=gpt-5.4-nano
export OPENAI_BASE_URL=https://...

---

Usage

1. Deterministic CLI (no LLM)

pkg-assistant skills                                      # list discoverable skills
pkg-assistant hardware                                    # probe CPU/RAM/GPU

pkg-assistant plan   --skill tred --input workspaces/data/tred/sample_input.hdf5
pkg-assistant test   --skill tred --input workspaces/data/tred/sample_input.hdf5 \
                                  --config workspaces/runs/tred/configs/generated/config.yaml
pkg-assistant run    --skill tred --input workspaces/data/tred/sample_input.hdf5 \
                                  --config workspaces/runs/tred/configs/generated/config.yaml
pkg-assistant check  --skill tred --log workspaces/runs/tred/logs/run.log

Output paths default to per-skill subdirs under --workspace (default workspaces/); pass --output, --log, --config-out, etc. to override. Input paths are user-supplied — workspaces/data/<skill>/ is the suggested location, never required.

plan writes:

File	Purpose
workspaces/runs/<skill>/configs/generated/config.yaml	rendered package config
workspaces/runs/<skill>/reports/resource_plan.json	hardware + plan + expected events
workspaces/runs/<skill>/reports/expected_events.json	event IDs the run should produce

2. Agent — interactive REPL

pkg-assistant chat

Then talk to it:

Plan a TRED run on workspaces/data/tred/sample_input.hdf5, then test the largest event.

The agent will call get_hardware → plan → test, report what it did, and quote concrete numbers. Every tool call is JSON-validated; the agent cannot pick a non-existent skill, edit a forbidden field, or apply a config change without your explicit "yes/apply/approve."

Each session writes a JSONL transcript to workspaces/logs/agent/session-<timestamp>.jsonl so every tool call and result is auditable.

3. Agent — one-shot

pkg-assistant ask "What hardware do we have? Plan a TRED run on workspaces/data/tred/sample_input.hdf5."

Agent tool surface (9 tools)

Tool	Purpose
list_skills	Catalog of all discoverable skills
describe_skill(skill)	Full SKILL.md + allowed fields + command template + estimation params
get_hardware	Probe CPU / RAM / GPU
plan(skill, input_path, base_config_path?)	Estimate + render config + write expected_events.json
test(skill, config, input, event_id?)	Run a single event; auto-retries once if the failure matches a known retry rule
run(skill, config, input)	Run the full job; same auto-retry
check(skill, log, expected?)	Compare expected vs completed event IDs
diagnose(skill, log)	Parse a log → error class, retry rules, allowed fields. Informs a fix proposal but does not apply one.
apply_suggestion(skill, config, edits, confirmed)	Apply field edits; confirmed=False returns a preview; confirmed=True validates against allowed_fields and writes the new config

---

Adding a new package

A new package = a new skill directory. No engine code changes.

skills/<name>/
├── SKILL.md                     # frontmatter `name`/`description` drives agent activation
├── skill.yaml                   # manifest, validated against agent_interface/skill_schema.json
├── templates/config.yaml.j2     # Jinja2 template; receives `plan` and `base`
├── log_patterns.yaml            # list of {name, pattern, error_class, severity, captures}
├── retry_rules.yaml             # error_class -> [{op: scale|set, field, factor|value, min?, max?}]
└── scripts/<helper>.py          # only when manifest declares it (e.g. input.helper)

Helper script contract

Helpers run as subprocesses with the engine's Python. They accept --in <path> and --out <path> and emit JSON conforming to a documented shape:

Helper kind	Output JSON shape
Input probe	{"input_path": ..., "events": [{"event_id", "size_metric", "extra"}], "extra": {...}}
Completion extractor	{"completed_event_ids": [...]}

For most packages, completion can be expressed declaratively as a log pattern with an event_id capture, so only the input probe needs a helper.

Validating your manifest

python -m pkg_assistant.skills._schema_export   # regenerates agent_interface/skill_schema.json
pytest tests/skills/                            # validates the schema artifact and your skill loads cleanly

See skills/tred/ for a worked example and AGENTS.md for the full contract.

---

Repository layout

src/pkg_assistant/
├── cli.py, main.py                # entry points (typer)
├── core/                          # package-agnostic engine
│   ├── hardware/    probe + schemas
│   ├── planning/    estimator, retry_policy
│   ├── runtime/     subprocess runner
│   ├── logs/        regex parser
│   ├── completion/  expected vs completed event diffing
│   ├── safety/      action validator
│   └── utils/       json/yaml IO
├── skills/                        # the engine's skill machinery (NOT skills themselves)
│   ├── manifest.py                # SkillManifest pydantic model + JSON-schema source of truth
│   ├── schemas.py                 # InputManifest, ResourcePlan, LogPattern, ...
│   ├── loader.py                  # discover skills/<name>/
│   ├── runtime.py                 # Skill class — declarative dispatch
│   └── _schema_export.py          # regenerate agent_interface/skill_schema.json
└── agent/                         # LLM-driven layer
    ├── catalog.py                 # parse SKILL.md frontmatter
    ├── prompt.py                  # build system prompt (cache-friendly)
    ├── tools.py                   # 9 tool specs + dispatch
    ├── client.py                  # OpenAI SDK wrapper (injectable)
    ├── loop.py                    # AgentSession with tool-call loop + JSONL transcript
    └── chat_cli.py                # `pkg-assistant chat` / `ask`

skills/tred/                       # the actual TRED skill (manifest + helpers)
agent_interface/skill_schema.json  # canonical JSON Schema for skill.yaml
prompts/                           # (Phase 3+) prompt templates
workspaces/                        # gitignored, per-skill runtime tree
├── packages/<skill>/              #   upstream source (e.g. TRED git checkout)
├── data/<skill>/                  #   input data files (recommended location)
├── runs/<skill>/                  #   logs, reports, tmp, configs/generated
└── logs/agent/                    #   global chat-session transcripts
tests/{core,skills,agent}/         # 60 tests, all offline (fake LLM client for agent)

---

Phase status

Phase	Scope	Status
1	Deterministic core + Skill format + TRED skill	✅
2	Retry policy executor (core/planning/retry_policy.py)	✅
3	LLM agent (catalog → tools → loop), OpenAI gpt-5.4-nano	✅
4	Action validator (allowed_fields enforcement)	✅ minimum slice
5	Skill packaging polish (SKILL.md activation hints, tool_manifest.json)	deferred
6	Optional FastAPI layer + dashboard	deferred

Phases 5 and 6 are intentionally deferred until a second package validates the design.

---

Development

pytest                                              # run all 60 tests
python -m pkg_assistant.skills._schema_export       # regen agent_interface/skill_schema.json

The agent test suite uses a fake LLM client; tests run fully offline and require no API key.

For the full agent contract, safety rules, and adapter-contract details, see AGENTS.md.