adversarial-evaluation

Persona agents with explicit memory, retrieval, adversarial smoke tests and a practical post-training roadmap.

ACL 2025: Search-based multilingual LLM evaluation that discovers English-correct, target-language-wrong failures; code and 6,713-pair dataset across 16 languages.

Agentic AI security bootcamp: labs for multi-agent observability, prompt injection, LPCI, red-teaming, adversarial evaluation — hands-on training.

Human-in-the-loop adversarial workflows for high-stakes research audit: from ChatGPT-Gemini duels to 4-model MAD. Copy-paste prompts, plus a browser page that sets up a run.

Adversarial security benchmark for agent authorization: does a compromised agent's policy-violating proposal become an unauthorized external effect? 61 trials, nine families, an independent oracle, per-mechanism ablation, confidence intervals. 0 unauthorized effects in 61 attack trials (95% CI [0.0%, 5.9%]). Reproduction is partial.

An adversarial AI expert workshop that stress-tests a research paper (rival-tradition referees argue; every comment quote-grounded and independently re-verified) and then rebuilds it: tracked-changes redline, clean version, your code re-run under a provenance wall, and a replication package. A Claude Code skill.

Three Claude Code skills for working with Codex CLI: codex-bridge (one-shot Codex calls), mad-build (Claude+Codex collaboration with cross-review), and mad-research (three-stream adversarial audit of papers, grants, reports with anonymized cross-critique and fresh-Codex synthesis).

RevealVLLMSafetyEval is a comprehensive pipeline for evaluating Vision-Language Models (VLMs) on their compliance with harm-related policies. It automates the creation of adversarial multi-turn datasets and the evaluation of model responses, supporting responsible AI development and red-teaming efforts.

Adversarial safety evaluation (AgentDojo-style): 25/30 held, 1 real gap found

Claude Code plugin implementing Anthropic's 3-agent harness (Planner, Generator, Evaluator) for long-running app development with pluggable rubrics and adversarial evaluation

Scientific QA robustness evaluation pipeline for evidence-missing RAG scenarios on PeerQA, with EM/F1 reliability analysis.

两个中文 skill（AI 痕迹审计 + 去 AI 味改写）和一个让它们互相对抗的实验台，重点测误杀率

Adversarial evaluation comparing black-box self-consistency checking vs. source-grounded NLI for hallucination detection in LLM summaries.

Architectural pattern for the multi-agent coordination layer above PDS and MCP. Adversarial verification, file-system state, negotiated contracts, Planner/Generator/Evaluator role decomposition. Companion spec to Progressive Discovery Spine.

A controlled adversarial web for measuring how browser agents fail

A multi-aspect LLM review harness for adversarial argument-error evaluation and purpose-annotated failure recycling.

CIDeR: a reproducible benchmark framework for causal exposure control in multi-agent LLM deliberation, comparing exposure-aware aggregation against voting, self-consistency, debate, causal-credit, social-choice, diversity, and adversarial baselines.

NI 43-101 adversarial resource extraction: DeepSeek extractor + GLM critic + deterministic final audit (field-level +-5% scoring, abstain-first)

Multi-agent deep research engine with SIA (Semantic Intelligence Architecture) — thermodynamic entropy control, adversarial critique, multi-reactor swarm orchestration

A public portfolio exploring adversarial AI evaluation, multimodal model behavior, and trust and safety systems.