English | 简体中文

Report suspected vulnerabilities through GitHub Security Advisories. Please include a minimal reproduction, the affected revision, and the expected impact. The maintainers aim to acknowledge reports within 72 hours.

OpenCollab-Eval can execute model-generated commands, attach to Docker, connect to remote workers, and run benchmark tests. Use disposable workers that contain no unrelated data. Keep credentials in files outside the repository and grant each worker access only to the current run.

Evaluation records can contain source patches, model transcripts, task identities, runtime paths, and provider metadata. Store them outside the source checkout and review them before publication.

The SWE Pro-Lite operations guide describes run-scoped credentials, workers, storage, and outputs. The evaluation integrity guide describes the trust boundary between benchmark data, Solver workspaces, candidate construction, official execution, and reports.

OpenCollab-Eval is pre-1.0. Security fixes are maintained on main.