Open Source Verification Pipeline

AgentRunDB Lab

Where every verified run is executed, captured and published.

CORE PRINCIPLE

"Website content is not written. It is run."

does-it-run on GitHub Methodology & Standards Reproduce Locally

The 5-Step Verification Pipeline

How every compatibility verdict is generated from raw execution facts in isolated environments.

GitHub Actions Trigger

Scheduled runs (every Saturday 03:17 UTC) or manual dispatch execute the matrix of agents and target models.

Isolated Environment

Spawns a clean temporary directory with a test fixture repository containing a failing unit test.

Headless Agent Execution

Invokes the agent CLI (e.g. claude -p) with provider endpoint environment injected and a 10-minute timeout.

Fact & Verdict Verification

Runs node --test (exit 0 = passed) and verifies file edits via SHA-256 diffs to establish capability facts.

Evidence Ingestion & Publication

Archives report.json, log.txt, and fixture diffs, then posts to /api/runs/import for automated site updates.

Recent Pipeline Runs

Live runs emitted from automated verifiers and ingested into AgentRunDB.

View all runs →

Date	Compatibility	Status	Duration	Tokens	Cost	Run
2026-08-19	OpenClaw × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	OpenClaw × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	OpenClaw × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	OpenCode × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	OpenCode × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	OpenCode × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	Claude Code × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	Claude Code × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	Claude Code × DeepSeek V4 Pro	Passed	0ms	—	—	details
2026-08-19	Claude Code × MiniMax-M3	Passed	15.6s	110,936	$0.17	details

Live Ecosystem Data

Dynamic metrics backed by continuous automated verification.

Verified Runs

Compatibility Records

Agents

Skills & MCP Servers

Reproduce Locally

Run the exact same verification pipeline on your local machine in under 2 minutes.

bash — does-it-run

# 1. Clone does-it-run verifier
git clone https://github.com/agentrundb/does-it-run
cd does-it-run

# 2. Run verification against target model
DEEPSEEK_API_KEY=sk-... npm run verify -- --agent claude-code --model deepseek

Standard Contract: All verifier outputs emit standard JSON matching the /api/runs/import contract. POST /api/runs/import

Contribute to the Lab

AgentRunDB and does-it-run are 100% open source. Help expand test coverage across the ecosystem.

Add an Agent Adapter

Create agents/<slug>.js implementing checkInstalled, buildEnv, and invoke. See the agents/ directory.

Browse agent adapters →

Add a Test Case

Define prompt, fixture, and expected outcomes in cases/*.json, mapped 1:1 to site test_cases.

Browse test cases →

Report an Issue / Suggest Targets

Found a broken integration, model API drift, or want to suggest a new agent harness? Open an issue.

Open GitHub issue →