StealthBench live results: space-bunny-free
Anonymous OpenCode Zen alias · keyless $0 free tier · measured 2026-10-04/05 · repo
Scoreboard
| Benchmark | Score | n | 95% CI | Cost |
|---|
| IFEval strict / loose (official scorer) | 0.573 / 0.584 | 541 (full set) | ±0.04 | $0 |
| MATH-500 (exact→numeric protocol) | 0.610 | 500 (full split) | ±0.04 | $0 |
| MMLU-Pro (letter-only extraction) | 0.712 | 500 stratified | ±0.04 | $0 |
| BFCL prompted slice | 30/50 eligible, overall null* | 50 | — | $0 |
*Partial 6/16-category slice: reported per-category, never zero-filled. Transport 100% on every run.
Headline scores with 95% CI
vs frontier and budget references (published full-set numbers)
MMLU-Pro by category (n=32–36 each)
BFCL prompted slice by category (n=8–9 each)
How it was measured
- Direct calls to the Zen chat API, temperature 0, bare user prompts, one sample per task, retries on 429/5xx only.
- IFEval graded by the pinned official scorer; MMLU-Pro/MATH-500 by frozen repo adapters (letter extraction, exact→numeric equivalence, no sympy).
- Pinned data revisions; seeded selections; transport failures kept missing, never zero.
Caveats
- MMLU-Pro/MATH-500 are subsamples with zero-shot prompts; published numbers use full sets and usually few-shot CoT. Directional only.
- MMLU-Pro needs a 1024-token budget on this alias (reasoning lives in traces); at 64 tokens it scored 0.36.
- Alias identity unresolved: similarity evidence only, never a probability.
- Code-execution benchmarks skipped: no container runtime on the host.