Title: CapCode: Detecting cheating in coding agents with capped, randomized tests
Open Graph Title: CapCode: Detecting cheating in coding agents with capped, randomized tests
Description: CapCode constructs coding benchmarks whose best non-cheating score is deliberately below 1, turning above-cap performance into a statistical signal of test-gaming.
Open Graph Description: A benchmark-construction framework for coding tasks with a known performance ceiling. Scores above the cap are evidence of test-gaming.
Opengraph URL: https://ishida-lab.github.io/blog_capcode.html
Domain: ishida-lab.github.io
| og:image | https://ishida-lab.github.io/assets/og-image.png |
| og:image:width | 1200 |
| og:image:height | 630 |
| twitter:card | summary_large_image |
| og:type | article |
Links:
| ← Back to the blog | https://ishida-lab.github.io/index.html |
| Paper | https://arxiv.org/abs/2606.07379 |
| Code | https://github.com/ishida-lab/capcode |
| Datasets | https://huggingface.co/datasets/ishidalab/capcode |
| Stein et al. (2026) | https://arxiv.org/abs/2604.11806 |
| Anthropic (2026) | https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf |
| CapBencher | https://ishida-lab.github.io/blog_capbencher.html |
| Harbor | https://github.com/harbor-framework/harbor |
| code | https://github.com/ishida-lab/capcode |
| datasets | https://huggingface.co/datasets/ishidalab/capcode |
| CapReward | https://ishida-lab.github.io/blog_capreward.html |
| ← Back to the blog | https://ishida-lab.github.io/index.html |
Viewport: width=device-width,initial-scale=1