| Category | Cases | Last run |
|---|---|---|
| startupSmall commands where interpreter startup dominates runtime. | 4 | 0.017 msbash median: 5.408 ms |
| arraysIndexed array reads, writes, expansion, and iteration. | 6 | 0.019 msbash median: 6.865 ms |
| stringsString expansion, pattern handling, and text manipulation. | 8 | 0.02 msbash median: 6.807 ms |
| pipesPipeline construction, streaming, and command chaining. | 6 | 0.021 msbash median: 7.346 ms |
| controlConditionals, loops, case statements, and branching scripts. | 9 | 0.026 msbash median: 8.239 ms |
| subshellCommand substitution and nested shell execution paths. | 6 | 0.028 msbash median: 7.704 ms |
| ioFile reads, writes, redirects, and filesystem-facing commands. | 6 | 0.036 msbash median: 9.014 ms |
| arithmeticInteger math, substitutions, and expression-heavy shell snippets. | 6 | 0.049 msbash median: 8.676 ms |
Benches
Latest benchmark snapshot
Static aggregate generated from repository result artifacts. Use the linked files for raw measurements and full eval traces.
| Category | Passed | Pass rate |
|---|---|---|
| system_info | 1/2 | 50%tasks passed |
| file_operations | 3/4 | 66.7%tasks passed |
| scripting | 5/7 | 68.6%tasks passed |
| json_processing | 8/8 | 100%tasks passed |
| data_transformation | 6/6 | 100%tasks passed |
| complex_tasks | 6/6 | 100%tasks passed |
| text_processing | 6/6 | 100%tasks passed |
| pipelines | 5/5 | 100%tasks passed |
| Run | Score | Tools |
|---|---|---|
| gpt-5.3-codex2026-05-26 | 93% | 86.8% |
| gpt-5.52026-05-26 | 92.7% | 91.5% |
| claude-opus-4-72026-05-26 | 97.8% | 90.3% |
| claude-sonnet-4-62026-05-26 | 94% | 91% |
| claude-haiku-4-5-202510012026-05-26 | 98.4% | 92.3% |
| claude-sonnet-4-62026-02-28 | 92.5% | 85.1% |
Indexes