← ChoreCode leaderboard · 20260706T163525Z
qwen3.6-27b
qwen3.6:27b-mlx
* latency reflects local hardware and load
Stats
release-0.1.1| Model | qwen3.6:27b-mlx |
| Accepted | 33/69 (48%) |
| Hard fail rate | 1% |
| $ / accepted chore | $0.0125 |
| $ / attempt | $0.0060 |
| Tokens / accepted chore | 5646 |
| Tokens in / out (total) | 16.2k / 170.1k |
| Mean latency | 157.5 s * |
Cost is recorded input/output tokens multiplied by current OpenRouter pricing for the same model or weights.
Results by task
17 tasks| Task | Accepted | Hard fails | Failure reasons seen |
|---|---|---|---|
| bare-except-chore | 5/5 | 0 | — |
| basic-function-off-by-one | 1/5 | 0 | patch_apply_failed |
| config-noop-chore | 1/5 | 0 | patch_apply_failed |
| debug-cruft-chore | 5/5 | 0 | — |
| dependency-conflict-chore | 1/3 | 0 | patch_apply_failed |
| dependency-existing-retry-chore | 0/3 | 0 | patch_apply_failed |
| dependency-stdlib-query-chore | 0/3 | 0 | patch_apply_failed |
| docstring-chore | 2/5 | 0 | patch_apply_failed |
| error-message-chore | 4/5 | 0 | patch_apply_failed |
| generated-cli-help-chore | 3/3 | 0 | — |
| generated-client-field-chore | 1/3 | 0 | patch_apply_failed |
| nextflow-orchestration-chore | 0/3 | 1 | forbidden_pattern, hidden_tests_failed, smoke_failed, patch_apply_failed |
| project-manager-chore | 0/5 | 0 | patch_apply_failed |
| pyproject-edit-chore | 0/3 | 0 | patch_apply_failed |
| tiny-python-type-chore | 3/5 | 0 | patch_apply_failed |
| type-annotation-chore | 5/5 | 0 | — |
| version-hallucination-chore | 2/3 | 0 | patch_apply_failed |
Attempts
expand a row for the patch and logsbare-except-chore · rep 0 · accepted · 207→1859 tok · $0.0045
no patch produced
bare-except-chore · rep 0 · accepted · 207→1865 tok · $0.0045
no patch produced
bare-except-chore · rep 1 · accepted · 207→2107 tok · $0.0051
no patch produced
bare-except-chore · rep 1 · accepted · 207→2774 tok · $0.0067
no patch produced
bare-except-chore · rep 2 · accepted · 207→2738 tok · $0.0066
no patch produced
basic-function-off-by-one · rep 0 · patch_apply_failed · 192→2953 tok · $0.0071
no patch produced
basic-function-off-by-one · rep 0 · patch_apply_failed · 192→1138 tok · $0.0028
no patch produced
basic-function-off-by-one · rep 1 · patch_apply_failed · 192→1615 tok · $0.0039
no patch produced
basic-function-off-by-one · rep 1 · accepted · 192→1674 tok · $0.0041
no patch produced
basic-function-off-by-one · rep 2 · patch_apply_failed · 192→1857 tok · $0.0045
no patch produced
config-noop-chore · rep 0 · patch_apply_failed · 198→3513 tok · $0.0085
no patch produced
config-noop-chore · rep 0 · accepted · 198→3574 tok · $0.0086
no patch produced
config-noop-chore · rep 1 · patch_apply_failed · 198→2383 tok · $0.0058
no patch produced
config-noop-chore · rep 1 · patch_apply_failed · 198→2955 tok · $0.0071
no patch produced
config-noop-chore · rep 2 · patch_apply_failed · 198→3928 tok · $0.0095
no patch produced
debug-cruft-chore · rep 0 · accepted · 198→1462 tok · $0.0036
no patch produced
debug-cruft-chore · rep 0 · accepted · 198→1922 tok · $0.0047
no patch produced
debug-cruft-chore · rep 1 · accepted · 198→721 tok · $0.0018
no patch produced
debug-cruft-chore · rep 1 · accepted · 198→719 tok · $0.0018
no patch produced
debug-cruft-chore · rep 2 · accepted · 198→1388 tok · $0.0034
no patch produced
dependency-conflict-chore · rep 0 · patch_apply_failed · 287→2353 tok · $0.0057
no patch produced
dependency-conflict-chore · rep 1 · accepted · 287→2186 tok · $0.0053
no patch produced
dependency-conflict-chore · rep 2 · patch_apply_failed · 287→2176 tok · $0.0053
no patch produced
dependency-existing-retry-chore · rep 0 · patch_apply_failed · 301→4977 tok · $0.0120
no patch produced
dependency-existing-retry-chore · rep 1 · patch_apply_failed · 301→4557 tok · $0.0110
no patch produced
dependency-existing-retry-chore · rep 2 · patch_apply_failed · 301→4910 tok · $0.0119
no patch produced
dependency-stdlib-query-chore · rep 0 · patch_apply_failed · 296→1987 tok · $0.0049
no patch produced
dependency-stdlib-query-chore · rep 1 · patch_apply_failed · 296→2318 tok · $0.0056
no patch produced
dependency-stdlib-query-chore · rep 2 · patch_apply_failed · 296→2973 tok · $0.0072
no patch produced
docstring-chore · rep 0 · patch_apply_failed · 157→2721 tok · $0.0066
no patch produced
docstring-chore · rep 0 · accepted · 157→1791 tok · $0.0043
no patch produced
docstring-chore · rep 1 · accepted · 157→1746 tok · $0.0042
no patch produced
docstring-chore · rep 1 · patch_apply_failed · 157→2613 tok · $0.0063
no patch produced
docstring-chore · rep 2 · patch_apply_failed · 157→1012 tok · $0.0025
no patch produced
error-message-chore · rep 0 · accepted · 243→2742 tok · $0.0067
no patch produced
error-message-chore · rep 0 · accepted · 243→2220 tok · $0.0054
no patch produced
error-message-chore · rep 1 · accepted · 243→2818 tok · $0.0068
no patch produced
error-message-chore · rep 1 · accepted · 243→2603 tok · $0.0063
no patch produced
error-message-chore · rep 2 · patch_apply_failed · 243→2590 tok · $0.0063
no patch produced
generated-cli-help-chore · rep 0 · accepted · 202→1919 tok · $0.0047
no patch produced
generated-cli-help-chore · rep 1 · accepted · 202→1592 tok · $0.0039
no patch produced
generated-cli-help-chore · rep 2 · accepted · 202→3222 tok · $0.0078
no patch produced
generated-client-field-chore · rep 0 · patch_apply_failed · 254→2472 tok · $0.0060
no patch produced
generated-client-field-chore · rep 1 · accepted · 254→2426 tok · $0.0059
no patch produced
generated-client-field-chore · rep 2 · patch_apply_failed · 254→2357 tok · $0.0057
no patch produced
nextflow-orchestration-chore · rep 0 · forbidden_pattern, hidden_tests_failed, smoke_failed · 450→4505 tok · $0.0109
no patch produced
nextflow-orchestration-chore · rep 1 · patch_apply_failed · 450→4883 tok · $0.0118
no patch produced
nextflow-orchestration-chore · rep 2 · patch_apply_failed · 450→3950 tok · $0.0096
no patch produced
project-manager-chore · rep 0 · patch_apply_failed · 288→3484 tok · $0.0084
no patch produced
project-manager-chore · rep 0 · patch_apply_failed · 288→3670 tok · $0.0089
no patch produced
project-manager-chore · rep 1 · patch_apply_failed · 288→2872 tok · $0.0070
no patch produced
project-manager-chore · rep 1 · patch_apply_failed · 288→2068 tok · $0.0050
no patch produced
project-manager-chore · rep 2 · patch_apply_failed · 288→4101 tok · $0.0099
no patch produced
pyproject-edit-chore · rep 0 · patch_apply_failed · 245→434 tok · $0.0011
no patch produced
pyproject-edit-chore · rep 1 · patch_apply_failed · 245→2036 tok · $0.0050
no patch produced
pyproject-edit-chore · rep 2 · patch_apply_failed · 245→484 tok · $0.0012
no patch produced
tiny-python-type-chore · rep 0 · accepted · 236→3317 tok · $0.0080
no patch produced
tiny-python-type-chore · rep 0 · accepted · 174→2463 tok · $0.0060
no patch produced
tiny-python-type-chore · rep 1 · patch_apply_failed · 236→2997 tok · $0.0073
no patch produced
tiny-python-type-chore · rep 1 · patch_apply_failed · 174→1928 tok · $0.0047
no patch produced
tiny-python-type-chore · rep 2 · accepted · 236→3172 tok · $0.0077
no patch produced
type-annotation-chore · rep 0 · accepted · 192→2573 tok · $0.0062
no patch produced
type-annotation-chore · rep 0 · accepted · 192→1647 tok · $0.0040
no patch produced
type-annotation-chore · rep 1 · accepted · 192→1058 tok · $0.0026
no patch produced
type-annotation-chore · rep 1 · accepted · 192→2655 tok · $0.0064
no patch produced
type-annotation-chore · rep 2 · accepted · 192→2759 tok · $0.0067
no patch produced
version-hallucination-chore · rep 0 · patch_apply_failed · 237→2164 tok · $0.0053
no patch produced
version-hallucination-chore · rep 1 · accepted · 237→892 tok · $0.0022
no patch produced
version-hallucination-chore · rep 2 · accepted · 237→1528 tok · $0.0037
no patch produced