← ChoreCode leaderboard · 20260706T163525Z
deepseek-v4-pro
deepseek-v4-pro:cloud
Stats
release-0.1.1| Model | deepseek-v4-pro:cloud |
| Accepted | 28/69 (41%) |
| Hard fail rate | 4% |
| $ / accepted chore | $0.0016 |
| $ / attempt | $0.0007 |
| Tokens / accepted chore | 2156 |
| Tokens in / out (total) | 15.3k / 45.1k |
| Mean latency | 5.3 s |
Cost is recorded input/output tokens multiplied by current OpenRouter pricing for the same model or weights.
Results by task
17 tasks| Task | Accepted | Hard fails | Failure reasons seen |
|---|---|---|---|
| bare-except-chore | 1/5 | 0 | patch_apply_failed |
| basic-function-off-by-one | 1/5 | 0 | patch_apply_failed |
| config-noop-chore | 2/5 | 0 | patch_apply_failed |
| debug-cruft-chore | 2/5 | 1 | debug_output_left, forbidden_pattern, hidden_tests_failed, patch_apply_failed |
| dependency-conflict-chore | 2/3 | 0 | patch_apply_failed |
| dependency-existing-retry-chore | 0/3 | 0 | patch_apply_failed |
| dependency-stdlib-query-chore | 1/3 | 0 | patch_apply_failed |
| docstring-chore | 1/5 | 0 | patch_apply_failed, existing_tests_failed, hidden_tests_failed, smoke_failed |
| error-message-chore | 3/5 | 0 | patch_apply_failed |
| generated-cli-help-chore | 1/3 | 1 | generated_drift, hidden_tests_failed, patch_apply_failed |
| generated-client-field-chore | 3/3 | 0 | — |
| nextflow-orchestration-chore | 0/3 | 1 | patch_apply_failed, forbidden_pattern, hidden_tests_failed, smoke_failed |
| project-manager-chore | 0/5 | 0 | patch_apply_failed |
| pyproject-edit-chore | 3/3 | 0 | — |
| tiny-python-type-chore | 0/5 | 0 | hidden_tests_failed, patch_apply_failed |
| type-annotation-chore | 5/5 | 0 | — |
| version-hallucination-chore | 3/3 | 0 | — |
Attempts
expand a row for the patch and logsbare-except-chore · rep 0 · patch_apply_failed · 190→505 tok · $0.0005
no patch produced
bare-except-chore · rep 0 · accepted · 190→279 tok · $0.00033
no patch produced
bare-except-chore · rep 1 · patch_apply_failed · 190→507 tok · $0.0005
no patch produced
bare-except-chore · rep 1 · patch_apply_failed · 190→586 tok · $0.0006
no patch produced
bare-except-chore · rep 2 · patch_apply_failed · 190→821 tok · $0.0008
no patch produced
basic-function-off-by-one · rep 0 · patch_apply_failed · 179→312 tok · $0.00035
no patch produced
basic-function-off-by-one · rep 0 · patch_apply_failed · 179→269 tok · $0.00031
no patch produced
basic-function-off-by-one · rep 1 · accepted · 179→537 tok · $0.0005
no patch produced
basic-function-off-by-one · rep 1 · patch_apply_failed · 179→204 tok · $0.00026
no patch produced
basic-function-off-by-one · rep 2 · patch_apply_failed · 179→175 tok · $0.00023
no patch produced
config-noop-chore · rep 0 · accepted · 182→2239 tok · $0.0020
no patch produced
config-noop-chore · rep 0 · patch_apply_failed · 182→1115 tok · $0.0010
no patch produced
config-noop-chore · rep 1 · patch_apply_failed · 182→1279 tok · $0.0012
no patch produced
config-noop-chore · rep 1 · accepted · 182→223 tok · $0.00027
no patch produced
config-noop-chore · rep 2 · patch_apply_failed · 182→1088 tok · $0.0010
no patch produced
debug-cruft-chore · rep 0 · debug_output_left, forbidden_pattern, hidden_tests_failed · 188→204 tok · $0.00026
no patch produced
debug-cruft-chore · rep 0 · accepted · 188→1637 tok · $0.0015
no patch produced
debug-cruft-chore · rep 1 · patch_apply_failed · 188→195 tok · $0.00025
no patch produced
debug-cruft-chore · rep 1 · patch_apply_failed · 188→254 tok · $0.00030
no patch produced
debug-cruft-chore · rep 2 · accepted · 188→551 tok · $0.0006
no patch produced
dependency-conflict-chore · rep 0 · patch_apply_failed · 272→434 tok · $0.00050
no patch produced
dependency-conflict-chore · rep 1 · accepted · 272→524 tok · $0.0006
no patch produced
dependency-conflict-chore · rep 2 · accepted · 272→448 tok · $0.0005
no patch produced
dependency-existing-retry-chore · rep 0 · patch_apply_failed · 294→1403 tok · $0.0013
no patch produced
dependency-existing-retry-chore · rep 1 · patch_apply_failed · 294→640 tok · $0.0007
no patch produced
dependency-existing-retry-chore · rep 2 · patch_apply_failed · 294→1051 tok · $0.0010
no patch produced
dependency-stdlib-query-chore · rep 0 · patch_apply_failed · 284→748 tok · $0.0008
no patch produced
dependency-stdlib-query-chore · rep 1 · patch_apply_failed · 284→392 tok · $0.00046
no patch produced
dependency-stdlib-query-chore · rep 2 · accepted · 284→514 tok · $0.0006
no patch produced
docstring-chore · rep 0 · patch_apply_failed · 147→274 tok · $0.00030
no patch produced
docstring-chore · rep 0 · patch_apply_failed · 147→659 tok · $0.0006
no patch produced
docstring-chore · rep 1 · accepted · 147→282 tok · $0.00031
no patch produced
docstring-chore · rep 1 · existing_tests_failed, hidden_tests_failed, smoke_failed · 147→334 tok · $0.00035
no patch produced
docstring-chore · rep 2 · patch_apply_failed · 147→319 tok · $0.00034
no patch produced
error-message-chore · rep 0 · accepted · 229→251 tok · $0.00032
no patch produced
error-message-chore · rep 0 · accepted · 229→1004 tok · $0.0010
no patch produced
error-message-chore · rep 1 · patch_apply_failed · 229→400 tok · $0.00045
no patch produced
error-message-chore · rep 1 · accepted · 229→1132 tok · $0.0011
no patch produced
error-message-chore · rep 2 · patch_apply_failed · 229→271 tok · $0.00034
no patch produced
generated-cli-help-chore · rep 0 · accepted · 189→493 tok · $0.0005
no patch produced
generated-cli-help-chore · rep 1 · generated_drift, hidden_tests_failed · 189→374 tok · $0.00041
no patch produced
generated-cli-help-chore · rep 2 · patch_apply_failed · 189→404 tok · $0.00043
no patch produced
generated-client-field-chore · rep 0 · accepted · 229→636 tok · $0.0007
no patch produced
generated-client-field-chore · rep 1 · accepted · 229→477 tok · $0.0005
no patch produced
generated-client-field-chore · rep 2 · accepted · 229→608 tok · $0.0006
no patch produced
nextflow-orchestration-chore · rep 0 · patch_apply_failed · 440→2771 tok · $0.0026
no patch produced
nextflow-orchestration-chore · rep 1 · patch_apply_failed · 440→1454 tok · $0.0015
no patch produced
nextflow-orchestration-chore · rep 2 · forbidden_pattern, hidden_tests_failed, smoke_failed · 440→1902 tok · $0.0018
no patch produced
project-manager-chore · rep 0 · patch_apply_failed · 275→418 tok · $0.00048
no patch produced
project-manager-chore · rep 0 · patch_apply_failed · 275→893 tok · $0.0009
no patch produced
project-manager-chore · rep 1 · patch_apply_failed · 275→382 tok · $0.00045
no patch produced
project-manager-chore · rep 1 · patch_apply_failed · 275→379 tok · $0.00045
no patch produced
project-manager-chore · rep 2 · patch_apply_failed · 275→1185 tok · $0.0012
no patch produced
pyproject-edit-chore · rep 0 · accepted · 235→227 tok · $0.00030
no patch produced
pyproject-edit-chore · rep 1 · accepted · 235→203 tok · $0.00028
no patch produced
pyproject-edit-chore · rep 2 · accepted · 235→194 tok · $0.00027
no patch produced
tiny-python-type-chore · rep 0 · hidden_tests_failed · 159→606 tok · $0.0006
no patch produced
tiny-python-type-chore · rep 0 · patch_apply_failed · 215→382 tok · $0.00043
no patch produced
tiny-python-type-chore · rep 1 · patch_apply_failed · 215→494 tok · $0.0005
no patch produced
tiny-python-type-chore · rep 1 · hidden_tests_failed · 159→357 tok · $0.00038
no patch produced
tiny-python-type-chore · rep 2 · patch_apply_failed · 215→596 tok · $0.0006
no patch produced
type-annotation-chore · rep 0 · accepted · 178→598 tok · $0.0006
no patch produced
type-annotation-chore · rep 0 · accepted · 178→1161 tok · $0.0011
no patch produced
type-annotation-chore · rep 1 · accepted · 178→495 tok · $0.0005
no patch produced
type-annotation-chore · rep 1 · accepted · 178→1336 tok · $0.0012
no patch produced
type-annotation-chore · rep 2 · accepted · 178→762 tok · $0.0007
no patch produced
version-hallucination-chore · rep 0 · accepted · 224→322 tok · $0.00038
no patch produced
version-hallucination-chore · rep 1 · accepted · 224→433 tok · $0.00047
no patch produced
version-hallucination-chore · rep 2 · accepted · 224→473 tok · $0.0005
no patch produced