← ChoreCode leaderboard · 20260706T163525Z
error_handling
1 public chores · 20/43 accepted · 0 hard fails
Tasks
category coverage| Task | Title | Accepted | Hard fails |
|---|---|---|---|
| bare-except-chore | Stop swallowing lookup errors | 20/43 (47%) | 0 |
Results
model × task| Task | Run config | Accepted | Hard fails |
|---|---|---|---|
| bare-except-chore | deepseek-v4-flash | 2/5 | 0 |
| bare-except-chore | deepseek-v4-pro | 1/5 | 0 |
| bare-except-chore | gemma4-26b | 5/8 | 0 |
| bare-except-chore | glm-5.2 | 4/5 | 0 |
| bare-except-chore | kimi-k2.6 | 3/5 | 0 |
| bare-except-chore | minimax-m2.7 | 0/5 | 0 |
| bare-except-chore | minimax-m3 | 0/5 | 0 |
| bare-except-chore | qwen3.6-27b | 5/5 | 0 |
Attempts
evidence for this categorybare-except-chore · deepseek-v4-flash · rep 0 · accepted · 190→426 tok · $0.000094
no patch produced
bare-except-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 190→272 tok · $0.000066
no patch produced
bare-except-chore · deepseek-v4-flash · rep 1 · accepted · 190→294 tok · $0.000070
no patch produced
bare-except-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 190→278 tok · $0.000067
no patch produced
bare-except-chore · deepseek-v4-flash · rep 2 · patch_apply_failed · 190→305 tok · $0.000072
no patch produced
bare-except-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 190→505 tok · $0.0005
no patch produced
bare-except-chore · deepseek-v4-pro · rep 0 · accepted · 190→279 tok · $0.00033
no patch produced
bare-except-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 190→507 tok · $0.0005
no patch produced
bare-except-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 190→586 tok · $0.0006
no patch produced
bare-except-chore · deepseek-v4-pro · rep 2 · patch_apply_failed · 190→821 tok · $0.0008
no patch produced
bare-except-chore · gemma4-26b · rep 0 · accepted · 217→9885 tok · $0.0033
no patch produced
bare-except-chore · gemma4-26b · rep 0 · accepted · 217→11005 tok · $0.0036
no patch produced
bare-except-chore · gemma4-26b · rep 0 · accepted · 217→10019 tok · $0.0033
no patch produced
bare-except-chore · gemma4-26b · rep 1 · patch_apply_failed · 217→8419 tok · $0.0028
no patch produced
bare-except-chore · gemma4-26b · rep 1 · accepted · 217→9421 tok · $0.0031
no patch produced
bare-except-chore · gemma4-26b · rep 1 · accepted · 217→8102 tok · $0.0027
no patch produced
bare-except-chore · gemma4-26b · rep 2 · patch_apply_failed · 217→5949 tok · $0.0020
no patch produced
bare-except-chore · gemma4-26b · rep 2 · patch_apply_failed · 217→7301 tok · $0.0024
no patch produced
bare-except-chore · glm-5.2 · rep 0 · accepted · 196→533 tok · $0.0010
no patch produced
bare-except-chore · glm-5.2 · rep 0 · accepted · 196→240 tok · $0.0005
no patch produced
bare-except-chore · glm-5.2 · rep 1 · patch_apply_failed · 196→162 tok · $0.00039
no patch produced
bare-except-chore · glm-5.2 · rep 1 · accepted · 196→760 tok · $0.0014
no patch produced
bare-except-chore · glm-5.2 · rep 2 · accepted · 196→986 tok · $0.0018
no patch produced
bare-except-chore · kimi-k2.6 · rep 0 · accepted · 198→1740 tok · $0.0061
no patch produced
bare-except-chore · kimi-k2.6 · rep 0 · patch_apply_failed · 198→2441 tok · $0.0085
no patch produced
bare-except-chore · kimi-k2.6 · rep 1 · patch_apply_failed · 198→1991 tok · $0.0069
no patch produced
bare-except-chore · kimi-k2.6 · rep 1 · accepted · 198→1887 tok · $0.0066
no patch produced
bare-except-chore · kimi-k2.6 · rep 2 · accepted · 198→4714 tok · $0.0162
no patch produced
bare-except-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 0→603 tok · $0.00043
no patch produced
bare-except-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 0→608 tok · $0.00044
no patch produced
bare-except-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 0→366 tok · $0.00026
no patch produced
bare-except-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 199→348 tok · $0.00029
no patch produced
bare-except-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 0→367 tok · $0.00026
no patch produced
bare-except-chore · minimax-m3 · rep 0 · patch_apply_failed · 355→175 tok · $0.00032
no patch produced
bare-except-chore · minimax-m3 · rep 0 · patch_apply_failed · 355→171 tok · $0.00031
no patch produced
bare-except-chore · minimax-m3 · rep 1 · patch_apply_failed · 355→703 tok · $0.0010
no patch produced
bare-except-chore · minimax-m3 · rep 1 · patch_apply_failed · 355→352 tok · $0.0005
no patch produced
bare-except-chore · minimax-m3 · rep 2 · patch_apply_failed · 355→176 tok · $0.00032
no patch produced
bare-except-chore · qwen3.6-27b · rep 0 · accepted · 207→1859 tok · $0.0045
no patch produced
bare-except-chore · qwen3.6-27b · rep 0 · accepted · 207→1865 tok · $0.0045
no patch produced
bare-except-chore · qwen3.6-27b · rep 1 · accepted · 207→2107 tok · $0.0051
no patch produced
bare-except-chore · qwen3.6-27b · rep 1 · accepted · 207→2774 tok · $0.0067
no patch produced
bare-except-chore · qwen3.6-27b · rep 2 · accepted · 207→2738 tok · $0.0066
no patch produced