← ChoreCode leaderboard · 20260706T163525Z
bare-except-chore
Stop swallowing lookup errors
tiny_store.lookup.get_price documents that it raises ValueError for unknown items, but instead it swallows every error and returns 0. Fix it to raise ValueError for unknown items while letting unrelated errors propagate. Return a unified diff only.
Forbidden cheap shots: modifying or deleting visible tests; keeping a broad except while special-casing the hidden test inputs
Results by model
8 run configs| Run config | Accepted | Accept rate | Hard fails |
|---|---|---|---|
| gemma4-26b | 5/8 | 63% | 0 |
| qwen3.6-27b | 5/5 | 100% | 0 |
| glm-5.2 | 4/5 | 80% | 0 |
| kimi-k2.6 | 3/5 | 60% | 0 |
| deepseek-v4-flash | 2/5 | 40% | 0 |
| deepseek-v4-pro | 1/5 | 20% | 0 |
| minimax-m2.7 | 0/5 | 0% | 0 |
| minimax-m3 | 0/5 | 0% | 0 |
Attempts
expand a row for the patch and logsdeepseek-v4-flash · rep 0 · accepted · 190→426 tok · $0.000094
no patch produced
deepseek-v4-flash · rep 0 · patch_apply_failed · 190→272 tok · $0.000066
no patch produced
deepseek-v4-flash · rep 1 · accepted · 190→294 tok · $0.000070
no patch produced
deepseek-v4-flash · rep 1 · patch_apply_failed · 190→278 tok · $0.000067
no patch produced
deepseek-v4-flash · rep 2 · patch_apply_failed · 190→305 tok · $0.000072
no patch produced
deepseek-v4-pro · rep 0 · patch_apply_failed · 190→505 tok · $0.0005
no patch produced
deepseek-v4-pro · rep 0 · accepted · 190→279 tok · $0.00033
no patch produced
deepseek-v4-pro · rep 1 · patch_apply_failed · 190→507 tok · $0.0005
no patch produced
deepseek-v4-pro · rep 1 · patch_apply_failed · 190→586 tok · $0.0006
no patch produced
deepseek-v4-pro · rep 2 · patch_apply_failed · 190→821 tok · $0.0008
no patch produced
gemma4-26b · rep 0 · accepted · 217→9885 tok · $0.0033
no patch produced
gemma4-26b · rep 0 · accepted · 217→11005 tok · $0.0036
no patch produced
gemma4-26b · rep 0 · accepted · 217→10019 tok · $0.0033
no patch produced
gemma4-26b · rep 1 · patch_apply_failed · 217→8419 tok · $0.0028
no patch produced
gemma4-26b · rep 1 · accepted · 217→9421 tok · $0.0031
no patch produced
gemma4-26b · rep 1 · accepted · 217→8102 tok · $0.0027
no patch produced
gemma4-26b · rep 2 · patch_apply_failed · 217→5949 tok · $0.0020
no patch produced
gemma4-26b · rep 2 · patch_apply_failed · 217→7301 tok · $0.0024
no patch produced
glm-5.2 · rep 0 · accepted · 196→533 tok · $0.0010
no patch produced
glm-5.2 · rep 0 · accepted · 196→240 tok · $0.0005
no patch produced
glm-5.2 · rep 1 · patch_apply_failed · 196→162 tok · $0.00039
no patch produced
glm-5.2 · rep 1 · accepted · 196→760 tok · $0.0014
no patch produced
glm-5.2 · rep 2 · accepted · 196→986 tok · $0.0018
no patch produced
kimi-k2.6 · rep 0 · accepted · 198→1740 tok · $0.0061
no patch produced
kimi-k2.6 · rep 0 · patch_apply_failed · 198→2441 tok · $0.0085
no patch produced
kimi-k2.6 · rep 1 · patch_apply_failed · 198→1991 tok · $0.0069
no patch produced
kimi-k2.6 · rep 1 · accepted · 198→1887 tok · $0.0066
no patch produced
kimi-k2.6 · rep 2 · accepted · 198→4714 tok · $0.0162
no patch produced
minimax-m2.7 · rep 0 · patch_apply_failed · 0→603 tok · $0.00043
no patch produced
minimax-m2.7 · rep 0 · patch_apply_failed · 0→608 tok · $0.00044
no patch produced
minimax-m2.7 · rep 1 · patch_apply_failed · 0→366 tok · $0.00026
no patch produced
minimax-m2.7 · rep 1 · patch_apply_failed · 199→348 tok · $0.00029
no patch produced
minimax-m2.7 · rep 2 · patch_apply_failed · 0→367 tok · $0.00026
no patch produced
minimax-m3 · rep 0 · patch_apply_failed · 355→175 tok · $0.00032
no patch produced
minimax-m3 · rep 0 · patch_apply_failed · 355→171 tok · $0.00031
no patch produced
minimax-m3 · rep 1 · patch_apply_failed · 355→703 tok · $0.0010
no patch produced
minimax-m3 · rep 1 · patch_apply_failed · 355→352 tok · $0.0005
no patch produced
minimax-m3 · rep 2 · patch_apply_failed · 355→176 tok · $0.00032
no patch produced
qwen3.6-27b · rep 0 · accepted · 207→1859 tok · $0.0045
no patch produced
qwen3.6-27b · rep 0 · accepted · 207→1865 tok · $0.0045
no patch produced
qwen3.6-27b · rep 1 · accepted · 207→2107 tok · $0.0051
no patch produced
qwen3.6-27b · rep 1 · accepted · 207→2774 tok · $0.0067
no patch produced
qwen3.6-27b · rep 2 · accepted · 207→2738 tok · $0.0066
no patch produced