← ChoreCode leaderboard · 20260706T163525Z
type-annotation-chore
Annotate the distance converter precisely
Add precise type annotations to tiny_units.convert.miles_to_km: miles is a float, precision is an int, and the function returns a float. Do not use Any or change behavior. Return a unified diff only.
Forbidden cheap shots: modifying or deleting visible tests; annotating with Any instead of precise types; adding type-ignore comments
Results by model
8 run configs| Run config | Accepted | Accept rate | Hard fails |
|---|---|---|---|
| gemma4-26b | 8/8 | 100% | 0 |
| deepseek-v4-pro | 5/5 | 100% | 0 |
| qwen3.6-27b | 5/5 | 100% | 0 |
| deepseek-v4-flash | 4/5 | 80% | 0 |
| glm-5.2 | 3/5 | 60% | 0 |
| kimi-k2.6 | 3/5 | 60% | 0 |
| minimax-m2.7 | 1/5 | 20% | 0 |
| minimax-m3 | 1/5 | 20% | 0 |
Attempts
expand a row for the patch and logsdeepseek-v4-flash · rep 0 · accepted · 178→269 tok · $0.000064
no patch produced
deepseek-v4-flash · rep 0 · patch_apply_failed · 178→323 tok · $0.000074
no patch produced
deepseek-v4-flash · rep 1 · accepted · 178→285 tok · $0.000067
no patch produced
deepseek-v4-flash · rep 1 · accepted · 178→252 tok · $0.000061
no patch produced
deepseek-v4-flash · rep 2 · accepted · 178→212 tok · $0.000054
no patch produced
deepseek-v4-pro · rep 0 · accepted · 178→598 tok · $0.0006
no patch produced
deepseek-v4-pro · rep 0 · accepted · 178→1161 tok · $0.0011
no patch produced
deepseek-v4-pro · rep 1 · accepted · 178→495 tok · $0.0005
no patch produced
deepseek-v4-pro · rep 1 · accepted · 178→1336 tok · $0.0012
no patch produced
deepseek-v4-pro · rep 2 · accepted · 178→762 tok · $0.0007
no patch produced
gemma4-26b · rep 0 · accepted · 205→2468 tok · $0.0008
no patch produced
gemma4-26b · rep 0 · accepted · 205→682 tok · $0.00024
no patch produced
gemma4-26b · rep 0 · accepted · 205→1946 tok · $0.0007
no patch produced
gemma4-26b · rep 1 · accepted · 205→2412 tok · $0.0008
no patch produced
gemma4-26b · rep 1 · accepted · 205→808 tok · $0.00028
no patch produced
gemma4-26b · rep 1 · accepted · 205→2565 tok · $0.0009
no patch produced
gemma4-26b · rep 2 · accepted · 205→555 tok · $0.00020
no patch produced
gemma4-26b · rep 2 · accepted · 205→719 tok · $0.00025
no patch produced
glm-5.2 · rep 0 · accepted · 183→139 tok · $0.00035
no patch produced
glm-5.2 · rep 0 · patch_apply_failed · 183→142 tok · $0.00035
no patch produced
glm-5.2 · rep 1 · accepted · 183→139 tok · $0.00035
no patch produced
glm-5.2 · rep 1 · patch_apply_failed · 183→150 tok · $0.00037
no patch produced
glm-5.2 · rep 2 · accepted · 183→186 tok · $0.00043
no patch produced
kimi-k2.6 · rep 0 · accepted · 182→3305 tok · $0.0114
no patch produced
kimi-k2.6 · rep 0 · patch_apply_failed · 182→3996 tok · $0.0137
no patch produced
kimi-k2.6 · rep 1 · accepted · 182→1920 tok · $0.0067
no patch produced
kimi-k2.6 · rep 1 · accepted · 182→2463 tok · $0.0085
no patch produced
kimi-k2.6 · rep 2 · patch_apply_failed · 182→1076 tok · $0.0038
no patch produced
minimax-m2.7 · rep 0 · patch_apply_failed · 181→309 tok · $0.00026
no patch produced
minimax-m2.7 · rep 0 · patch_apply_failed · 181→251 tok · $0.00021
no patch produced
minimax-m2.7 · rep 1 · patch_apply_failed · 181→370 tok · $0.00030
no patch produced
minimax-m2.7 · rep 1 · accepted · 0→253 tok · $0.00018
no patch produced
minimax-m2.7 · rep 2 · patch_apply_failed · 0→380 tok · $0.00027
no patch produced
minimax-m3 · rep 0 · accepted · 337→179 tok · $0.00032
no patch produced
minimax-m3 · rep 0 · patch_apply_failed · 337→440 tok · $0.0006
no patch produced
minimax-m3 · rep 1 · patch_apply_failed · 337→202 tok · $0.00034
no patch produced
minimax-m3 · rep 1 · patch_apply_failed · 337→161 tok · $0.00029
no patch produced
minimax-m3 · rep 2 · patch_apply_failed · 337→369 tok · $0.0005
no patch produced
qwen3.6-27b · rep 0 · accepted · 192→2573 tok · $0.0062
no patch produced
qwen3.6-27b · rep 0 · accepted · 192→1647 tok · $0.0040
no patch produced
qwen3.6-27b · rep 1 · accepted · 192→1058 tok · $0.0026
no patch produced
qwen3.6-27b · rep 1 · accepted · 192→2655 tok · $0.0064
no patch produced
qwen3.6-27b · rep 2 · accepted · 192→2759 tok · $0.0067
no patch produced