← ChoreCode leaderboard · 20260706T163525Z
type_annotation
1 public chores · 30/43 accepted · 0 hard fails
Tasks
category coverage| Task | Title | Accepted | Hard fails |
|---|---|---|---|
| type-annotation-chore | Annotate the distance converter precisely | 30/43 (70%) | 0 |
Results
model × task| Task | Run config | Accepted | Hard fails |
|---|---|---|---|
| type-annotation-chore | deepseek-v4-flash | 4/5 | 0 |
| type-annotation-chore | deepseek-v4-pro | 5/5 | 0 |
| type-annotation-chore | gemma4-26b | 8/8 | 0 |
| type-annotation-chore | glm-5.2 | 3/5 | 0 |
| type-annotation-chore | kimi-k2.6 | 3/5 | 0 |
| type-annotation-chore | minimax-m2.7 | 1/5 | 0 |
| type-annotation-chore | minimax-m3 | 1/5 | 0 |
| type-annotation-chore | qwen3.6-27b | 5/5 | 0 |
Attempts
evidence for this categorytype-annotation-chore · deepseek-v4-flash · rep 0 · accepted · 178→269 tok · $0.000064
no patch produced
type-annotation-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 178→323 tok · $0.000074
no patch produced
type-annotation-chore · deepseek-v4-flash · rep 1 · accepted · 178→285 tok · $0.000067
no patch produced
type-annotation-chore · deepseek-v4-flash · rep 1 · accepted · 178→252 tok · $0.000061
no patch produced
type-annotation-chore · deepseek-v4-flash · rep 2 · accepted · 178→212 tok · $0.000054
no patch produced
type-annotation-chore · deepseek-v4-pro · rep 0 · accepted · 178→598 tok · $0.0006
no patch produced
type-annotation-chore · deepseek-v4-pro · rep 0 · accepted · 178→1161 tok · $0.0011
no patch produced
type-annotation-chore · deepseek-v4-pro · rep 1 · accepted · 178→495 tok · $0.0005
no patch produced
type-annotation-chore · deepseek-v4-pro · rep 1 · accepted · 178→1336 tok · $0.0012
no patch produced
type-annotation-chore · deepseek-v4-pro · rep 2 · accepted · 178→762 tok · $0.0007
no patch produced
type-annotation-chore · gemma4-26b · rep 0 · accepted · 205→2468 tok · $0.0008
no patch produced
type-annotation-chore · gemma4-26b · rep 0 · accepted · 205→682 tok · $0.00024
no patch produced
type-annotation-chore · gemma4-26b · rep 0 · accepted · 205→1946 tok · $0.0007
no patch produced
type-annotation-chore · gemma4-26b · rep 1 · accepted · 205→2412 tok · $0.0008
no patch produced
type-annotation-chore · gemma4-26b · rep 1 · accepted · 205→808 tok · $0.00028
no patch produced
type-annotation-chore · gemma4-26b · rep 1 · accepted · 205→2565 tok · $0.0009
no patch produced
type-annotation-chore · gemma4-26b · rep 2 · accepted · 205→555 tok · $0.00020
no patch produced
type-annotation-chore · gemma4-26b · rep 2 · accepted · 205→719 tok · $0.00025
no patch produced
type-annotation-chore · glm-5.2 · rep 0 · accepted · 183→139 tok · $0.00035
no patch produced
type-annotation-chore · glm-5.2 · rep 0 · patch_apply_failed · 183→142 tok · $0.00035
no patch produced
type-annotation-chore · glm-5.2 · rep 1 · accepted · 183→139 tok · $0.00035
no patch produced
type-annotation-chore · glm-5.2 · rep 1 · patch_apply_failed · 183→150 tok · $0.00037
no patch produced
type-annotation-chore · glm-5.2 · rep 2 · accepted · 183→186 tok · $0.00043
no patch produced
type-annotation-chore · kimi-k2.6 · rep 0 · accepted · 182→3305 tok · $0.0114
no patch produced
type-annotation-chore · kimi-k2.6 · rep 0 · patch_apply_failed · 182→3996 tok · $0.0137
no patch produced
type-annotation-chore · kimi-k2.6 · rep 1 · accepted · 182→1920 tok · $0.0067
no patch produced
type-annotation-chore · kimi-k2.6 · rep 1 · accepted · 182→2463 tok · $0.0085
no patch produced
type-annotation-chore · kimi-k2.6 · rep 2 · patch_apply_failed · 182→1076 tok · $0.0038
no patch produced
type-annotation-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 181→309 tok · $0.00026
no patch produced
type-annotation-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 181→251 tok · $0.00021
no patch produced
type-annotation-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 181→370 tok · $0.00030
no patch produced
type-annotation-chore · minimax-m2.7 · rep 1 · accepted · 0→253 tok · $0.00018
no patch produced
type-annotation-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 0→380 tok · $0.00027
no patch produced
type-annotation-chore · minimax-m3 · rep 0 · accepted · 337→179 tok · $0.00032
no patch produced
type-annotation-chore · minimax-m3 · rep 0 · patch_apply_failed · 337→440 tok · $0.0006
no patch produced
type-annotation-chore · minimax-m3 · rep 1 · patch_apply_failed · 337→202 tok · $0.00034
no patch produced
type-annotation-chore · minimax-m3 · rep 1 · patch_apply_failed · 337→161 tok · $0.00029
no patch produced
type-annotation-chore · minimax-m3 · rep 2 · patch_apply_failed · 337→369 tok · $0.0005
no patch produced
type-annotation-chore · qwen3.6-27b · rep 0 · accepted · 192→2573 tok · $0.0062
no patch produced
type-annotation-chore · qwen3.6-27b · rep 0 · accepted · 192→1647 tok · $0.0040
no patch produced
type-annotation-chore · qwen3.6-27b · rep 1 · accepted · 192→1058 tok · $0.0026
no patch produced
type-annotation-chore · qwen3.6-27b · rep 1 · accepted · 192→2655 tok · $0.0064
no patch produced
type-annotation-chore · qwen3.6-27b · rep 2 · accepted · 192→2759 tok · $0.0067
no patch produced