← ChoreCode leaderboard · 20260706T163525Z
type_integrity
1 public chores · 18/43 accepted · 0 hard fails
Tasks
category coverage| Task | Title | Accepted | Hard fails |
|---|---|---|---|
| tiny-python-type-chore | Preserve fractional mean results | 18/43 (42%) | 0 |
Results
model × task| Task | Run config | Accepted | Hard fails |
|---|---|---|---|
| tiny-python-type-chore | deepseek-v4-flash | 2/5 | 0 |
| tiny-python-type-chore | deepseek-v4-pro | 0/5 | 0 |
| tiny-python-type-chore | gemma4-26b | 4/8 | 0 |
| tiny-python-type-chore | glm-5.2 | 1/5 | 0 |
| tiny-python-type-chore | kimi-k2.6 | 2/5 | 0 |
| tiny-python-type-chore | minimax-m2.7 | 2/5 | 0 |
| tiny-python-type-chore | minimax-m3 | 4/5 | 0 |
| tiny-python-type-chore | qwen3.6-27b | 3/5 | 0 |
Attempts
evidence for this categorytiny-python-type-chore · deepseek-v4-flash · rep 0 · accepted · 215→296 tok · $0.000073
no patch produced
tiny-python-type-chore · deepseek-v4-flash · rep 0 · hidden_tests_failed · 159→610 tok · $0.00012
no patch produced
tiny-python-type-chore · deepseek-v4-flash · rep 1 · hidden_tests_failed · 159→496 tok · $0.00010
no patch produced
tiny-python-type-chore · deepseek-v4-flash · rep 1 · accepted · 215→450 tok · $0.00010
no patch produced
tiny-python-type-chore · deepseek-v4-flash · rep 2 · hidden_tests_failed, smoke_failed · 215→771 tok · $0.00016
no patch produced
tiny-python-type-chore · deepseek-v4-pro · rep 0 · hidden_tests_failed · 159→606 tok · $0.0006
no patch produced
tiny-python-type-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 215→382 tok · $0.00043
no patch produced
tiny-python-type-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 215→494 tok · $0.0005
no patch produced
tiny-python-type-chore · deepseek-v4-pro · rep 1 · hidden_tests_failed · 159→357 tok · $0.00038
no patch produced
tiny-python-type-chore · deepseek-v4-pro · rep 2 · patch_apply_failed · 215→596 tok · $0.0006
no patch produced
tiny-python-type-chore · gemma4-26b · rep 0 · patch_apply_failed · 252→1834 tok · $0.0006
no patch produced
tiny-python-type-chore · gemma4-26b · rep 0 · patch_apply_failed · 185→2835 tok · $0.0009
no patch produced
tiny-python-type-chore · gemma4-26b · rep 0 · accepted · 252→3713 tok · $0.0012
no patch produced
tiny-python-type-chore · gemma4-26b · rep 1 · accepted · 252→4816 tok · $0.0016
no patch produced
tiny-python-type-chore · gemma4-26b · rep 1 · patch_apply_failed · 185→3203 tok · $0.0011
no patch produced
tiny-python-type-chore · gemma4-26b · rep 1 · accepted · 252→2684 tok · $0.0009
no patch produced
tiny-python-type-chore · gemma4-26b · rep 2 · patch_apply_failed · 252→2816 tok · $0.0009
no patch produced
tiny-python-type-chore · gemma4-26b · rep 2 · accepted · 252→2280 tok · $0.0008
no patch produced
tiny-python-type-chore · glm-5.2 · rep 0 · accepted · 222→308 tok · $0.0007
no patch produced
tiny-python-type-chore · glm-5.2 · rep 0 · patch_apply_failed · 166→189 tok · $0.00043
no patch produced
tiny-python-type-chore · glm-5.2 · rep 1 · patch_apply_failed · 222→6446 tok · $0.0115
no patch produced
tiny-python-type-chore · glm-5.2 · rep 1 · patch_apply_failed · 166→544 tok · $0.0011
no patch produced
tiny-python-type-chore · glm-5.2 · rep 2 · patch_apply_failed · 222→479 tok · $0.0010
no patch produced
tiny-python-type-chore · kimi-k2.6 · rep 0 · patch_apply_failed · 167→2238 tok · $0.0077
no patch produced
tiny-python-type-chore · kimi-k2.6 · rep 0 · patch_apply_failed · 222→12239 tok · $0.0419
no patch produced
tiny-python-type-chore · kimi-k2.6 · rep 1 · patch_apply_failed · 167→16384 tok · $0.0560
no patch produced
tiny-python-type-chore · kimi-k2.6 · rep 1 · accepted · 222→1564 tok · $0.0055
no patch produced
tiny-python-type-chore · kimi-k2.6 · rep 2 · accepted · 222→2816 tok · $0.0097
no patch produced
tiny-python-type-chore · minimax-m2.7 · rep 0 · accepted · 170→560 tok · $0.00043
no patch produced
tiny-python-type-chore · minimax-m2.7 · rep 0 · accepted · 224→492 tok · $0.00039
no patch produced
tiny-python-type-chore · minimax-m2.7 · rep 1 · hidden_tests_failed · 170→494 tok · $0.00039
no patch produced
tiny-python-type-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 224→624 tok · $0.00049
no patch produced
tiny-python-type-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 224→1002 tok · $0.0008
no patch produced
tiny-python-type-chore · minimax-m3 · rep 0 · accepted · 380→814 tok · $0.0011
no patch produced
tiny-python-type-chore · minimax-m3 · rep 0 · accepted · 326→968 tok · $0.0013
no patch produced
tiny-python-type-chore · minimax-m3 · rep 1 · accepted · 326→978 tok · $0.0013
no patch produced
tiny-python-type-chore · minimax-m3 · rep 1 · accepted · 380→981 tok · $0.0013
no patch produced
tiny-python-type-chore · minimax-m3 · rep 2 · hidden_tests_failed, smoke_failed · 380→1595 tok · $0.0020
no patch produced
tiny-python-type-chore · qwen3.6-27b · rep 0 · accepted · 236→3317 tok · $0.0080
no patch produced
tiny-python-type-chore · qwen3.6-27b · rep 0 · accepted · 174→2463 tok · $0.0060
no patch produced
tiny-python-type-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 236→2997 tok · $0.0073
no patch produced
tiny-python-type-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 174→1928 tok · $0.0047
no patch produced
tiny-python-type-chore · qwen3.6-27b · rep 2 · accepted · 236→3172 tok · $0.0077
no patch produced