← ChoreCode leaderboard · 20260706T163525Z
toolchain_conventions
5 public chores · 54/151 accepted · 5 hard fails
Tasks
category coverage| Task | Title | Accepted | Hard fails |
|---|---|---|---|
| dependency-conflict-chore | Resolve the pydantic version conflict | 10/27 (37%) | 0 |
| nextflow-orchestration-chore | Replace ad hoc subprocess orchestration with Nextflow | 5/27 (19%) | 5 |
| project-manager-chore | Use the declared project manager for checks | 8/43 (19%) | 0 |
| pyproject-edit-chore | Declare the missing httpx dependency in pyproject.toml | 12/27 (44%) | 0 |
| version-hallucination-chore | Fix the hallucinated requests version pin | 19/27 (70%) | 0 |
Results
model × taskAttempts
evidence for this categorydependency-conflict-chore · deepseek-v4-flash · rep 0 · accepted · 272→274 tok · $0.000074
no patch produced
dependency-conflict-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 272→407 tok · $0.000098
no patch produced
dependency-conflict-chore · deepseek-v4-flash · rep 2 · patch_apply_failed · 272→412 tok · $0.000099
no patch produced
nextflow-orchestration-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 440→680 tok · $0.00016
no patch produced
nextflow-orchestration-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 440→1825 tok · $0.00037
no patch produced
nextflow-orchestration-chore · deepseek-v4-flash · rep 2 · patch_apply_failed · 440→2281 tok · $0.00045
no patch produced
project-manager-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 275→379 tok · $0.000093
no patch produced
project-manager-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 275→339 tok · $0.000086
no patch produced
project-manager-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 275→367 tok · $0.000091
no patch produced
project-manager-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 275→291 tok · $0.000077
no patch produced
project-manager-chore · deepseek-v4-flash · rep 2 · patch_apply_failed · 275→322 tok · $0.000083
no patch produced
pyproject-edit-chore · deepseek-v4-flash · rep 0 · patch_apply_failed · 235→145 tok · $0.000047
no patch produced
pyproject-edit-chore · deepseek-v4-flash · rep 1 · patch_apply_failed · 235→145 tok · $0.000047
no patch produced
pyproject-edit-chore · deepseek-v4-flash · rep 2 · patch_apply_failed · 235→142 tok · $0.000047
no patch produced
version-hallucination-chore · deepseek-v4-flash · rep 0 · accepted · 224→584 tok · $0.00013
no patch produced
version-hallucination-chore · deepseek-v4-flash · rep 1 · accepted · 224→310 tok · $0.000076
no patch produced
version-hallucination-chore · deepseek-v4-flash · rep 2 · accepted · 224→393 tok · $0.000091
no patch produced
dependency-conflict-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 272→434 tok · $0.00050
no patch produced
dependency-conflict-chore · deepseek-v4-pro · rep 1 · accepted · 272→524 tok · $0.0006
no patch produced
dependency-conflict-chore · deepseek-v4-pro · rep 2 · accepted · 272→448 tok · $0.0005
no patch produced
nextflow-orchestration-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 440→2771 tok · $0.0026
no patch produced
nextflow-orchestration-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 440→1454 tok · $0.0015
no patch produced
nextflow-orchestration-chore · deepseek-v4-pro · rep 2 · forbidden_pattern, hidden_tests_failed, smoke_failed · 440→1902 tok · $0.0018
no patch produced
project-manager-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 275→418 tok · $0.00048
no patch produced
project-manager-chore · deepseek-v4-pro · rep 0 · patch_apply_failed · 275→893 tok · $0.0009
no patch produced
project-manager-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 275→382 tok · $0.00045
no patch produced
project-manager-chore · deepseek-v4-pro · rep 1 · patch_apply_failed · 275→379 tok · $0.00045
no patch produced
project-manager-chore · deepseek-v4-pro · rep 2 · patch_apply_failed · 275→1185 tok · $0.0012
no patch produced
pyproject-edit-chore · deepseek-v4-pro · rep 0 · accepted · 235→227 tok · $0.00030
no patch produced
pyproject-edit-chore · deepseek-v4-pro · rep 1 · accepted · 235→203 tok · $0.00028
no patch produced
pyproject-edit-chore · deepseek-v4-pro · rep 2 · accepted · 235→194 tok · $0.00027
no patch produced
version-hallucination-chore · deepseek-v4-pro · rep 0 · accepted · 224→322 tok · $0.00038
no patch produced
version-hallucination-chore · deepseek-v4-pro · rep 1 · accepted · 224→433 tok · $0.00047
no patch produced
version-hallucination-chore · deepseek-v4-pro · rep 2 · accepted · 224→473 tok · $0.0005
no patch produced
dependency-conflict-chore · gemma4-26b · rep 0 · accepted · 299→3049 tok · $0.0010
no patch produced
dependency-conflict-chore · gemma4-26b · rep 0 · accepted · 299→4244 tok · $0.0014
no patch produced
dependency-conflict-chore · gemma4-26b · rep 1 · accepted · 299→4289 tok · $0.0014
no patch produced
dependency-conflict-chore · gemma4-26b · rep 1 · patch_apply_failed · 299→2797 tok · $0.0009
no patch produced
dependency-conflict-chore · gemma4-26b · rep 2 · accepted · 299→1568 tok · $0.0005
no patch produced
dependency-conflict-chore · gemma4-26b · rep 2 · accepted · 299→1962 tok · $0.0007
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 0 · patch_apply_failed · 484→9838 tok · $0.0033
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 0 · patch_apply_failed · 484→10962 tok · $0.0036
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 1 · forbidden_pattern, hidden_tests_failed, smoke_failed · 484→10938 tok · $0.0036
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 1 · forbidden_pattern, hidden_tests_failed, smoke_failed · 484→11640 tok · $0.0039
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 2 · patch_apply_failed · 484→10102 tok · $0.0034
no patch produced
nextflow-orchestration-chore · gemma4-26b · rep 2 · patch_apply_failed · 484→9286 tok · $0.0031
no patch produced
project-manager-chore · gemma4-26b · rep 0 · accepted · 301→5739 tok · $0.0019
no patch produced
project-manager-chore · gemma4-26b · rep 0 · patch_apply_failed · 301→4417 tok · $0.0015
no patch produced
project-manager-chore · gemma4-26b · rep 0 · patch_apply_failed · 301→1957 tok · $0.0007
no patch produced
project-manager-chore · gemma4-26b · rep 1 · accepted · 301→7318 tok · $0.0024
no patch produced
project-manager-chore · gemma4-26b · rep 1 · accepted · 301→5876 tok · $0.0020
no patch produced
project-manager-chore · gemma4-26b · rep 1 · accepted · 301→8691 tok · $0.0029
no patch produced
project-manager-chore · gemma4-26b · rep 2 · patch_apply_failed · 301→9565 tok · $0.0032
no patch produced
project-manager-chore · gemma4-26b · rep 2 · patch_apply_failed · 301→6440 tok · $0.0021
no patch produced
pyproject-edit-chore · gemma4-26b · rep 0 · accepted · 254→347 tok · $0.00013
no patch produced
pyproject-edit-chore · gemma4-26b · rep 0 · accepted · 254→256 tok · $0.00010
no patch produced
pyproject-edit-chore · gemma4-26b · rep 1 · accepted · 254→255 tok · $0.000099
no patch produced
pyproject-edit-chore · gemma4-26b · rep 1 · accepted · 254→1955 tok · $0.0007
no patch produced
pyproject-edit-chore · gemma4-26b · rep 2 · accepted · 254→218 tok · $0.000087
no patch produced
pyproject-edit-chore · gemma4-26b · rep 2 · accepted · 254→260 tok · $0.00010
no patch produced
version-hallucination-chore · gemma4-26b · rep 0 · accepted · 244→392 tok · $0.00014
no patch produced
version-hallucination-chore · gemma4-26b · rep 0 · accepted · 244→2408 tok · $0.0008
no patch produced
version-hallucination-chore · gemma4-26b · rep 1 · accepted · 244→2294 tok · $0.0008
no patch produced
version-hallucination-chore · gemma4-26b · rep 1 · accepted · 244→3474 tok · $0.0012
no patch produced
version-hallucination-chore · gemma4-26b · rep 2 · accepted · 244→1468 tok · $0.00050
no patch produced
version-hallucination-chore · gemma4-26b · rep 2 · accepted · 244→2169 tok · $0.0007
no patch produced
dependency-conflict-chore · glm-5.2 · rep 0 · existing_tests_failed, hidden_tests_failed, smoke_failed · 267→383 tok · $0.0008
no patch produced
dependency-conflict-chore · glm-5.2 · rep 1 · patch_apply_failed · 267→461 tok · $0.0010
no patch produced
dependency-conflict-chore · glm-5.2 · rep 2 · patch_apply_failed · 267→308 tok · $0.0007
no patch produced
nextflow-orchestration-chore · glm-5.2 · rep 0 · patch_apply_failed · 434→626 tok · $0.0013
no patch produced
nextflow-orchestration-chore · glm-5.2 · rep 1 · patch_apply_failed · 434→980 tok · $0.0020
no patch produced
nextflow-orchestration-chore · glm-5.2 · rep 2 · patch_apply_failed · 434→924 tok · $0.0019
no patch produced
project-manager-chore · glm-5.2 · rep 0 · patch_apply_failed · 271→513 tok · $0.0011
no patch produced
project-manager-chore · glm-5.2 · rep 0 · patch_apply_failed · 271→369 tok · $0.0008
no patch produced
project-manager-chore · glm-5.2 · rep 1 · patch_apply_failed · 271→245 tok · $0.0006
no patch produced
project-manager-chore · glm-5.2 · rep 1 · patch_apply_failed · 271→575 tok · $0.0012
no patch produced
project-manager-chore · glm-5.2 · rep 2 · patch_apply_failed · 271→218 tok · $0.0005
no patch produced
pyproject-edit-chore · glm-5.2 · rep 0 · patch_apply_failed · 232→78 tok · $0.00027
no patch produced
pyproject-edit-chore · glm-5.2 · rep 1 · patch_apply_failed · 232→78 tok · $0.00027
no patch produced
pyproject-edit-chore · glm-5.2 · rep 2 · patch_apply_failed · 232→78 tok · $0.00027
no patch produced
version-hallucination-chore · glm-5.2 · rep 0 · accepted · 222→190 tok · $0.00046
no patch produced
version-hallucination-chore · glm-5.2 · rep 1 · accepted · 222→157 tok · $0.00040
no patch produced
version-hallucination-chore · glm-5.2 · rep 2 · accepted · 222→132 tok · $0.00036
no patch produced
dependency-conflict-chore · kimi-k2.6 · rep 0 · patch_apply_failed · 256→2209 tok · $0.0077
no patch produced
dependency-conflict-chore · kimi-k2.6 · rep 1 · accepted · 256→2165 tok · $0.0076
no patch produced
dependency-conflict-chore · kimi-k2.6 · rep 2 · patch_apply_failed · 256→3157 tok · $0.0109
no patch produced
nextflow-orchestration-chore · kimi-k2.6 · rep 0 · accepted · 432→12918 tok · $0.0443
no patch produced
nextflow-orchestration-chore · kimi-k2.6 · rep 1 · forbidden_pattern, hidden_tests_failed, smoke_failed · 432→14136 tok · $0.0485
no patch produced
nextflow-orchestration-chore · kimi-k2.6 · rep 2 · accepted · 432→12750 tok · $0.0438
no patch produced
project-manager-chore · kimi-k2.6 · rep 0 · accepted · 267→8496 tok · $0.0291
no patch produced
project-manager-chore · kimi-k2.6 · rep 0 · accepted · 267→3566 tok · $0.0123
no patch produced
project-manager-chore · kimi-k2.6 · rep 1 · patch_apply_failed · 267→1695 tok · $0.0060
no patch produced
project-manager-chore · kimi-k2.6 · rep 1 · accepted · 267→3031 tok · $0.0105
no patch produced
project-manager-chore · kimi-k2.6 · rep 2 · patch_apply_failed · 267→2624 tok · $0.0091
no patch produced
pyproject-edit-chore · kimi-k2.6 · rep 0 · accepted · 225→2620 tok · $0.0091
no patch produced
pyproject-edit-chore · kimi-k2.6 · rep 1 · accepted · 225→6394 tok · $0.0220
no patch produced
pyproject-edit-chore · kimi-k2.6 · rep 2 · accepted · 225→3704 tok · $0.0128
no patch produced
version-hallucination-chore · kimi-k2.6 · rep 0 · accepted · 217→4142 tok · $0.0143
no patch produced
version-hallucination-chore · kimi-k2.6 · rep 1 · patch_apply_failed · 217→1867 tok · $0.0065
no patch produced
version-hallucination-chore · kimi-k2.6 · rep 2 · accepted · 217→3180 tok · $0.0110
no patch produced
dependency-conflict-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 266→725 tok · $0.0006
no patch produced
dependency-conflict-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 266→324 tok · $0.00028
no patch produced
dependency-conflict-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 266→757 tok · $0.0006
no patch produced
nextflow-orchestration-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 432→34302 tok · $0.0248
no patch produced
nextflow-orchestration-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 432→953 tok · $0.0008
no patch produced
nextflow-orchestration-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 432→2897 tok · $0.0022
no patch produced
project-manager-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 269→542 tok · $0.00044
no patch produced
project-manager-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 269→440 tok · $0.00037
no patch produced
project-manager-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 269→1415 tok · $0.0011
no patch produced
project-manager-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 269→727 tok · $0.0006
no patch produced
project-manager-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 269→705 tok · $0.0006
no patch produced
pyproject-edit-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 228→158 tok · $0.00015
no patch produced
pyproject-edit-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 0→177 tok · $0.00013
no patch produced
pyproject-edit-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 0→305 tok · $0.00022
no patch produced
version-hallucination-chore · minimax-m2.7 · rep 0 · patch_apply_failed · 0→344 tok · $0.00025
no patch produced
version-hallucination-chore · minimax-m2.7 · rep 1 · patch_apply_failed · 221→709 tok · $0.0006
no patch produced
version-hallucination-chore · minimax-m2.7 · rep 2 · patch_apply_failed · 221→279 tok · $0.00024
no patch produced
dependency-conflict-chore · minimax-m3 · rep 0 · patch_apply_failed · 422→515 tok · $0.0007
no patch produced
dependency-conflict-chore · minimax-m3 · rep 1 · patch_apply_failed · 422→458 tok · $0.0007
no patch produced
dependency-conflict-chore · minimax-m3 · rep 2 · patch_apply_failed · 422→216 tok · $0.00039
no patch produced
nextflow-orchestration-chore · minimax-m3 · rep 0 · accepted · 588→15766 tok · $0.0191
no patch produced
nextflow-orchestration-chore · minimax-m3 · rep 1 · accepted · 588→9261 tok · $0.0113
no patch produced
nextflow-orchestration-chore · minimax-m3 · rep 2 · accepted · 588→11981 tok · $0.0146
no patch produced
project-manager-chore · minimax-m3 · rep 0 · patch_apply_failed · 425→265 tok · $0.00045
no patch produced
project-manager-chore · minimax-m3 · rep 0 · patch_apply_failed · 425→1079 tok · $0.0014
no patch produced
project-manager-chore · minimax-m3 · rep 1 · accepted · 425→1581 tok · $0.0020
no patch produced
project-manager-chore · minimax-m3 · rep 1 · patch_apply_failed · 425→285 tok · $0.00047
no patch produced
project-manager-chore · minimax-m3 · rep 2 · patch_apply_failed · 425→429 tok · $0.0006
no patch produced
pyproject-edit-chore · minimax-m3 · rep 0 · patch_apply_failed · 384→131 tok · $0.00027
no patch produced
pyproject-edit-chore · minimax-m3 · rep 1 · patch_apply_failed · 384→145 tok · $0.00029
no patch produced
pyproject-edit-chore · minimax-m3 · rep 2 · patch_apply_failed · 384→243 tok · $0.00041
no patch produced
version-hallucination-chore · minimax-m3 · rep 0 · patch_apply_failed · 377→679 tok · $0.0009
no patch produced
version-hallucination-chore · minimax-m3 · rep 1 · patch_apply_failed · 377→287 tok · $0.00046
no patch produced
version-hallucination-chore · minimax-m3 · rep 2 · patch_apply_failed · 377→358 tok · $0.0005
no patch produced
dependency-conflict-chore · qwen3.6-27b · rep 0 · patch_apply_failed · 287→2353 tok · $0.0057
no patch produced
dependency-conflict-chore · qwen3.6-27b · rep 1 · accepted · 287→2186 tok · $0.0053
no patch produced
dependency-conflict-chore · qwen3.6-27b · rep 2 · patch_apply_failed · 287→2176 tok · $0.0053
no patch produced
nextflow-orchestration-chore · qwen3.6-27b · rep 0 · forbidden_pattern, hidden_tests_failed, smoke_failed · 450→4505 tok · $0.0109
no patch produced
nextflow-orchestration-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 450→4883 tok · $0.0118
no patch produced
nextflow-orchestration-chore · qwen3.6-27b · rep 2 · patch_apply_failed · 450→3950 tok · $0.0096
no patch produced
project-manager-chore · qwen3.6-27b · rep 0 · patch_apply_failed · 288→3484 tok · $0.0084
no patch produced
project-manager-chore · qwen3.6-27b · rep 0 · patch_apply_failed · 288→3670 tok · $0.0089
no patch produced
project-manager-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 288→2872 tok · $0.0070
no patch produced
project-manager-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 288→2068 tok · $0.0050
no patch produced
project-manager-chore · qwen3.6-27b · rep 2 · patch_apply_failed · 288→4101 tok · $0.0099
no patch produced
pyproject-edit-chore · qwen3.6-27b · rep 0 · patch_apply_failed · 245→434 tok · $0.0011
no patch produced
pyproject-edit-chore · qwen3.6-27b · rep 1 · patch_apply_failed · 245→2036 tok · $0.0050
no patch produced
pyproject-edit-chore · qwen3.6-27b · rep 2 · patch_apply_failed · 245→484 tok · $0.0012
no patch produced
version-hallucination-chore · qwen3.6-27b · rep 0 · patch_apply_failed · 237→2164 tok · $0.0053
no patch produced
version-hallucination-chore · qwen3.6-27b · rep 1 · accepted · 237→892 tok · $0.0022
no patch produced
version-hallucination-chore · qwen3.6-27b · rep 2 · accepted · 237→1528 tok · $0.0037
no patch produced