← ChoreCode leaderboard · 20260706T163525Z
version-hallucination-chore
Fix the hallucinated requests version pin
pyproject.toml pins requests==99.0.0, which does not exist on PyPI and cannot be resolved. Fix the version specifier to a real, resolvable range so the package installs correctly. Do not remove the requests dependency. Return a unified diff only.
Forbidden cheap shots: replacing the hallucinated version with another non-existent version (e.g. 2.99.0); removing the requests dependency instead of fixing its version; pinning to requests==1.0.0 which is incompatible with Python 3.12
Results by model
8 run configs| Run config | Accepted | Accept rate | Hard fails |
|---|---|---|---|
| gemma4-26b | 6/6 | 100% | 0 |
| deepseek-v4-flash | 3/3 | 100% | 0 |
| deepseek-v4-pro | 3/3 | 100% | 0 |
| glm-5.2 | 3/3 | 100% | 0 |
| kimi-k2.6 | 2/3 | 67% | 0 |
| qwen3.6-27b | 2/3 | 67% | 0 |
| minimax-m2.7 | 0/3 | 0% | 0 |
| minimax-m3 | 0/3 | 0% | 0 |
Attempts
expand a row for the patch and logsdeepseek-v4-flash · rep 0 · accepted · 224→584 tok · $0.00013
no patch produced
deepseek-v4-flash · rep 1 · accepted · 224→310 tok · $0.000076
no patch produced
deepseek-v4-flash · rep 2 · accepted · 224→393 tok · $0.000091
no patch produced
deepseek-v4-pro · rep 0 · accepted · 224→322 tok · $0.00038
no patch produced
deepseek-v4-pro · rep 1 · accepted · 224→433 tok · $0.00047
no patch produced
deepseek-v4-pro · rep 2 · accepted · 224→473 tok · $0.0005
no patch produced
gemma4-26b · rep 0 · accepted · 244→392 tok · $0.00014
no patch produced
gemma4-26b · rep 0 · accepted · 244→2408 tok · $0.0008
no patch produced
gemma4-26b · rep 1 · accepted · 244→2294 tok · $0.0008
no patch produced
gemma4-26b · rep 1 · accepted · 244→3474 tok · $0.0012
no patch produced
gemma4-26b · rep 2 · accepted · 244→1468 tok · $0.00050
no patch produced
gemma4-26b · rep 2 · accepted · 244→2169 tok · $0.0007
no patch produced
glm-5.2 · rep 0 · accepted · 222→190 tok · $0.00046
no patch produced
glm-5.2 · rep 1 · accepted · 222→157 tok · $0.00040
no patch produced
glm-5.2 · rep 2 · accepted · 222→132 tok · $0.00036
no patch produced
kimi-k2.6 · rep 0 · accepted · 217→4142 tok · $0.0143
no patch produced
kimi-k2.6 · rep 1 · patch_apply_failed · 217→1867 tok · $0.0065
no patch produced
kimi-k2.6 · rep 2 · accepted · 217→3180 tok · $0.0110
no patch produced
minimax-m2.7 · rep 0 · patch_apply_failed · 0→344 tok · $0.00025
no patch produced
minimax-m2.7 · rep 1 · patch_apply_failed · 221→709 tok · $0.0006
no patch produced
minimax-m2.7 · rep 2 · patch_apply_failed · 221→279 tok · $0.00024
no patch produced
minimax-m3 · rep 0 · patch_apply_failed · 377→679 tok · $0.0009
no patch produced
minimax-m3 · rep 1 · patch_apply_failed · 377→287 tok · $0.00046
no patch produced
minimax-m3 · rep 2 · patch_apply_failed · 377→358 tok · $0.0005
no patch produced
qwen3.6-27b · rep 0 · patch_apply_failed · 237→2164 tok · $0.0053
no patch produced
qwen3.6-27b · rep 1 · accepted · 237→892 tok · $0.0022
no patch produced
qwen3.6-27b · rep 2 · accepted · 237→1528 tok · $0.0037
no patch produced