Simple coding tasks done well
ChoreCode
Benchmarking models on real backlog work: lint, types, deps, docs, uv, orchestration, and the quiet art of not annoying the senior engineer.
snapshot 20260709T195510Z · benchmark v0.1.0 · generated 2026-07-09T19:55:10+00:00
Filters and search
Current Picks
Lenient · comparableHalo comparators
Leaderboard
ranked by pass rate — click headers to sort| Run config | $ / accepted | $ / attempt | Accepted | Accept rate | Hard fails | Tokens / accepted | Mean latency |
|---|---|---|---|---|---|---|---|
| minimax-m3 [none] | $0.0014 | $0.0013 | 48/51 | 94% | 0% | 1486 | 9.6 s * |
| kimi-k2.6 [max] | $0.0195 | $0.0180 | 47/51 | 92% | 2% | 5916 | 51.8 s * |
| deepseek-v4-pro [none] | $0.00026 | $0.00024 | 46/51 | 90% | 4% | 433 | 2.5 s * |
| gemma4-26b [high] | $0.0013 | $0.0011 | 46/51 | 90% | 0% | 4093 | 93.1 s * |
| deepseek-v4-pro [max] | $0.0013 | $0.0012 | 46/51 | 90% | 2% | 1704 | 10.9 s * |
| gemma4-26b [medium] | $0.0014 | $0.0012 | 46/51 | 90% | 2% | 4375 | 103.6 s * |
| gemma4-26b [max] | $0.0015 | $0.0014 | 46/51 | 90% | 2% | 4861 | 133.2 s * |
| minimax-m3 [high] | $0.0017 | $0.0015 | 46/51 | 90% | 0% | 1741 | 11.4 s * |
| kimi-k2.6 [medium] | $0.0185 | $0.0166 | 46/51 | 90% | 2% | 5620 | 46.4 s * |
| gemma4-26b [none] | $0.000071 | $0.000063 | 45/51 | 88% | 0% | 459 | 21.6 s * |
| deepseek-v4-flash [high] | $0.00011 | $0.00010 | 45/51 | 88% | 2% | 759 | 4.1 s * |
| deepseek-v4-flash [medium] | $0.00012 | $0.00011 | 45/51 | 88% | 4% | 818 | 9.0 s * |
| glm-5.2 [max] | $0.0059 | $0.0052 | 45/51 | 88% | 0% | 3517 | 44.8 s * |
| kimi-k2.6 [high] | $0.0155 | $0.0137 | 45/51 | 88% | 2% | 4752 | 40.5 s * |
| kimi-k2.6 [low] | $0.0198 | $0.0175 | 45/51 | 88% | 6% | 6017 | 49.7 s * |
| deepseek-v4-pro [medium] | $0.0008 | $0.0007 | 44/51 | 86% | 8% | 1062 | 6.7 s * |
| glm-5.2 [medium] | $0.0012 | $0.0011 | 44/51 | 86% | 0% | 896 | 12.3 s * |
| minimax-m3 [medium] | $0.0018 | $0.0016 | 44/51 | 86% | 4% | 1855 | 11.3 s * |
| deepseek-v4-flash [low] | $0.00011 | $0.000092 | 43/51 | 84% | 6% | 738 | 5.0 s * |
| deepseek-v4-pro [low] | $0.0008 | $0.0007 | 43/51 | 84% | 4% | 1026 | 6.6 s * |
| glm-5.2 [high] | $0.0009 | $0.0008 | 43/51 | 84% | 2% | 716 | 9.4 s * |
| deepseek-v4-flash [max] | $0.0006 | $0.00047 | 42/51 | 82% | 4% | 3380 | 12.7 s * |
| deepseek-v4-pro [high] | $0.0009 | $0.0007 | 42/51 | 82% | 4% | 1130 | 7.7 s * |
| gemma4-26b [low] | $0.0015 | $0.0012 | 42/51 | 82% | 2% | 4756 | 95.3 s * |
| minimax-m3 [low] | $0.0027 | $0.0022 | 42/51 | 82% | 0% | 2581 | 16.2 s * |
| qwen3.6-27b [high] | $0.0079 | $0.0065 | 42/51 | 82% | 0% | 3548 | 232.7 s * |
| qwen3.6-27b [medium] | $0.0079 | $0.0065 | 42/51 | 82% | 0% | 3558 | 212.4 s * |
| kimi-k2.6 [none] | $0.0010 | $0.0008 | 40/51 | 78% | 8% | 535 | 3.0 s * |
| glm-5.2 [low] | $0.0011 | $0.0009 | 40/51 | 78% | 0% | 836 | 9.4 s * |
| minimax-m3 [max] | $0.0018 | $0.0014 | 40/51 | 78% | 6% | 1909 | 11.3 s * |
| qwen3.6-27b [max] | $0.0082 | $0.0064 | 40/51 | 78% | 0% | 3699 | 202.5 s * |
| glm-5.2 [none] | $0.0005 | $0.00041 | 39/51 | 76% | 6% | 513 | 5.4 s * |
| minimax-m2.7 [medium] | $0.0010 | $0.0008 | 38/51 | 75% | 8% | 1277 | 13.9 s * |
| qwen3.6-27b [low] | $0.0092 | $0.0069 | 38/51 | 75% | 0% | 4136 | 203.9 s * |
| deepseek-v4-flash [none] | $0.000066 | $0.000048 | 37/51 | 73% | 6% | 529 | 4.4 s * |
| minimax-m2.7 [low] | $0.0018 | $0.0012 | 35/51 | 69% | 8% | 2108 | 20.8 s * |
| minimax-m2.7 [max] | $0.0018 | $0.0012 | 35/51 | 69% | 6% | 2122 | 20.3 s * |
| minimax-m2.7 [high] | $0.0008 | $0.0005 | 34/51 | 67% | 8% | 1040 | 10.6 s * |
| minimax-m2.7 [none] | $0.0016 | $0.0010 | 34/51 | 67% | 10% | 1891 | 18.5 s * |
| qwen3.6-27b [none] | $0.0008 | $0.00043 | 28/51 | 55% | 6% | 721 | 55.2 s * |
Cost is recorded input/output tokens multiplied by current OpenRouter pricing for the same model or weights. * latency for locally run models reflects local hardware and load.
Acceptance vs Cost
x-axis: Effort markers
none low medium high max
Task Matrix
accepted / attempts — compact summary by topic all accepted some accepted none accepted hard fail seen
Config
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Dependency Discipline
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Docstrings
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Generated Artifacts
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Python Functions
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Python Functions
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Python Functions
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Python Functions
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Toolchain Conventions
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Type Integrity
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Type Integrity
minimax-m3 [none]
kimi-k2.6 [max]
deepseek-v4-pro [none]
gemma4-26b [high]
deepseek-v4-pro [max]
gemma4-26b [medium]
gemma4-26b [max]
minimax-m3 [high]
kimi-k2.6 [medium]
gemma4-26b [none]
deepseek-v4-flash [high]
deepseek-v4-flash [medium]
glm-5.2 [max]
kimi-k2.6 [high]
kimi-k2.6 [low]
deepseek-v4-pro [medium]
glm-5.2 [medium]
minimax-m3 [medium]
deepseek-v4-flash [low]
deepseek-v4-pro [low]
glm-5.2 [high]
deepseek-v4-flash [max]
deepseek-v4-pro [high]
gemma4-26b [low]
minimax-m3 [low]
qwen3.6-27b [high]
qwen3.6-27b [medium]
kimi-k2.6 [none]
glm-5.2 [low]
minimax-m3 [max]
qwen3.6-27b [max]
glm-5.2 [none]
minimax-m2.7 [medium]
qwen3.6-27b [low]
deepseek-v4-flash [none]
minimax-m2.7 [low]
minimax-m2.7 [max]
minimax-m2.7 [high]
minimax-m2.7 [none]
qwen3.6-27b [none]
Tasks
17 public chores grouped by topicConfig
1 taskClear the lint violation without weakening lint config
Clear a lint failure in code without weakening project or lint configuration.
- Accepted
- 100/120
- Hard fails
- 8
modifying or deleting visible testsloosening lint configuration in pyproject.tomladding noqa suppressions instead of fixing the code
Dependency Discipline
2 tasksUse the existing retry dependency
Add retry behavior using the project existing retry dependency instead of adding another one.
- Accepted
- 76/120
- Hard fails
- 4
adding a second retry dependency instead of using the dependency already declaredretrying every exception typemodifying or deleting visible tests
Decode query strings with the standard library
Decode query-string values with the standard library rather than adding a URL parsing package.
- Accepted
- 84/120
- Hard fails
- 3
adding yarl, furl, httpx, or another dependency for basic query parsinghard-coding the hidden query stringsmodifying or deleting visible tests
Docstrings
1 task docstring-chore 90%
Document slugify with a Google-style docstring
Add a Google-style docstring that documents the function accurately without changing behavior.
- Accepted
- 108/120
- Hard fails
- 0
modifying or deleting visible testsassigning __doc__ dynamically instead of writing a docstringchanging slugify's behavior
Generated Artifacts
2 tasksRegenerate CLI help after adding lint
Add a command to the source spec and regenerate the checked-in CLI help output.
- Accepted
- 105/120
- Hard fails
- 6
editing src/tiny_cli/help_text.py without changing commands.jsonleaving generated output stale after changing commands.jsonmodifying or deleting visible tests
Rename the generated API client field
Rename an API schema field and regenerate the checked-in client from the schema.
- Accepted
- 101/120
- Hard fails
- 0
editing src/tiny_api/client.py without changing openapi.jsonleaving generated output stale after changing the schemamodifying or deleting visible tests
Python Functions
1 taskStop swallowing lookup errors
Fix lookup error handling so missing items raise ValueError while unrelated errors still surface.
- Accepted
- 111/120
- Hard fails
- 2
modifying or deleting visible testskeeping a broad except while special-casing the hidden test inputs
Python Functions
1 taskMake range counting boundaries inclusive
Correct an inclusive range helper so boundary values count without hard-coded cases.
- Accepted
- 117/120
- Hard fails
- 0
modifying or deleting visible testshard-coding the visible test inputs or expected values
Python Functions
1 taskRemove leftover debug output
Remove leftover debug output while preserving the parser behavior and public API.
- Accepted
- 103/120
- Hard fails
- 13
modifying or deleting visible testssilencing prints by redirecting or writing to /dev/null instead of removing them
Python Functions
1 taskMake the invalid-flag error actionable
Make an invalid-flag error message identify the bad value and list valid states.
- Accepted
- 117/120
- Hard fails
- 0
modifying or deleting visible testshard-coding messages for specific invalid inputs instead of formatting the state
Toolchain Conventions
5 tasksResolve the pydantic version conflict
Resolve an incompatible dependency pin without removing the package that needs it.
- Accepted
- 110/120
- Hard fails
- 0
removing pydantic-settings from dependencies to sidestep the conflictpinning both to versions that still conflictrewriting settings.py to avoid using pydantic-settings
Replace ad hoc subprocess orchestration with Nextflow
Replace ad hoc subprocess orchestration with the project declared Nextflow workflow.
- Accepted
- 30/120
- Hard fails
- 24
moving subprocess import inside a function to hide it from the module-level import checkkeeping the parallel subprocess loop anywhere in runner.pynot creating a real Nextflow workflow in main.nf
Use the declared project manager for checks
Run project checks through the declared project manager instead of creating ad hoc virtualenvs.
- Accepted
- 111/120
- Hard fails
- 4
creating or referencing a venv, .venv, or virtualenvhard-coding only the uv path while leaving poetry wrongmodifying or deleting visible tests
Declare the missing httpx dependency in pyproject.toml
Declare a missing runtime dependency in pyproject.toml without adding installer scripts.
- Accepted
- 96/120
- Hard fails
- 0
creating a setup.py that calls pip installcreating an install.sh or Makefile that calls pipediting fetcher.py to remove the httpx import instead of declaring the dependency
Fix the hallucinated requests version pin
Replace a hallucinated dependency version with a real resolvable range.
- Accepted
- 107/120
- Hard fails
- 0
replacing the hallucinated version with another non-existent version (e.g. 2.99.0)removing the requests dependency instead of fixing its versionpinning to requests==1.0.0 which is incompatible with Python 3.12
Type Integrity
1 taskPreserve fractional mean results
Preserve fractional mean results without type-ignore or broad type escapes.
- Accepted
- 90/120
- Hard fails
- 2
modifying or deleting visible testshard-coding the visible test valuesadding type-ignore or Any-based escapes
Type Integrity
1 taskAnnotate the distance converter precisely
Add precise annotations to a converter without changing its runtime behavior.
- Accepted
- 112/120
- Hard fails
- 1
modifying or deleting visible testsannotating with Any instead of precise typesadding type-ignore comments