Do smaller open-weight models need a different tool format?
Smaller open-weight models
Many open-weight models have been trained with tool-use examples, and some handle simple single-tool calls reliably. Multi-step flows and tools with many parameters are harder. Results can also shift with the serving software, quantization and prompt format. Latency and cost can also be strong reasons to test a smaller option.
Check the model card and any published function-calling notes. Those sources usually describe the expected tool format and any known limits, which helps you set up the request correctly.
Checking support before you commit
Write a small test set with your real tools and typical requests. Measure how often the model picks the right tool and produces valid arguments. Compare that result with a larger model to see whether the savings are worth any drop in reliability. Document the results so the choice is easy to revisit later.
Add validation and retries in your code. Even a good model will sometimes return malformed arguments, so your handler should reject them clearly.
- Test with your real tools, not generic examples.
- Check the required tool format on the model card.
- Track both tool choice and argument accuracy.
- Add validation and a clear retry rule.
Common mistakes
- Assuming a model that handles chat well will handle tool calls equally well.
- Skipping argument validation because the model usually gets it right.
