Can open-source models do tool calling?
How open-source tool calling works
Open-source models can perform tool calling if they are trained or fine-tuned to output structured requests, such as JSON with a function name and arguments. This is similar to how proprietary models like GPT-4 handle function calling.
However, not all open-source models have this ability out of the box. Some require prompt engineering or additional fine-tuning to reliably generate tool calls. The quality and consistency can vary widely depending on the model size and training data.
Popular open-source models with tool calling
Several open-source models now include tool calling support. For example, Meta's Llama 3.1 and 3.2 versions have built-in tool use, and Mistral AI offers models with function calling capabilities. Qwen and DeepSeek also provide models that can call tools.
These models often come in different sizes, from small (e.g., 7B parameters) to large (e.g., 70B+), and the larger ones tend to be more reliable. You can run them locally or via cloud providers that host open-source models.
- Llama 3.1 and 3.2 (Meta) – support tool calling in instruct versions.
- Mistral models (e.g., Mistral Large, Mixtral) – offer function calling.
- Qwen (Alibaba) – some versions have tool use.
- DeepSeek – models with function calling.
- Command R (Cohere) – open weights? Actually Cohere's Command R is open-weight and supports tools.
Considerations for using open-source tool calling
When using open-source models for tool calling, you may need to handle the parsing and execution of tool calls yourself, as there's no standardized API like OpenAI's. You'll also need to ensure the model's output format matches your expectations.
Performance can be less consistent than proprietary models, especially for complex or multi-step tool use. Testing and possibly fine-tuning on your specific tools is recommended for production applications.
Common mistakes
- Assuming all open-source models can do tool calling without checking their documentation or capabilities.
- Expecting open-source tool calling to be as reliable as proprietary models like GPT-4 without additional fine-tuning.
- Forgetting that you often need to implement the tool execution logic yourself when using open-source models.