Skip to main content

​
Introduction

Function calling fine-tuning allows you to adapt models to reliably invoke tools and structured functions in response to user queries. This is useful for building agents and models that can reliably call APIs. This guide covers the specific steps for function calling fine-tuning. For general fine-tuning concepts, environment setup, and hyperparameter details, see the fine-tuning overview.

​
Function Calling Dataset

Dataset Requirements:
  • Format: .jsonl file
  • Supported types: Conversational, Preferential — more details on their purpose here
  • Each line may contain a tools field listing the tools the model can use
  • Assistant messages can include tool_calls (structured invocation requests) instead of content
  • Tool call results are provided via messages with the tool role

​
Conversation Tool Calling Format

This is what one row/example from the function calling dataset looks like in conversation format: 
{
  "messages": [
    {"role": "system", "content": "You are a helpful travel planning assistant."},
    {"role": "user", "content": "What is the current temperature in San Francisco?"},
    {"role": "assistant", "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "getCurrentWeather",
          "arguments": "{\"location\": \"San Francisco, CA\"}"
        }
      }
    ]},
    {"role": "tool", "content": "{\"location\": \"San Francisco\", \"temperature\": \"65\", \"unit\": \"fahrenheit\"}"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "getCurrentWeather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA."
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}

​
Preference Tool Calling Format

For preference fine-tuning, the tools field should be defined inside input: 
{
  "input": {
    "messages": [
      {"role": "system", "content": "You are a helpful travel planning assistant."},
      {"role": "user", "content": "What is the current temperature in San Francisco?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "getCurrentWeather",
          "description": "Get the current weather in a given location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA."
              }
            },
            "required": ["location"]
          }
        }
      }
    ]
  },
  "preferred_output": [
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "getCurrentWeather",
            "arguments": "{\"location\": \"San Francisco, CA\"}"
          }
        }
      ]
    }
  ],
  "non_preferred_output": [
    {
      "role": "assistant",
      "content": "Sorry, I can't help you with that."
    }
  ]
}

​
Supported Models

The following models support function calling fine-tuning:
OrganizationModel NameModel String for API
QwenQwen 2.5 1.5BQwen/Qwen2.5-1.5B
QwenQwen 2.5 1.5B InstructQwen/Qwen2.5-1.5B-Instruct
QwenQwen 2.5 3BQwen/Qwen2.5-3B
QwenQwen 2.5 3B InstructQwen/Qwen2.5-3B-Instruct
QwenQwen 2.5 7BQwen/Qwen2.5-7B
QwenQwen 2.5 7B InstructQwen/Qwen2.5-7B-Instruct
QwenQwen 2.5 14BQwen/Qwen2.5-14B
QwenQwen 2.5 14B InstructQwen/Qwen2.5-14B-Instruct
QwenQwen 2.5 32BQwen/Qwen2.5-32B
QwenQwen 2.5 32B InstructQwen/Qwen2.5-32B-Instruct
QwenQwen 2.5 72BQwen/Qwen2.5-72B
QwenQwen 2.5 72B InstructQwen/Qwen2.5-72B-Instruct
QwenQwen 3 0.6BQwen/Qwen3-0.6B
QwenQwen 3 1.7BQwen/Qwen3-1.7B
QwenQwen 3 4BQwen/Qwen3-4B
QwenQwen 3 8BQwen/Qwen3-8B
QwenQwen 3 14BQwen/Qwen3-14B
QwenQwen 3 32BQwen/Qwen3-32B
QwenQwen 3 32B 16kQwen/Qwen3-32B-16k
QwenQwen 3 30B A3BQwen/Qwen3-30B-A3B
QwenQwen 3 30B A3B Instruct 2507Qwen/Qwen3-30B-A3B-Instruct-2507
QwenQwen 3 235B A22BQwen/Qwen3-235B-A22B
QwenQwen 3 235B A22B Instruct 2507Qwen/Qwen3-235B-A22B-Instruct-2507
QwenQwen 3 VL 8B InstructQwen/Qwen3-VL-8B-Instruct
QwenQwen 3 VL 32B InstructQwen/Qwen3-VL-32B-Instruct
QwenQwen 3 VL 30B A3B InstructQwen/Qwen3-VL-30B-A3B-Instruct
QwenQwen 3 VL 235B A22B InstructQwen/Qwen3-VL-235B-A22B-Instruct
QwenQwen 3 Coder 30B A3B InstructQwen/Qwen3-Coder-30B-A3B-Instruct
QwenQwen 3 Coder 480B A35B InstructQwen/Qwen3-Coder-480B-A35B-Instruct
QwenQwen 3 Next 80B A3B InstructQwen/Qwen3-Next-80B-A3B-Instruct
QwenQwen 3 Next 80B A3B ThinkingQwen/Qwen3-Next-80B-A3B-Thinking
QwenQwen 3.5 0.8BQwen/Qwen3.5-0.8B
QwenQwen 3.5 2BQwen/Qwen3.5-2B
QwenQwen 3.5 4BQwen/Qwen3.5-4B
QwenQwen 3.5 9BQwen/Qwen3.5-9B
QwenQwen 3.5 27BQwen/Qwen3.5-27B
QwenQwen 3.5 35B A3BQwen/Qwen3.5-35B-A3B
QwenQwen 3.5 35B A3B BaseQwen/Qwen3.5-35B-A3B-Base
QwenQwen 3.5 122B A10BQwen/Qwen3.5-122B-A10B
QwenQwen 3.5 397B A17BQwen/Qwen3.5-397B-A17B
QwenQwen 3.6 35B A3BQwen/Qwen3.6-35B-A3B
Moonshot AIKimi K2 Instructmoonshotai/Kimi-K2-Instruct
Moonshot AIKimi K2 Thinkingmoonshotai/Kimi-K2-Thinking
Moonshot AIKimi K2 Basemoonshotai/Kimi-K2-Base
Moonshot AIKimi K2 Instruct 0905moonshotai/Kimi-K2-Instruct-0905
Moonshot AIKimi K2.5moonshotai/Kimi-K2.5
Z.aiGLM 4.6zai-org/GLM-4.6
Z.aiGLM 4.7zai-org/GLM-4.7
Z.aiGLM 5zai-org/GLM-5
Z.aiGLM 5.1zai-org/GLM-5.1
GoogleGemma 4 31B-itgoogle/gemma-4-31B-it
GoogleGemma 4 26B-A4B-itgoogle/gemma-4-26B-A4B-it
NVIDIANemotron Nano 9B v2nvidia/NVIDIA-Nemotron-Nano-9B-v2
NVIDIANemotron 3 Super 120B A12B BF16nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
MetaLlama 3.1 8B Instruct Referencemeta-llama/Meta-Llama-3.1-8B-Instruct-Reference
MetaLlama 3.1 8B 131k Instruct Referencemeta-llama/Meta-Llama-3.1-8B-131k-Instruct-Reference
MetaLlama 3.1 70B Instruct Referencemeta-llama/Meta-Llama-3.1-70B-Instruct-Reference
MetaLlama 3.1 70B 32k Instruct Referencemeta-llama/Meta-Llama-3.1-70B-32k-Instruct-Reference
MetaLlama 3.1 70B 131k Instruct Referencemeta-llama/Meta-Llama-3.1-70B-131k-Instruct-Reference
MetaLlama 3.1 405B Instruct Referencemeta-llama/Meta-Llama-3.1-405B-Instruct-Reference
MetaLlama 3.1 405B 8k Instruct Referencemeta-llama/Meta-Llama-3.1-405B-8k-Instruct-Reference
MetaLlama 3.1 405B 10k Instruct Referencemeta-llama/Meta-Llama-3.1-405B-10k-Instruct-Reference
MetaLlama 3.2 1B Instructmeta-llama/Llama-3.2-1B-Instruct
MetaLlama 3.2 3B Instructmeta-llama/Llama-3.2-3B-Instruct
MetaLlama 3.3 70B Instruct Referencemeta-llama/Llama-3.3-70B-Instruct-Reference
MetaLlama 3.3 70B 32k Instruct Referencemeta-llama/Llama-3.3-70B-32k-Instruct-Reference
MetaLlama 3.3 70B 131k Instruct Referencemeta-llama/Llama-3.3-70B-131k-Instruct-Reference
MetaLlama 4 Scout 17B 16E Instructmeta-llama/Llama-4-Scout-17B-16E-Instruct
MetaLlama 4 Scout 17B 16E Instruct VLMmeta-llama/Llama-4-Scout-17B-16E-Instruct-VLM
MetaLlama 4 Maverick 17B 128E Instructmeta-llama/Llama-4-Maverick-17B-128E-Instruct
MetaLlama 4 Maverick 17B 128E Instruct VLMmeta-llama/Llama-4-Maverick-17B-128E-Instruct-VLM
OpenAIGPT-OSS 20Bopenai/gpt-oss-20b
OpenAIGPT-OSS 120Bopenai/gpt-oss-120b

​
Check and Upload Dataset

To upload your data, use the CLI or our Python library: 
tg files check "function_calling_dataset.jsonl"

tg files upload "function_calling_dataset.jsonl"
You’ll see the following output once the upload finishes: 
{
  "id": "file-629e58b4-ff73-438c-b2cc-f69542b27980",
  "object": "file",
  "created_at": 1732573871,
  "type": null,
  "purpose": "fine-tune",
  "filename": "function_calling_dataset.jsonl",
  "bytes": 0,
  "line_count": 0,
  "processed": false,
  "FileType": "jsonl"
}
You’ll be using your file’s ID (the string that begins with file-) to start your fine-tuning job, so store it somewhere before moving on.

​
Starting a Fine-tuning Job

We support both LoRA and full fine-tuning for function calling models. For an exhaustive list of all the available fine-tuning parameters, refer to the Together AI Fine-tuning API Reference. 
tg fine-tuning create \
  --training-file "file-629e58b4-ff73-438c-b2cc-f69542b27980" \
  --model "Qwen/Qwen3-8B" \
  --lora

​
Full Fine-tuning

tg fine-tuning create \
  --training-file "file-629e58b4-ff73-438c-b2cc-f69542b27980" \
  --model "Qwen/Qwen3-8B" \
  --no-lora
You can specify many more fine-tuning parameters to customize your job. See the full list of hyperparameters and their definitions here.

​
Monitoring Your Fine-tuning Job

Fine-tuning can take time depending on the model size, dataset size, and hyperparameters. Your job will progress through several states: Pending, Queued, Running, Uploading, and Completed. Dashboard Monitoring You can monitor your job on the Together AI jobs dashboard. Check Status via API 
tg fine-tuning retrieve "your-job-id"

tg fine-tuning list-events "your-job-id"

​
Using Your Fine-tuned Model

Once your fine-tuning job completes, your model will be available for use. You can view your fine-tuned models in your models dashboard.

​
Dedicated Endpoint Deployment

You can deploy your fine-tuned model on a dedicated endpoint for production use:
  1. Visit your models dashboard
  2. Find your fine-tuned model and click ”+ CREATE DEDICATED ENDPOINT”
  3. Select your hardware configuration and scaling options
  4. Click “DEPLOY”
You can also deploy programmatically: 
import os
from together import Together

client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

response = client.endpoints.create(
    display_name="Fine-tuned Qwen3-8B Function Calling",
    model="your-username/Qwen3-8B-your-suffix",
    hardware="4x_nvidia_h100_80gb_sxm",
    autoscaling={"min_replicas": 1, "max_replicas": 1},
)

print(response)
Running this code will deploy a dedicated endpoint, which incurs charges. For detailed documentation around how to deploy, delete and modify endpoints see the Endpoints API Reference. For more details, read the detailed walkthrough How-to: Fine-tuning.