Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.together.ai/llms.txt

Use this file to discover all available pages before exploring further.

This page lists all supported model sources for the Evaluations API. You can use serverless models, dedicated endpoints, or external models from providers like OpenAI, Anthropic, and Google.

​
Serverless models

Set model_source = "serverless" to use Together’s serverless inference.
Any Together serverless model that supports structured outputs can be used.

​
Supported models

ModelModel ID
DeepSeek-R1deepseek-ai/DeepSeek-R1
DeepSeek-V3.1deepseek-ai/DeepSeek-V3.1
DeepSeek-V4 Prodeepseek-ai/DeepSeek-V4-Pro
Llama 3.3 70B Instruct Turbometa-llama/Llama-3.3-70B-Instruct-Turbo
LFM2 24B A2BLiquidAI/LFM2-24B-A2B
MiniMax M2.7MiniMaxAI/MiniMax-M2.7
Kimi K2.5moonshotai/Kimi-K2.5
Kimi K2.6moonshotai/Kimi-K2.6
Qwen3 235B A22B Instruct 2507Qwen/Qwen3-235B-A22B-Instruct-2507-tput
Qwen3 Coder 480B A35B Instruct FP8Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8
Qwen3 Coder Next FP8Qwen/Qwen3-Coder-Next-FP8
Qwen 3.5 397B A17BQwen/Qwen3.5-397B-A17B
Qwen 3.5 9BQwen/Qwen3.5-9B
Qwen 3.6 PlusQwen/Qwen3.6-Plus
Gemma 3n E4Bgoogle/gemma-3n-E4B-it
Gemma 4 31Bgoogle/gemma-4-31B-it
Cogito V2 1 671Bdeepcogito/cogito-v2-1-671b
Essential AI RNJ-1essentialai/rnj-1-instruct
GLM-5zai-org/GLM-5
GLM-5.1zai-org/GLM-5.1
GPT OSS 20Bopenai/gpt-oss-20b
GPT OSS 120Bopenai/gpt-oss-120b
Example configuration:
Python
from together import Together

client = Together()

model_config = {
    "model": "deepseek-ai/DeepSeek-V3.1",
    "model_source": "serverless",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
Dedicated models

Set model_source = "dedicated" to use your own dedicated endpoint.
A user-launched dedicated endpoint must be created before running evaluations. After launching an endpoint, copy-paste the endpoint ID into the model field.
Example configuration:
Python
from together import Together

client = Together()

model_config = {
    "model": "your-endpoint-id",
    "model_source": "dedicated",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
External models

Set model_source = "external" to use models from external providers.
External models require an API token from the respective provider. Set the external_api_token parameter with your provider’s API key.

​
Supported shortcuts

Use these shortcuts in the model field - the API base URL will be determined automatically:
ProviderModel NameModel String for API
OpenAIGPT-5.5openai/gpt-5.5
OpenAIGPT-5.4openai/gpt-5.4
OpenAIGPT-5.4 Miniopenai/gpt-5.4-mini
OpenAIGPT-5.4 Nanoopenai/gpt-5.4-nano
OpenAIGPT-5.3 Chat Latestopenai/gpt-5.3-chat-latest
OpenAIGPT-4.1openai/gpt-4.1
OpenAIGPT-4.1 Miniopenai/gpt-4.1-mini
OpenAIGPT-4.1 Nanoopenai/gpt-4.1-nano
OpenAIGPT-4oopenai/gpt-4o
OpenAIGPT-4o Miniopenai/gpt-4o-mini
OpenAIo4-miniopenai/o4-mini
OpenAIo3openai/o3
AnthropicClaude Opus 4.7anthropic/claude-opus-4-7
AnthropicClaude Opus 4.6anthropic/claude-opus-4-6
AnthropicClaude Opus 4.5anthropic/claude-opus-4-5
AnthropicClaude Sonnet 4.6anthropic/claude-sonnet-4-6
AnthropicClaude Sonnet 4.5anthropic/claude-sonnet-4-5
AnthropicClaude Haiku 4.5anthropic/claude-haiku-4-5
GoogleGemini 3.1 Pro Previewgoogle/gemini-3.1-pro-preview
GoogleGemini 3.1 Flash Litegoogle/gemini-3.1-flash-lite
GoogleGemini 3 Pro Previewgoogle/gemini-3-pro-preview
GoogleGemini 3 Flash Previewgoogle/gemini-3-flash-preview
GoogleGemini 2.5 Progoogle/gemini-2.5-pro
GoogleGemini 2.5 Flashgoogle/gemini-2.5-flash
GoogleGemini 2.5 Flash Litegoogle/gemini-2.5-flash-lite
Example configuration with shortcut:
Python
from together import Together

client = Together()

model_config = {
    "model": "openai/gpt-5",
    "model_source": "external",
    "external_api_token": "your-openai-api-key",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
Custom base URL

You can also use any OpenAI chat/completions-compatible API by specifying a custom external_base_url:
Python
from together import Together

client = Together()

model_config = {
    "model": "mistral-small-latest",
    "model_source": "external",
    "external_api_token": "your-mistral-api-key",
    "external_base_url": "https://api.mistral.ai/",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}
The external API must be OpenAI chat/completions-compatible.