Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.together.ai/llms.txt

Use this file to discover all available pages before exploring further.

This page lists all supported model sources for the Evaluations API. You can use serverless models, dedicated endpoints, or external models from providers like OpenAI, Anthropic, and Google.

​
Serverless models

Set model_source = "serverless" to use Together’s serverless inference.
Any Together serverless model that supports structured outputs can be used.
Example configuration:
Python
from together import Together

client = Together()

model_config = {
    "model": "deepseek-ai/DeepSeek-V3.1",
    "model_source": "serverless",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
Dedicated models

Set model_source = "dedicated" to use your own dedicated endpoint.
A user-launched dedicated endpoint must be created before running evaluations. After launching an endpoint, copy-paste the endpoint ID into the model field.
Example configuration:
Python
from together import Together

client = Together()

model_config = {
    "model": "your-endpoint-id",
    "model_source": "dedicated",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
External models

Set model_source = "external" to use models from external providers.
External models require an API token from the respective provider. Set the external_api_token parameter with your provider’s API key.

​
Supported shortcuts

Use these shortcuts in the model field - the API base URL will be determined automatically:
ProviderModel NameModel String for API
OpenAIGPT-5openai/gpt-5
OpenAIGPT-5 Miniopenai/gpt-5-mini
OpenAIGPT-5 Nanoopenai/gpt-5-nano
OpenAIGPT-5.2openai/gpt-5.2
OpenAIGPT-5.2 Proopenai/gpt-5.2-pro
OpenAIGPT-5.2 Chat Latestopenai/gpt-5.2-chat-latest
OpenAIGPT-4openai/gpt-4
OpenAIGPT-4.1openai/gpt-4.1
OpenAIGPT-4o Miniopenai/gpt-4o-mini
OpenAIGPT-4oopenai/gpt-4o
AnthropicClaude Sonnet 4.5anthropic/claude-sonnet-4-5
AnthropicClaude Haiku 4.5anthropic/claude-haiku-4-5
AnthropicClaude Sonnet 4.0anthropic/claude-sonnet-4-0
AnthropicClaude Opus 4.5anthropic/claude-opus-4-5
AnthropicClaude Opus 4.1anthropic/claude-opus-4-1
AnthropicClaude Opus 4.0anthropic/claude-opus-4-0
GoogleGemini 2.5 Progoogle/gemini-2.5-pro
GoogleGemini 2.5 Flashgoogle/gemini-2.5-flash
GoogleGemini 2.5 Flash Litegoogle/gemini-2.5-flash-lite
GoogleGemini 3 Pro Previewgoogle/gemini-3-pro-preview
Example configuration with shortcut:
Python
from together import Together

client = Together()

model_config = {
    "model": "openai/gpt-5",
    "model_source": "external",
    "external_api_token": "your-openai-api-key",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}

​
Custom base URL

You can also use any OpenAI chat/completions-compatible API by specifying a custom external_base_url:
Python
from together import Together

client = Together()

model_config = {
    "model": "mistral-small-latest",
    "model_source": "external",
    "external_api_token": "your-mistral-api-key",
    "external_base_url": "https://api.mistral.ai/",
    "system_template": "You are a helpful assistant.",
    "input_template": "{{prompt}}",
    "max_tokens": 512,
    "temperature": 0.7,
}
The external API must be OpenAI chat/completions-compatible.