POST /v1/messages. If your app is built on the Anthropic SDKs, change the API key, base URL, and model to run it on open-source models hosted by Together.
Set up the client
Install the Anthropic SDK for your language:api_key to your Together API key and base_url to https://api.together.ai, then send a message:
x-api-key header or an Authorization: Bearer header. If you send both, the Authorization header takes precedence.
Supported models
Supported models differ between serverless inference and dedicated model inference:
To send requests to dedicated model inference, set the base URL to
https://api-inference.together.ai and pass your endpoint string as model.
Other serverless models can accept Messages API requests, but they aren’t supported and may return malformed output. To use the Messages API with an existing dedicated endpoint or a model that isn’t supported yet, contact sales.
Endpoint support
Stream responses
To receive server-sent events as the model generates them, setstream to true. The SDKs’ messages.stream helper sets it for you and collects the text:
message_start, content_block_start, content_block_delta, content_block_stop, message_delta, and message_stop. Deltas arrive as text_delta, thinking_delta, or input_json_delta. The stream ends after message_stop, with no [DONE] line.
In message_start, input_tokens counts the whole prompt, including cached tokens, and output_tokens is 0. Read the final token counts from the message_delta event, or from the final message that the SDK helpers return.
Use extended thinking
Setthinking.type to enabled to get the model’s reasoning back as thinking blocks ahead of the answer:
thinking.type:enabledandadaptivereturnthinkingblocks. Withdisabled, or with nothinkingfield, the response contains only the answer.budget_tokens: It’s required withenabled, but Together doesn’t enforce it as a limit. The only limit ismax_tokens, which covers reasoning and the answer combined.- Hidden reasoning: Reasoning models can reason even when they return no
thinkingblock, and GLM-5.3 reasons even whenthinking.typeisdisabled. Those tokens count towardmax_tokensandoutput_tokens. If a response has empty text and astop_reasonofmax_tokens, raisemax_tokens. output_config.effort: Together passes the level to the model’s chat template, so its effect depends on the model.- Signatures:
thinkingblocks have an emptysignature. To keep reasoning in a multi-turn tool loop, pass the blocks back unchanged in the assistant turn.
Use tools
Define tools withname, description, and input_schema. When the model calls a tool, the response has stop_reason set to tool_use and one tool_use block per call. Run each tool, then send the results back in a single user turn with one tool_result block per call:
tool_choice accepts auto, any, tool (with a name), and none. To limit the model to one tool call per turn, set tool_choice.disable_parallel_tool_use to true.
Together doesn’t run tools itself, so every tool runs in your app. Define custom tools, or use Anthropic’s bash, text_editor, and memory client tools.
A request with a server tool (web_search, web_fetch, or code_execution) or the computer tool returns a 400. The one exception is a web_search tool that you send alongside other tools and that tool_choice doesn’t force. Together removes that tool from the request. Together also ignores mcp_servers.
Token usage and caching
usage reports input_tokens and output_tokens. Thinking tokens, including hidden reasoning, count as output tokens.
Together caches repeated prompt prefixes automatically, so cache_control has no effect. cache_read_input_tokens counts the prompt tokens served from the cache, and input_tokens counts the rest. cache_creation_input_tokens is always 0.
Together bills Messages API requests the same way as chat completions: per token at each model’s serverless prices, or per GPU-minute on dedicated model inference.
Other differences from Anthropic’s API
model: Use a Together model string. Claude model names, such asclaude-sonnet-5-5, return a404.anthropic-betaheader: Together accepts any value, but a beta header doesn’t turn on Anthropic beta features. The one exception isinterleaved-thinking-2025-05-14, which letsbudget_tokensexceedmax_tokens.- Response
id: It uses Together’s format rather than amsg_prefix. Don’t parse it. - Structured outputs:
output_config.formatreturns a400. Ask for JSON in the prompt, or use structured outputs with chat completions. metadataandservice_tier: Neither changes the response.service_tieraccepts onlyautoandstandard_only.- Images and documents: GLM-5.3 and GLM-5.2 don’t accept images, so a request with an image block returns a
400. PDFdocumentblocks also return a400, so send documents as text.
Errors
Errors use the same JSON format as Anthropic’s API:error.type, and the Anthropic SDKs raise their usual exception class for each status:
Together usually returns
503 for overloaded_error, where Anthropic’s API returns 529. The Anthropic Python SDK raises InternalServerError for a 503 rather than OverloadedError, so catch both if you handle overloads separately.
A failure after a stream starts arrives as an error event in the stream.
A few gateway-level failures, such as a 502 when Together can’t connect to the model’s servers, return Together’s OpenAI-style error body instead. To handle both formats, branch on the HTTP status rather than the error body. For causes and fixes by status code, see Error codes.
Next steps
Responses API
Call Together models with the OpenAI Responses API.
Together Link
Run Claude Code and other coding agents on Together models.