Skip to main content
Together AI implements the Anthropic Messages API at POST /v1/messages. If your app is built on the Anthropic SDKs, change the API key, base URL, and model to run it on open-source models hosted by Together.

Set up the client

Install the Anthropic SDK for your language:
Set api_key to your Together API key and base_url to https://api.together.ai, then send a message:
The Messages API accepts your API key in either an x-api-key header or an Authorization: Bearer header. If you send both, the Authorization header takes precedence.

Supported models

Supported models differ between serverless inference and dedicated model inference: To send requests to dedicated model inference, set the base URL to https://api-inference.together.ai and pass your endpoint string as model. Other serverless models can accept Messages API requests, but they aren’t supported and may return malformed output. To use the Messages API with an existing dedicated endpoint or a model that isn’t supported yet, contact sales.

Endpoint support

Stream responses

To receive server-sent events as the model generates them, set stream to true. The SDKs’ messages.stream helper sets it for you and collects the text:
The stream uses Anthropic’s event types: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, and message_stop. Deltas arrive as text_delta, thinking_delta, or input_json_delta. The stream ends after message_stop, with no [DONE] line. In message_start, input_tokens counts the whole prompt, including cached tokens, and output_tokens is 0. Read the final token counts from the message_delta event, or from the final message that the SDK helpers return.
GLM-5.3 reasons before it answers, even when thinking is off, and that reasoning counts toward max_tokens. A short answer can take about 1,000 tokens of reasoning. If the reasoning uses up max_tokens, the stream still ends normally: the text block is empty, message_delta reports a stop_reason of max_tokens, and the SDK helpers raise no error. Set max_tokens well above the length of the answer you expect, and check stop_reason before you use the text.

Use extended thinking

Set thinking.type to enabled to get the model’s reasoning back as thinking blocks ahead of the answer:
Thinking on Together differs from Anthropic’s API:
  • thinking.type: enabled and adaptive return thinking blocks. With disabled, or with no thinking field, the response contains only the answer.
  • budget_tokens: It’s required with enabled, but Together doesn’t enforce it as a limit. The only limit is max_tokens, which covers reasoning and the answer combined.
  • Hidden reasoning: Reasoning models can reason even when they return no thinking block, and GLM-5.3 reasons even when thinking.type is disabled. Those tokens count toward max_tokens and output_tokens. If a response has empty text and a stop_reason of max_tokens, raise max_tokens.
  • output_config.effort: Together passes the level to the model’s chat template, so its effect depends on the model.
  • Signatures: thinking blocks have an empty signature. To keep reasoning in a multi-turn tool loop, pass the blocks back unchanged in the assistant turn.
For how each reasoning model behaves on Together, see Reasoning.

Use tools

Define tools with name, description, and input_schema. When the model calls a tool, the response has stop_reason set to tool_use and one tool_use block per call. Run each tool, then send the results back in a single user turn with one tool_result block per call:
tool_choice accepts auto, any, tool (with a name), and none. To limit the model to one tool call per turn, set tool_choice.disable_parallel_tool_use to true. Together doesn’t run tools itself, so every tool runs in your app. Define custom tools, or use Anthropic’s bash, text_editor, and memory client tools. A request with a server tool (web_search, web_fetch, or code_execution) or the computer tool returns a 400. The one exception is a web_search tool that you send alongside other tools and that tool_choice doesn’t force. Together removes that tool from the request. Together also ignores mcp_servers.

Token usage and caching

usage reports input_tokens and output_tokens. Thinking tokens, including hidden reasoning, count as output tokens. Together caches repeated prompt prefixes automatically, so cache_control has no effect. cache_read_input_tokens counts the prompt tokens served from the cache, and input_tokens counts the rest. cache_creation_input_tokens is always 0. Together bills Messages API requests the same way as chat completions: per token at each model’s serverless prices, or per GPU-minute on dedicated model inference.

Other differences from Anthropic’s API

  • model: Use a Together model string. Claude model names, such as claude-sonnet-5-5, return a 404.
  • anthropic-beta header: Together accepts any value, but a beta header doesn’t turn on Anthropic beta features. The one exception is interleaved-thinking-2025-05-14, which lets budget_tokens exceed max_tokens.
  • Response id: It uses Together’s format rather than a msg_ prefix. Don’t parse it.
  • Structured outputs: output_config.format returns a 400. Ask for JSON in the prompt, or use structured outputs with chat completions.
  • metadata and service_tier: Neither changes the response. service_tier accepts only auto and standard_only.
  • Images and documents: GLM-5.3 and GLM-5.2 don’t accept images, so a request with an image block returns a 400. PDF document blocks also return a 400, so send documents as text.

Errors

Errors use the same JSON format as Anthropic’s API:
Each HTTP status maps to an error.type, and the Anthropic SDKs raise their usual exception class for each status: Together usually returns 503 for overloaded_error, where Anthropic’s API returns 529. The Anthropic Python SDK raises InternalServerError for a 503 rather than OverloadedError, so catch both if you handle overloads separately. A failure after a stream starts arrives as an error event in the stream. A few gateway-level failures, such as a 502 when Together can’t connect to the model’s servers, return Together’s OpenAI-style error body instead. To handle both formats, branch on the HTTP status rather than the error body. For causes and fixes by status code, see Error codes.

Next steps

Responses API

Call Together models with the OpenAI Responses API.

Together Link

Run Claude Code and other coding agents on Together models.