Responses API

Together AI implements the OpenAI Responses API at POST /v1/responses. If your app is built on the Responses API, change the API key, base URL, and model to run it on open-source models hosted by Together. Together doesn’t support stored responses, so send the full conversation with each request.

Set up the client

Install the OpenAI SDK for your language:

pip install openai

npm install openai

Set api_key to your Together API key and base_url to https://api.together.ai/v1, then create a response:

import os
import openai

client = openai.OpenAI(
    api_key=os.environ["TOGETHER_API_KEY"],
    base_url="https://api.together.ai/v1",
)

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Explain what a mixture-of-experts model is in two sentences.",
)

print(response.output_text)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOGETHER_API_KEY,
  baseURL: "https://api.together.ai/v1",
});

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Explain what a mixture-of-experts model is in two sentences.",
});

console.log(response.output_text);

curl https://api.together.ai/v1/responses \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Explain what a mixture-of-experts model is in two sentences."
  }'

When the model reasons, the response’s output array starts with a reasoning item, followed by a message item with the answer. The SDKs’ output_text helper returns the answer text. With cURL, read it from the message item’s content[0].text. input accepts a string or a list of items. For a system prompt, pass instructions or include a system or developer message in the input list.

Supported models

Supported models differ between serverless inference and dedicated model inference:

Model	API model string	Serverless inference	Dedicated model inference
GLM-5.2	`zai-org/GLM-5.2`	✓	✓
GLM-5.3	`zai-org/GLM-5.3`	✓	-
GLM-5.3 Flash	`zai-org/GLM-5.3-Flash`	-	✓

Other serverless models can accept Responses API requests, but they aren’t supported and may return malformed output. To send requests to dedicated model inference, set the base URL to https://api-inference.together.ai/v1 and pass your endpoint string as model. To use the Responses API with an existing dedicated endpoint or a model that isn’t supported yet, contact sales.

Endpoint support

OpenAI SDK method	Endpoint	Status
`responses.create`	`POST /v1/responses`	Supported
`responses.retrieve`, `responses.delete`, `responses.cancel`, `responses.input_items.list`	`/v1/responses/{response_id}`	Not supported. Returns `404` with the code `unsupported_operation`.
`responses.input_tokens.count`	`POST /v1/responses/input_tokens`	Not supported. Returns `404` with the code `unsupported_operation`.
`responses.compact`	`POST /v1/responses/compact`	Not supported. Returns `404` with the code `unsupported_operation`.
`conversations.*`	`/v1/conversations`	Not supported.

Stream responses

Set stream to true to receive server-sent events as the model generates them:

stream = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Write a haiku about the ocean.",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

const stream = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Write a haiku about the ocean.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}

curl https://api.together.ai/v1/responses \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Write a haiku about the ocean.",
    "stream": true
  }'

The stream starts with response.created and response.in_progress, and ends with response.completed. In between, each output item arrives as response.output_item.added, then its deltas, then response.output_item.done. The delta events depend on the item type:

Reasoning: response.reasoning_text.delta and response.reasoning_text.done.
Answer text: response.content_part.added, response.output_text.delta, response.output_text.done, and response.content_part.done.
Function calls: response.function_call_arguments.delta and response.function_call_arguments.done.

Build multi-turn conversations

Together doesn’t support stored responses, so previous_response_id and the Conversations API aren’t available. To continue a conversation, append the previous response’s output items to your input, add the next user message, and send the full list:

history = [{"role": "user", "content": "My name is Ada. Remember it."}]

response = client.responses.create(model="zai-org/GLM-5.3", input=history)
history += response.output
history.append({"role": "user", "content": "What's my name?"})

response = client.responses.create(model="zai-org/GLM-5.3", input=history)
print(response.output_text)

const history: OpenAI.Responses.ResponseInput = [
  { role: "user", content: "My name is Ada. Remember it." },
];

let response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: history,
});
history.push(...(response.output as OpenAI.Responses.ResponseInputItem[]));
history.push({ role: "user", content: "What's my name?" });

response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: history,
});
console.log(response.output_text);

Call functions

Define each function with type: "function", a name, a description, and JSON Schema parameters. When the model calls a function, the response has a function_call item with the function name, a JSON-encoded arguments string, and a call_id. To return the results, append the response’s output items to your input, add a function_call_output item with the matching call_id for each call, and send the full list back:

import json

tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }
]


def get_weather(city):
    return json.dumps({"city": city, "temperature_c": 18})


history = [
    {"role": "user", "content": "What's the weather in Paris and Tokyo?"}
]

while True:
    response = client.responses.create(
        model="zai-org/GLM-5.3",
        input=history,
        tools=tools,
    )
    calls = [item for item in response.output if item.type == "function_call"]
    if not calls:
        break
    for item in response.output:
        history.append(item)
        if item.type == "function_call":
            args = json.loads(item.arguments)
            history.append(
                {
                    "type": "function_call_output",
                    "call_id": item.call_id,
                    "output": get_weather(**args),
                }
            )

print(response.output_text)

const tools: OpenAI.Responses.Tool[] = [
  {
    type: "function",
    name: "get_weather",
    description: "Get the current weather for a city.",
    parameters: {
      type: "object",
      properties: { city: { type: "string" } },
      required: ["city"],
    },
    strict: false,
  },
];

function getWeather(city: string): string {
  return JSON.stringify({ city, temperature_c: 18 });
}

const history: OpenAI.Responses.ResponseInput = [
  { role: "user", content: "What's the weather in Paris and Tokyo?" },
];

while (true) {
  const response = await client.responses.create({
    model: "zai-org/GLM-5.3",
    input: history,
    tools,
  });
  const calls = response.output.filter((item) => item.type === "function_call");
  if (calls.length === 0) {
    console.log(response.output_text);
    break;
  }
  for (const item of response.output) {
    history.push(item as OpenAI.Responses.ResponseInputItem);
    if (item.type === "function_call") {
      const { city } = JSON.parse(item.arguments);
      history.push({
        type: "function_call_output",
        call_id: item.call_id,
        output: getWeather(city),
      });
    }
  }
}

tool_choice accepts auto, required, none, and a named function ({"type": "function", "name": "get_weather"}). Set parallel_tool_calls to false to get at most one call per turn. Together doesn’t run tools itself, so your app runs every tool the model calls. Use function tools, custom tools, or the shell, local_shell, and apply_patch tools. Hosted tools (such as file_search, code_interpreter, image_generation, and computer_use_preview) and remote MCP servers return a 400. Together removes a web_search tool from the request, but forcing one with tool_choice returns a 400.

Control reasoning

Set reasoning.effort to none, minimal, low, medium, high, or xhigh:

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="A bat and a ball cost $1.10 in total. The bat costs $1.00 more "
    "than the ball. How much is the ball?",
    reasoning={"effort": "low"},
)

for item in response.output:
    if item.type == "reasoning":
        print("Reasoning:", item.content[0].text)

print("Answer:", response.output_text)

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input:
    "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?",
  reasoning: { effort: "low" },
});

for (const item of response.output) {
  if (item.type === "reasoning" && item.content) {
    console.log("Reasoning:", item.content[0].text);
  }
}
console.log("Answer:", response.output_text);

Reasoning on Together differs from OpenAI’s API:

Effort levels: Together passes the level to the model’s chat template, so its effect depends on the model. GLM-5.3 reasons at every level, including none. On GLM-5.2, minimal turns reasoning off, low and medium request maximum effort, and high requests high effort.
Reasoning text: Each reasoning item carries the full reasoning text in content, as reasoning_text parts.
reasoning.summary: Together doesn’t generate summaries, so summary is always empty.
encrypted_content: If you request include: ["reasoning.encrypted_content"], encrypted_content holds a base64-encoded copy of the reasoning text. It isn’t encrypted.

Get structured outputs

To constrain the answer to a JSON schema, pass the schema in text.format:

import json

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Extract the person: Ada Lovelace was born in 1815.",
    text={
        "format": {
            "type": "json_schema",
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "birth_year": {"type": "integer"},
                },
                "required": ["name", "birth_year"],
                "additionalProperties": False,
            },
        }
    },
)

print(json.loads(response.output_text))

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Extract the person: Ada Lovelace was born in 1815.",
  text: {
    format: {
      type: "json_schema",
      name: "person",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          birth_year: { type: "integer" },
        },
        required: ["name", "birth_year"],
        additionalProperties: false,
      },
    },
  },
});

console.log(JSON.parse(response.output_text));

Token usage and billing

The response’s usage object reports input_tokens, output_tokens, and total_tokens. input_tokens_details.cached_tokens counts the prompt tokens served from Together’s automatic prefix cache. output_tokens_details.reasoning_tokens counts reasoning tokens, which are included in output_tokens. Together bills Responses API requests the same way as chat completions: per token at each model’s serverless prices, or per GPU-minute on dedicated model inference.

Other differences from OpenAI’s API

store: Together doesn’t store responses, so the response reports store: false even when you set it to true.
previous_response_id and conversation: Both return a 400. Send the full history instead.
background: Setting it to true returns a 400.
Images and files: GLM-5.3 and GLM-5.2 don’t accept images, so an input_image part returns a 400. The exception is an input_image part that references only a file_id, which Together drops without an error. An input_file part also returns a 400.

Errors

Errors use the same JSON format as OpenAI’s API, and the OpenAI SDKs raise their usual exception class for each HTTP status code. For example, responses.retrieve returns a 404 with this body:

{
  "error": {
    "message": "Retrieving, deleting, canceling, and listing responses are not supported.",
    "type": "invalid_request_error",
    "param": null,
    "code": "unsupported_operation"
  }
}

Together’s type and code values don’t always match OpenAI’s, so branch on the HTTP status code, not on type or code. For the causes and fixes of each status code, see Error codes.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Set up the client

Supported models

Endpoint support

Stream responses

Build multi-turn conversations

Call functions

Control reasoning

Get structured outputs

Token usage and billing

Other differences from OpenAI’s API

Errors

Next steps

Messages API

Together Link

​Set up the client

​Supported models

​Endpoint support

​Stream responses

​Build multi-turn conversations

​Call functions

​Control reasoning

​Get structured outputs

​Token usage and billing

​Other differences from OpenAI’s API

​Errors

​Next steps

Messages API

Together Link

Set up the client

Supported models

Endpoint support

Stream responses

Build multi-turn conversations

Call functions

Control reasoning

Get structured outputs

Token usage and billing

Other differences from OpenAI’s API

Errors

Next steps