> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Responses API

> Call Together AI models with the OpenAI Responses API through the OpenAI Python and TypeScript SDKs.

Together AI implements the OpenAI [Responses API](https://platform.openai.com/docs/api-reference/responses) at `POST /v1/responses`. If your app is built on the Responses API, change the API key, base URL, and model to run it on open-source models hosted by Together. Together doesn't support stored responses, so send the full conversation with each request.

## Set up the client

Install the OpenAI SDK for your language:

```bash Python theme={null}
pip install openai
```

```bash TypeScript theme={null}
npm install openai
```

Set `api_key` to your [Together API key](https://docs.together.ai/docs/api-keys-authentication.md) and `base_url` to `https://api.together.ai/v1`, then create a response:

```python Python theme={null}
import os
import openai

client = openai.OpenAI(
    api_key=os.environ["TOGETHER_API_KEY"],
    base_url="https://api.together.ai/v1",
)

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Explain what a mixture-of-experts model is in two sentences.",
)

print(response.output_text)
```

```typescript TypeScript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOGETHER_API_KEY,
  baseURL: "https://api.together.ai/v1",
});

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Explain what a mixture-of-experts model is in two sentences.",
});

console.log(response.output_text);
```

```bash cURL theme={null}
curl https://api.together.ai/v1/responses \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Explain what a mixture-of-experts model is in two sentences."
  }'
```

When the model reasons, the response's `output` array starts with a `reasoning` item, followed by a `message` item with the answer. The SDKs' `output_text` helper returns the answer text. With cURL, read it from the `message` item's `content[0].text`.

`input` accepts a string or a list of items. For a system prompt, pass `instructions` or include a `system` or `developer` message in the `input` list.

## Supported models

Supported models differ between serverless inference and [dedicated model inference](https://docs.together.ai/docs/dedicated-endpoints/overview.md):

| Model | API model string | Serverless inference | Dedicated model inference |
| - | - | :-: | :-: |
| GLM-5.2 | `zai-org/GLM-5.2` | ✓ | ✓ |
| GLM-5.3 | `zai-org/GLM-5.3` | ✓ | - |
| GLM-5.3 Flash | `zai-org/GLM-5.3-Flash` | - | ✓ |

Other serverless models can accept Responses API requests, but they aren't supported and may return malformed output.

To [send requests to dedicated model inference](https://docs.together.ai/docs/dedicated-endpoints/requests.md), set the base URL to `https://api-inference.together.ai/v1` and pass your endpoint string as `model`.

To use the Responses API with an existing [dedicated endpoint](https://docs.together.ai/docs/dedicated-endpoints/overview.md) or a model that isn't supported yet, [contact sales](https://www.together.ai/contact-sales).

## Endpoint support

| OpenAI SDK method | Endpoint | Status |
| - | - | - |
| `responses.create` | `POST /v1/responses` | Supported |
| `responses.retrieve`, `responses.delete`, `responses.cancel`, `responses.input_items.list` | `/v1/responses/{response_id}` | Not supported. Returns `404` with the code `unsupported_operation`. |
| `responses.input_tokens.count` | `POST /v1/responses/input_tokens` | Not supported. Returns `404` with the code `unsupported_operation`. |
| `responses.compact` | `POST /v1/responses/compact` | Not supported. Returns `404` with the code `unsupported_operation`. |
| `conversations.*` | `/v1/conversations` | Not supported. |

## Stream responses

Set `stream` to `true` to receive server-sent events as the model generates them:

```python Python theme={null}
stream = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Write a haiku about the ocean.",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
```

```typescript TypeScript theme={null}
const stream = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Write a haiku about the ocean.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}
```

```bash cURL theme={null}
curl https://api.together.ai/v1/responses \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Write a haiku about the ocean.",
    "stream": true
  }'
```

The stream starts with `response.created` and `response.in_progress`, and ends with `response.completed`. In between, each output item arrives as `response.output_item.added`, then its deltas, then `response.output_item.done`. The delta events depend on the item type:

* **Reasoning:** `response.reasoning_text.delta` and `response.reasoning_text.done`.
* **Answer text:** `response.content_part.added`, `response.output_text.delta`, `response.output_text.done`, and `response.content_part.done`.
* **Function calls:** `response.function_call_arguments.delta` and `response.function_call_arguments.done`.

## Build multi-turn conversations

Together doesn't support stored responses, so `previous_response_id` and the Conversations API aren't available. To continue a conversation, append the previous response's `output` items to your `input`, add the next user message, and send the full list:

```python Python theme={null}
history = [{"role": "user", "content": "My name is Ada. Remember it."}]

response = client.responses.create(model="zai-org/GLM-5.3", input=history)
history += response.output
history.append({"role": "user", "content": "What's my name?"})

response = client.responses.create(model="zai-org/GLM-5.3", input=history)
print(response.output_text)
```

```typescript TypeScript theme={null}
const history: OpenAI.Responses.ResponseInput = [
  { role: "user", content: "My name is Ada. Remember it." },
];

let response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: history,
});
history.push(...(response.output as OpenAI.Responses.ResponseInputItem[]));
history.push({ role: "user", content: "What's my name?" });

response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: history,
});
console.log(response.output_text);
```

## Call functions

Define each function with `type: "function"`, a `name`, a `description`, and JSON Schema `parameters`. When the model calls a function, the response has a `function_call` item with the function `name`, a JSON-encoded `arguments` string, and a `call_id`.

To return the results, append the response's `output` items to your `input`, add a `function_call_output` item with the matching `call_id` for each call, and send the full list back:

```python Python theme={null}
import json

tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }
]


def get_weather(city):
    return json.dumps({"city": city, "temperature_c": 18})


history = [
    {"role": "user", "content": "What's the weather in Paris and Tokyo?"}
]

while True:
    response = client.responses.create(
        model="zai-org/GLM-5.3",
        input=history,
        tools=tools,
    )
    calls = [item for item in response.output if item.type == "function_call"]
    if not calls:
        break
    for item in response.output:
        history.append(item)
        if item.type == "function_call":
            args = json.loads(item.arguments)
            history.append(
                {
                    "type": "function_call_output",
                    "call_id": item.call_id,
                    "output": get_weather(**args),
                }
            )

print(response.output_text)
```

```typescript TypeScript theme={null}
const tools: OpenAI.Responses.Tool[] = [
  {
    type: "function",
    name: "get_weather",
    description: "Get the current weather for a city.",
    parameters: {
      type: "object",
      properties: { city: { type: "string" } },
      required: ["city"],
    },
    strict: false,
  },
];

function getWeather(city: string): string {
  return JSON.stringify({ city, temperature_c: 18 });
}

const history: OpenAI.Responses.ResponseInput = [
  { role: "user", content: "What's the weather in Paris and Tokyo?" },
];

while (true) {
  const response = await client.responses.create({
    model: "zai-org/GLM-5.3",
    input: history,
    tools,
  });
  const calls = response.output.filter((item) => item.type === "function_call");
  if (calls.length === 0) {
    console.log(response.output_text);
    break;
  }
  for (const item of response.output) {
    history.push(item as OpenAI.Responses.ResponseInputItem);
    if (item.type === "function_call") {
      const { city } = JSON.parse(item.arguments);
      history.push({
        type: "function_call_output",
        call_id: item.call_id,
        output: getWeather(city),
      });
    }
  }
}
```

`tool_choice` accepts `auto`, `required`, `none`, and a named function (`{"type": "function", "name": "get_weather"}`). Set `parallel_tool_calls` to `false` to get at most one call per turn.

Together doesn't run tools itself, so your app runs every tool the model calls. Use function tools, custom tools, or the `shell`, `local_shell`, and `apply_patch` tools. Hosted tools (such as `file_search`, `code_interpreter`, `image_generation`, and `computer_use_preview`) and remote MCP servers return a `400`. Together removes a `web_search` tool from the request, but forcing one with `tool_choice` returns a `400`.

## Control reasoning

Set `reasoning.effort` to `none`, `minimal`, `low`, `medium`, `high`, or `xhigh`:

```python Python theme={null}
response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="A bat and a ball cost $1.10 in total. The bat costs $1.00 more "
    "than the ball. How much is the ball?",
    reasoning={"effort": "low"},
)

for item in response.output:
    if item.type == "reasoning":
        print("Reasoning:", item.content[0].text)

print("Answer:", response.output_text)
```

```typescript TypeScript theme={null}
const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input:
    "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?",
  reasoning: { effort: "low" },
});

for (const item of response.output) {
  if (item.type === "reasoning" && item.content) {
    console.log("Reasoning:", item.content[0].text);
  }
}
console.log("Answer:", response.output_text);
```

Reasoning on Together differs from OpenAI's API:

* **Effort levels:** Together passes the level to the model's chat template, so its effect depends on the model. GLM-5.3 reasons at every level, including `none`. On GLM-5.2, `minimal` turns reasoning off, `low` and `medium` request maximum effort, and `high` requests high effort.
* **Reasoning text:** Each `reasoning` item carries the full reasoning text in `content`, as `reasoning_text` parts.
* **`reasoning.summary`:** Together doesn't generate summaries, so `summary` is always empty.
* **`encrypted_content`:** If you request `include: ["reasoning.encrypted_content"]`, `encrypted_content` holds a base64-encoded copy of the reasoning text. It isn't encrypted.

## Get structured outputs

To constrain the answer to a JSON schema, pass the schema in `text.format`:

```python Python theme={null}
import json

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Extract the person: Ada Lovelace was born in 1815.",
    text={
        "format": {
            "type": "json_schema",
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "birth_year": {"type": "integer"},
                },
                "required": ["name", "birth_year"],
                "additionalProperties": False,
            },
        }
    },
)

print(json.loads(response.output_text))
```

```typescript TypeScript theme={null}
const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  input: "Extract the person: Ada Lovelace was born in 1815.",
  text: {
    format: {
      type: "json_schema",
      name: "person",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          birth_year: { type: "integer" },
        },
        required: ["name", "birth_year"],
        additionalProperties: false,
      },
    },
  },
});

console.log(JSON.parse(response.output_text));
```

## Token usage and billing

The response's `usage` object reports `input_tokens`, `output_tokens`, and `total_tokens`. `input_tokens_details.cached_tokens` counts the prompt tokens served from Together's automatic prefix cache. `output_tokens_details.reasoning_tokens` counts reasoning tokens, which are included in `output_tokens`.

Together bills Responses API requests the same way as chat completions: per token at each model's [serverless prices](https://docs.together.ai/docs/serverless/models.md), or per GPU-minute on [dedicated model inference](https://docs.together.ai/docs/dedicated-endpoints/pricing.md).

## Other differences from OpenAI's API

* **`store`:** Together doesn't store responses, so the response reports `store: false` even when you set it to `true`.
* **`previous_response_id` and `conversation`:** Both return a `400`. [Send the full history](#build-multi-turn-conversations) instead.
* **`background`:** Setting it to `true` returns a `400`.
* **Images and files:** GLM-5.3 and GLM-5.2 don't accept images, so an `input_image` part returns a `400`. The exception is an `input_image` part that references only a `file_id`, which Together drops without an error. An `input_file` part also returns a `400`.

## Errors

Errors use the same JSON format as OpenAI's API, and the OpenAI SDKs raise their usual exception class for each HTTP status code. For example, `responses.retrieve` returns a `404` with this body:

```json theme={null}
{
  "error": {
    "message": "Retrieving, deleting, canceling, and listing responses are not supported.",
    "type": "invalid_request_error",
    "param": null,
    "code": "unsupported_operation"
  }
}
```

Together's `type` and `code` values don't always match OpenAI's, so branch on the HTTP status code, not on `type` or `code`. For the causes and fixes of each status code, see [Error codes](https://docs.together.ai/docs/error-codes.md).

## Next steps

> [card] [Messages API](https://docs.together.ai/docs/inference/messages-api.md)
>
> Call Together models with the Anthropic Messages API.

> [card] [Together Link](https://docs.together.ai/docs/togetherlink.md)
>
> Run Codex and other coding agents on Together models.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.