Create a dedicated endpoint - Together AI docs

# Docs for v1 can be found by changing the above selector ^ from together import Together import os client = Together( api_key=os.environ.get("TOGETHER_API_KEY"), ) endpoint = client.endpoints.create( model="Qwen/Qwen3.5-9B-FP8", hardware="1x_nvidia_a100_80gb_sxm", autoscaling={ "min_replicas": 2, "max_replicas": 5, } ) print(endpoint.id)

{ "object": "<unknown>", "id": "endpoint-d23901de-ef8f-44bf-b3e7-de9c1ca8f2d7", "name": "devuser/deepseek-ai/DeepSeek-R1-a32b82a1", "display_name": "My DeepSeek R1 endpoint", "model": "deepseek-ai/DeepSeek-R1", "hardware": "8x_nvidia_h200_140gb_sxm", "type": "dedicated", "owner": "devuser", "state": "STARTED", "autoscaling": { "min_replicas": 123, "max_replicas": 123 }, "created_at": "2025-02-04T10:43:55.405Z" }

Authorizations

Authorization

string

header

default:default

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

model

string

required

The model to deploy on this endpoint

Example:

"deepseek-ai/DeepSeek-R1"

hardware

string

required

The hardware configuration to use for this endpoint

Example:

"1x_nvidia_a100_80gb_sxm"

autoscaling

object

required

Configuration for automatic scaling of the endpoint

Show child attributes

display_name

string

A human-readable name for the endpoint

Example:

"My Llama3 70b endpoint"

disable_prompt_cache

boolean

default:false

deprecated

This parameter is deprecated and no longer has any effect.

disable_speculative_decoding

boolean

default:false

Whether to disable speculative decoding for this endpoint

state

enum<string>

default:STARTED

The desired state of the endpoint

Available options:

STARTED,

STOPPED

Example:

"STARTED"

inactive_timeout

integer | null

The number of minutes of inactivity after which the endpoint stops automatically. Set to null, omit, or set to 0 to disable automatic timeout.

Example:

60

availability_zone

string

Create the endpoint in a specified availability zone (e.g., us-central-4b)

Response

200

Details about a dedicated endpoint deployment

object

any

required

The object type, which is always endpoint.

string

required

Unique identifier for the endpoint

Example:

"endpoint-d23901de-ef8f-44bf-b3e7-de9c1ca8f2d7"

name

string

required

System name for the endpoint

Example:

"devuser/deepseek-ai/DeepSeek-R1-a32b82a1"

display_name

string

required

Human-readable name for the endpoint

Example:

"My DeepSeek R1 endpoint"

model

string

required

The model deployed on this endpoint

Example:

"deepseek-ai/DeepSeek-R1"

hardware

string

required

The hardware configuration used for this endpoint

Example:

"8x_nvidia_h200_140gb_sxm"

type

enum<string>

required

The type of endpoint

Available options:

dedicated

Example:

"dedicated"

owner

string

required

The owner of this endpoint

Example:

"devuser"

state

enum<string>

required

Current state of the endpoint

Available options:

PENDING,

STARTING,

STARTED,

STOPPING,

STOPPED,

ERROR

Example:

"STARTED"

autoscaling

object

required

Configuration for automatic scaling of the endpoint

Show child attributes

created_at

string<date-time>

required

Timestamp when the endpoint was created

Example:

"2025-02-04T10:43:55.405Z"