Endpoints

Deploy models on dedicated model inference, manage the resulting endpoints and deployments, and test or replace deployments with A/B tests, shadow experiments, and rollouts. Commands run within a Together project.

These commands require Together CLI 3.0 or later. In CLI 2.x, they were tg beta endpoints, and tg endpoints managed the retired v1 dedicated endpoints.

Deploy

Deploy a model to a new endpoint. If you pass the name or ID of an existing endpoint to --endpoint, the model deploys to that endpoint. If you pass a new name, the CLI creates the endpoint first, then deploys. The model is passed as the positional MODEL argument.

Shell

tg endpoints deploy zai-org/GLM-5.2 \
  --endpoint my-glm-endpoint \
  --min-replicas 1 \
  --max-replicas 3

When a model has more than one deployment profile, pass the profile’s exact modelName (from tg models public) as MODEL to select that profile, or re-run with --config <cr_...>. If the name still matches more than one profile, deploy returns an error that lists each profile’s modelName, config ID, quantization, GPUs, and parallelism, and shows an example --model <modelName> --config <cr_...> command. When a model has a single profile, the CLI selects it automatically. When MODEL is a low-rank adapter (LoRA), pass --attach-adapter or --merge. --attach-adapter attaches the adapter to a deployment of its base model on the endpoint whose deployment profile serves adapters. If the endpoint has no such deployment, the CLI creates one, then attaches the adapter. --merge creates a new deployment that serves the adapter merged into its base model. See Multi-LoRA deployment for the attach workflow.

Shell

tg endpoints deploy ml_abc123 \
  --endpoint my-sweep \
  --attach-adapter \
  --config cr_abc123

The CLI defaults --min-replicas and --max-replicas to 1, which may differ from the raw API defaults. If you pass only one bound, the CLI infers the other: --min-replicas alone mirrors into the max (including 0 to create the deployment stopped), and --max-replicas 0 alone lowers the min to 0. Passing 0 for one bound and a positive value for the other is an error.

Parameters

Flag	Description
`MODEL`	(required) The model to deploy. Accepts a public architecture name (for example `zai-org/GLM-5.2`), a profile `modelName` from `tg models public` (for example `zai-org/GLM-5.2-FP8`), a private model name, a private model ID (for example `ml_abc123`), or a fully resolved model path. An exact profile `modelName` selects that profile when it uniquely matches.
`--endpoint [string]`	(required) The endpoint to deploy to. Pass an existing endpoint name or ID to add the deployment to it, or a new name to create the endpoint first.
`--config [string]`	Config ID (`cr_...`) to deploy the model with. Auto-selected when the model has a single deployment profile, or when `MODEL` is an exact profile `modelName` that uniquely matches. Required when more than one profile still matches. For a private model or a LoRA adapter with more than one config, the CLI prompts you to pick one unless it runs non-interactively. With `--attach-adapter`, the config must have `adapter_mode` set to `dynamic` or `fixed`. To list a model’s configs, run `tg models configs <model_id>`.
`--min-replicas [number]`	Minimum number of replicas. Defaults to 1. Passed alone, it also sets `--max-replicas` to the same value, so `--min-replicas 0` alone creates the deployment stopped.
`--max-replicas [number]`	Maximum number of replicas. Must be greater than or equal to `--min-replicas`. Defaults to the `--min-replicas` value, or 1 when neither flag is set. `--max-replicas 0` alone also lowers the minimum to `0`.
`--scale-up-window [string]`	How long the scaling metric must stay above target before the platform adds replicas, in seconds (for example `30` or `30s`). Keeps brief spikes from triggering a scale-up.
`--scale-down-window [string]`	Cooldown after a scale-down before the platform removes more replicas, in seconds (for example `60` or `60s`). Higher values remove replicas more gradually.
`--scaling-metric [string]`	Autoscaling metric to scale on: `inflight_requests`, `active_sessions`, `gpu_utilization`, `token_utilization`, `cache_hit_rate`, `throughput_per_replica`, `ttft`, `decoding_speed`, or `e2e_latency`. Must be set together with `--scaling-target`. See Configure autoscaling.
`--scaling-target [number]`	Target value for `--scaling-metric`. Utilization metrics use `0`–`100`. Other metrics use their native units.
`--scaling-percentile [p50 \| p90 \| p95 \| p99]`	Percentile to scale on when `--scaling-metric` is a latency metric (`ttft`, `decoding_speed`, or `e2e_latency`). Defaults to `p95`.
`--inactive-timeout [number]`	Minutes without an inference request before the deployment scales to zero, counted from when it first becomes ready. Accepts `30` to `1440`, or `0` to disable. Off when unset. See Automatic idle shutdown.
`--max-concurrent-requests-per-replica [number]`	Maximum number of in-flight inference requests per replica. Omit to use the platform default. `0` means unlimited when the config’s per-replica concurrency limit is 1 or less.
`--deployment-name [string]`	Name for the deployment this command creates. Defaults to the model name plus a short unique suffix. With `--attach-adapter`, the CLI attaches the adapter to the endpoint’s deployment of this name, if one exists.
`--model-revision [string]`	Deprecated. Model revision ID to pin the deployment to. Instead, pass a fully qualified model path that ends in `/revisions/<REVISION_ID>` as `MODEL`.
`--placement [string]`	Placement profile ID (`pp_...`) to attach.
`--placement.regions [string]`	Comma-separated inline placement regions.
`--placement.constraint [required \| preferred]`	How strictly to enforce the inline placement regions.
`--placement.hipaa`	Require HIPAA-eligible placement, which runs replicas only on HIPAA-attested clusters. Enforced strictly, regardless of `--placement.constraint`. See compliance policy.
`--attach-adapter`	For a LoRA adapter `MODEL`, attach the adapter to a deployment of its base model whose profile has `adapter_mode` set to `dynamic` or `fixed`, preferring `dynamic`. If the endpoint has no such deployment, the CLI creates one first. Replica, autoscaling, placement, and `--inactive-timeout` flags apply only to a deployment the CLI creates.
`--merge`	For a LoRA adapter `MODEL`, create a new deployment that serves the adapter merged into its base model.
`--traffic-weight [number]`	Relative capacity weight for this deployment in the endpoint’s live traffic split. Set to `0` to send it no live traffic, or omit to leave routing unchanged.

List

List endpoints in the current project.

Shell

tg endpoints ls

Parameters

Flag	Description
`--org`	List organization-scoped endpoints instead of project-scoped ones.
`--limit [number]`	Maximum number of endpoints to return.
`--after [string]`	Pagination cursor to start from.

Get

Print details for an endpoint, deployment, or rollout. Pass an endpoint name or ID (ep_...) to see its deployments, traffic split, and active rollout, a deployment name or ID (dep_...) to inspect that deployment directly, or a rollout ID (rol_...) to see the rollout’s state, strategy, and progress. You can also omit get and pass the name or ID as the first argument (tg endpoints <name_or_id>). Endpoint responses include at most the 10 newest deployment summaries per endpoint. To list every deployment, use the deployments list API.

Shell

# By name
tg endpoints get my-endpoint

# By ID
tg endpoints get ep_abc123
tg endpoints get dep_abc123
tg endpoints get rol_abc123

# Implicit get (same as `get`)
tg endpoints my-endpoint

If more than one deployment shares the same bare name across endpoints, the CLI errors and asks you to pass a deployment ID (dep_...) or a fully qualified deployment name. For an endpoint, each deployment row lists the LoRA adapters attached to it. For a deployment, the output also shows its config’s selectors, such as adapter_mode. With --json, endpoint responses expose deployment state under deployments[].state, while deployment responses expose it under status.state.

Parameters

Flag	Description
`ID`	(required) Endpoint name, endpoint ID (`ep_...`), deployment name, deployment ID (`dep_...`), or rollout ID (`rol_...`).

Update

Update a deployment’s parameters: change its replica bounds, adjust autoscaling, set its share of endpoint traffic, or change an A/B variant’s percent. Pass the deployment ID (dep_...). The CLI resolves its parent endpoint automatically. At least one option must be set. Deployment names are immutable after creation.

Shell

# Set a deployment's replica bounds
tg endpoints update dep_abc123 --min-replicas 2 --max-replicas 4

# Scale on a specific metric and target
tg endpoints update dep_abc123 --scaling-metric gpu_utilization --scaling-target 70

# Stop a deployment by scaling it to zero (both flags required)
tg endpoints update dep_abc123 --min-replicas 0 --max-replicas 0

# Stop the deployment automatically after 60 idle minutes
tg endpoints update dep_abc123 --inactive-timeout 60

# Set a deployment's weight in the endpoint traffic split
tg endpoints update dep_abc123 --traffic-weight 30

# Take a deployment out of rotation without scaling it down
tg endpoints update dep_abc123 --traffic-weight 0

# Set an A/B variant's percent (takes from or returns to control)
tg endpoints update dep_variant456 --ab-percent 20

To stop a deployment, pass both --min-replicas 0 and --max-replicas 0. Passing only one zero bound returns an error. Partial nonzero updates (for example, only --min-replicas 3) still patch that field alone.

Parameters

Flag	Description
`ID`	(required) The deployment ID to update (`dep_...`).
`--min-replicas [number]`	Updated minimum number of replicas. To stop the deployment, pass both `--min-replicas 0` and `--max-replicas 0`. A single zero bound is an error.
`--max-replicas [number]`	Updated maximum number of replicas. Must be greater than or equal to `--min-replicas`.
`--scale-up-window [string]`	How long the scaling metric must stay above target before the platform adds replicas, in seconds.
`--scale-down-window [string]`	Cooldown after a scale-down before the platform removes more replicas, in seconds.
`--scaling-metric [string]`	Autoscaling metric to scale on: `inflight_requests`, `active_sessions`, `gpu_utilization`, `token_utilization`, `cache_hit_rate`, `throughput_per_replica`, `ttft`, `decoding_speed`, or `e2e_latency`. Must be set together with `--scaling-target`. See Configure autoscaling.
`--scaling-target [number]`	Target value for `--scaling-metric`. Utilization metrics use `0`–`100`. Other metrics use their native units.
`--scaling-percentile [p50 \| p90 \| p95 \| p99]`	Percentile to scale on when `--scaling-metric` is a latency metric (`ttft`, `decoding_speed`, or `e2e_latency`). Defaults to `p95`.
`--inactive-timeout [number]`	Updated inactivity timeout, in minutes (`30` to `1440`, or `0` to disable). Changing the value restarts the idle clock, so the new period counts from the update. See Automatic idle shutdown.
`--max-concurrent-requests-per-replica [number]`	Updated maximum number of in-flight inference requests per replica. `0` means unlimited when the config’s per-replica concurrency limit is 1 or less.
`--traffic-weight [number]`	Capacity weight for this deployment in the endpoint’s traffic split. Other deployments keep their weights. Set to `0` to stop routing traffic to this deployment. See Split traffic.
`--ab-percent [number]`	Traffic percentage (1–99) for this variant deployment in its A/B test. Raising it takes traffic from the control, and lowering it returns traffic to the control, so other variants are unchanged. Not valid for the control or for a deployment outside an A/B test. See Ramp the variant.
`--etag [string]`	ETag for optimistic concurrency on the deployment update. Does not apply to `--ab-percent` or `--traffic-weight`.

A deployment serves LoRA adapters only if its config has adapter_mode set to dynamic or fixed. You can’t change a deployment’s config with update, so to serve adapters, create a new deployment with such a config or let deploy --attach-adapter create one.

Delete

Delete an endpoint, deployment, A/B test, shadow experiment, or rollout. To detach a LoRA adapter from a deployment, pass the adapter attachment ID (dad_...). The command infers the resource type from the ID prefix (see Resource IDs).

Shell

tg endpoints rm ep_abc123

Alias: tg endpoints -d.

Parameters

Flag	Description
`ID`	(required) The ID of the resource to delete or detach (`ep_...`, `dep_...`, `dad_...`, `abx_...`, `exp_...`, or `rol_...`).
`--force`	Force-delete an endpoint that still has deployments.

Events

List an endpoint’s audit and lifecycle events, newest first. The feed merges endpoint-scoped events with the deployment-scoped events of every deployment under the endpoint. See Monitoring for how to read the feed.

Shell

tg endpoints events ep_abc123

The command prints one page of events with time, type, source, and message columns. When more events remain, it prints the --after command that displays the next page. Add --json for the raw event objects, including fields the table view omits, such as the event ID, level, and source kind.

Parameters

Flag	Description
`ID`	(required) The endpoint ID or name whose events to list.
`--deployment-ids [string]`	Comma-separated deployment IDs whose events should be included. Filtering by deployment excludes endpoint-scoped events.
`--min-level [debug \| info \| warn \| error]`	Minimum severity to include. Omit to disable severity filtering.
`--types [string]`	Comma-separated event types to include, such as `deployment.scaled` or `condition.set`.
`--subject-id [string]`	ID of a subject associated with the event, such as a rollout (`rol_...`), to read one subject’s audit trail out of the feed.
`--since [datetime]`	Return only events at or after this time.
`--until [datetime]`	Return only events strictly before this time.
`--limit [number]`	Maximum number of events to return. Max 10000, default 50.
`--after [string]`	Pagination cursor from a previous response.

A/B test

Fork a percentage of an endpoint’s live traffic from a control deployment to a new variant model, then compare the two. See A/B testing for the full workflow.

Shell

tg endpoints ab zai-org/GLM-5.2 \
  --control dep_control123 \
  --percent 10

Parameters

Flag	Description
`MODEL`	(required) The variant model to test. Accepts the same forms as `deploy`.
`--control [string]`	(required) ID of the control deployment that’s currently serving live traffic.
`--percent [number]`	(required) Percentage of traffic (1–99) to route to the variant. The control keeps the remainder, which must be at least 1%.
`--config [string]`	Config revision ID (`cr_...`) for the variant deployment. Auto-selected only when the model has a single compatible config.
`--name [string]`	Name for the variant deployment. Defaults to the model name with a short suffix.

Shadow

Mirror a fraction of an endpoint’s live traffic to a new model without affecting responses returned to clients. The model is the positional argument, and the source endpoint is passed with --endpoint. See Shadow deployments for details.

Shell

tg endpoints shadow zai-org/GLM-5.2 \
  --endpoint ep_abc123 \
  --rate 0.1

Parameters

Flag	Description
`MODEL`	(required) The model to shadow to. Accepts the same forms as `deploy`.
`--endpoint [string]`	(required) The endpoint (ID or name) whose live traffic is mirrored.
`--config [string]`	Config revision ID (`cr_...`) for the shadow deployment. Auto-selected only when the model has a single compatible config.
`--name [string]`	Name for the shadow deployment. Defaults to the model name with a short suffix.
`--rate [number]`	Fraction of live traffic to mirror (0.0–1.0). Required unless `--target-qps` is set.
`--key [string]`	Request-body field to use for sticky, key-based sampling.
`--target-qps [number]`	Per-gateway-replica target QPS for adaptive sampling.
`--window [string]`	Sliding window for adaptive-sampling QPS observation. Default: `60s`.

Rollout

Shift traffic from a source deployment to a target deployment under the same endpoint, or control the endpoint’s active rollout. See Rollouts for the full workflow, strategies, and requirements. To create a rollout, pass the target deployment and one strategy flag (--blue-green, --canary, or --rolling). A canary rollout can also carry a single metric gate, set with the --metric* flags. To control an active rollout, pass the endpoint (preferred), the deployment, or the rollout ID, plus one control flag.

Shell

tg endpoints rollout dep_target456 --canary

tg endpoints rollout dep_target456 --canary \
  --steps 25,50,75,100 --interval 300s

tg endpoints rollout dep_target456 --canary \
  --metric router_latency --metric-stat p95 \
  --metric-max-regression 50 --metric-direction higher-is-worse

tg endpoints rollout ep_abc123 --pause --reason "holding for review"

If the target deployment is a shadow experiment target or an A/B test member, pass --detach and the CLI detaches it from the experiment first (deleting the experiment if it can’t continue without the deployment), then starts the rollout.

Parameters

Flag	Description
`ID`	(required) To create a rollout, the target deployment’s ID or name. To control one, the endpoint ID (preferred), endpoint name, deployment ID or name, or rollout ID (`rol_...`).
`--source [string]`	Active deployment (ID or name) to shift traffic away from. Inferred from the endpoint’s traffic split when exactly one other deployment is taking traffic.
`--blue-green`	Blue-green strategy (single cutover).
`--canary`	Canary strategy (gradual traffic ladder). Without `--steps` and `--interval`, the rollout uses the server defaults: steps of 5, 25, 50, and 100 percent, with a 3-minute soak between steps. When the source and target already share traffic (typically after a cancel), the platform derives the default ladder from the target’s current share.
`--rolling`	Rolling strategy (capacity-preserving batch swap).
`--steps [string]`	Comma-separated canary traffic percentages, for example `10,50,100`. Only valid with `--canary`. Steps must strictly increase and end at 100. When the source and target already share traffic, the first step must be strictly above the target’s current share of the pair. Omit to use the server’s default ladder.
`--interval [string]`	Soak between canary steps, as a duration, for example `300s` or `10m`. Only valid with `--canary`. Omit to use the server default.
`--metric [string]`	Metric name for a single canary gate. Only valid with `--canary`. Must be a supported metric and set with either a threshold or a regression check. For `router_latency`, also set `--metric-stat`.
`--metric-stat [string]`	Aggregation applied to `--metric` over the metric window: `avg` (the mean) or a percentile (`p50`, `p90`, `p95`, or `p99`). Percentiles apply only to the latency histograms. Required for `router_latency`. Optional for the ratio and gauge metrics (`router_error_rate` and `inflight_requests`), where an omitted value is recorded as `AVG`.
`--metric-threshold [number]`	Absolute threshold for the metric, in the metric’s unit. Must be set with `--metric-operator`. Mutually exclusive with the regression flags.
`--metric-operator [string]`	Comparison operator for `--metric-threshold`: `gt`, `gte`, `lt`, or `lte`.
`--metric-max-regression [number]`	Maximum allowed regression percent versus the source. Must be at least 0 and set with `--metric-direction`. Mutually exclusive with the threshold flags.
`--metric-direction [string]`	Which direction counts as a regression: `higher-is-worse` or `lower-is-worse`.
`--metric-window [string]`	Query window for the metric gate, for example `300s`. Defaults to `300s`, and must be at least `60s`.
`--final-source-replicas [number]`	Source replica count after the rollout completes. Must be at least 0. Defaults to `0`, which drains and stops the source.
`--final-target-replicas [number]`	Target replica floor when the rollout completes, and for a rolling rollout, also the batch count. Must be at least 1. Defaults to the source deployment’s replica count, or to the pair’s combined count when both deployments are already in the traffic split (the frozen pair a cancel leaves behind). The completed target’s floor becomes the higher of this value and its own `min_replicas`. Its max is raised once to the highest of this value, the source’s `max_replicas`, and its own `max_replicas`, unless you edit the max mid-run, in which case your edit wins. Preview reports this expected end state as `landingMinReplicas` and `landingMaxReplicas`, and warns with `FINAL_BELOW_SOURCE_MIN` if the target is stopped and this value is below the source’s `min_replicas`.
`--detach`	Detach the target deployment from any shadow experiment or A/B test before starting the rollout. If the experiment or test can’t continue without the deployment, the CLI deletes it. Required when the target belongs to one. The detach can’t be undone, even if creating the rollout then fails.
`--cancel [freeze \| revert]`	Cancel the active rollout. `freeze` leaves the deployments at their current split, and `revert` restores the source deployment. Without a value, the server’s default cancel behavior applies.
`--pause`	Pause the active rollout at the current traffic split.
`--resume`	Resume a paused rollout from its current step.
`--promote`	Complete the rollout immediately by shifting 100% of traffic to the target.
`--reason [string]`	Reason recorded for auditing on a `--cancel` or `--pause` action.

The control flags (--cancel, --pause, --resume, and --promote) are mutually exclusive with each other and with the create options.

Global options

Every command also accepts the global parameters, including --json for machine-readable output and --project to override the target project. To set the project without --project, export the TOGETHER_PROJECT_ID environment variable. With neither set, the CLI uses the project you selected with tg login or tg switch. If you authenticate with an API key, an interactive deploy asks you to confirm the key’s project. CI jobs, agents, --non-interactive mode, and --json mode can’t answer that prompt, so set the project explicitly in those contexts before deploying:

Shell

export TOGETHER_PROJECT_ID=<your_project_id>

Resource IDs

An endpoint is a stable address that points at one or more deployments. Most commands take a resource ID, and its prefix identifies the resource type:

Prefix	Resource	Returned by
`ep_`	Endpoint.	`tg endpoints deploy`, `tg endpoints ls`
`dep_`	Deployment (a model plus config running behind an endpoint).	`tg endpoints get`
`dad_`	Adapter attachment (a LoRA adapter attached to a deployment).	`tg endpoints deploy --attach-adapter --json`
`abx_`	A/B test.	`tg endpoints ab`
`exp_`	Shadow experiment.	`tg endpoints shadow`
`rol_`	Rollout.	`tg endpoints rollout`

TOGETHER CLI

COMMANDS

Deploy

Parameters

List

Parameters

Get

Parameters

Update

Parameters

Delete

Parameters

Events

Parameters

A/B test

Parameters

Shadow

Parameters

Rollout

Parameters

Global options

Resource IDs

​Deploy

​Parameters

​List

​Parameters

​Get

​Parameters

​Update

​Parameters

​Delete

​Parameters

​Events

​Parameters

​A/B test

​Parameters

​Shadow

​Parameters

​Rollout

​Parameters

​Global options

​Resource IDs

Deploy

Parameters

List

Parameters

Get

Parameters

Update

Parameters

Delete

Parameters

Events

Parameters

A/B test

Parameters

Shadow

Parameters

Rollout

Parameters

Global options

Resource IDs