Skip to main content
Running more than one deployment on an endpoint lets you serve traffic across them for high availability, providing redundancy in case one of them fails. To split traffic between multiple deployments, list each with its relative weight. A weight sets a deployment’s share of traffic relative to its capacity: the actual share a deployment receives is proportional to its weight times its number of ready replicas.
Set each weight with endpoints update --traffic-weight, passing the deployment ID (dep_...). The CLI resolves its parent endpoint and preserves the other deployments’ weights, so run it once per deployment:
Shell
When each deployment has the same number of ready replicas, the weights behave like a direct ratio. For example, with equal replica counts, the weights in the example above send roughly 70% of traffic to one deployment and 30% to the other.
Weights are relative ratios, not percentages, so they don’t have to sum to any particular number: a split of 7 and 3 is equivalent to a split of 70 and 30. But weight sets relative capacity, not a fixed percentage. Each weight must be non-negative and finite. Two deployments with the same weight but different replica counts do not receive equal traffic. A deployment with weight 1 and two ready replicas draws the same traffic as one with weight 2 and a single ready replica. A deployment receives no traffic if it has a weight of 0, is absent from the split, or has zero ready replicas. Scaling a deployment to zero replicas takes it out of rotation even when it keeps a non-zero weight. Requests to an endpoint with no routable deployment return HTTP 400 with the error code endpoint_not_configured.

​
Shift traffic with replica counts

Treat weights as a stable definition of each deployment’s relative capacity, and shift traffic between deployments by changing their replica counts rather than editing weights. Because a deployment’s share tracks its ready replicas, scaling one deployment up (or another down) moves traffic without you having to recompute a set of weights. To migrate traffic from one deployment to another on live traffic, use a rollout instead of shifting the split by hand: it moves traffic in controlled steps, can gate each step on live metrics, and drains the source when it completes. A deployment’s weight is remembered when it scales to zero and reapplies automatically when it scales back up, so you don’t need to re-add it to the split after a scale-down.

​
Troubleshooting

  • Update traffic split returns 409 with cannot update traffic_split while rollout ... is active: A rollout in any non-terminal state (including pending or paused) owns the endpoint’s traffic split. Complete, cancel, or delete the rollout before editing weights.
  • Update traffic split returns 400 with the deployment is a shadow experiment target and cannot serve live traffic; remove the shadow target first: The deployment is registered as a target in an active shadow experiment. Remove it from the experiment (or delete the experiment) before giving it a non-zero weight in the traffic split.

​
Next steps

Route traffic

Understand how an endpoint resolves each request to a deployment.

Configure autoscaling

Set the replica bounds that shift traffic between deployments.

Run an A/B test

Compare a candidate deployment against a baseline on live traffic.

Manage deployments

Create, poll, scale, stop, and delete deployments.