Skip to main content
POST
Together AI SDK (Python)

Authorizations

Authorization
string
header
default:default
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

Episode shape, model, GPU type, and concurrency for a rollout-only estimate

episodes
integer
required

Total number of episodes (N). Must be between 1 and 10000000.

Required range: 1 <= x <= 10000000
concurrency
integer
required

Maximum concurrent episodes across all generator replicas (B). Must be positive.

Required range: x >= 1
prompt_tokens
integer
required

Initial prompt tokens per episode (P). Must be positive.

Required range: x >= 1
turns
integer
required

Assistant turns per episode (T). Must be between 1 and 1024.

Required range: 1 <= x <= 1024
added_tokens_per_turn
integer
required

Tokens added before each later turn (K), including the prior generated tokens followed by environment tokens. Must be at least generated_tokens_per_turn.

generated_tokens_per_turn
integer
required

Tokens generated on each turn (O). Must be positive.

Required range: x >= 1
base_model
string
required

Base model to estimate. The model, GPU type, and training mode must support rollout estimates.

Required string length: 1 - 256
Pattern: ^[A-Za-z0-9._/-]+$
Example:

"zai-org/GLM-5.3"

distinct_prompts_per_replica
integer

Number of distinct initial prompts among the concurrent episodes of one replica (Q). When episodes_per_prompt is given, defaults to ceil(concurrent episodes / G). Otherwise defaults to 1 when shared_prefix_tokens equals prompt_tokens, and to the concurrent episodes per replica when it does not. Must be 1 when shared_prefix_tokens equals prompt_tokens.

Required range: x >= 1
episodes_per_prompt
integer

Maximum sibling episodes per initial prompt (G). When distinct_prompts_per_replica is given, defaults to ceil(concurrent episodes / Q). Otherwise defaults to the concurrent episodes per replica when shared_prefix_tokens equals prompt_tokens, and to 1 when it does not. When shared_prefix_tokens equals prompt_tokens, must be at least the concurrent episodes per replica.

Required range: x >= 1
shared_prefix_tokens
integer
default:0

Leading tokens shared by all distinct initial prompts (H). Defaults to 0; cannot exceed prompt_tokens.

compute_config
object

Compute layout to estimate, as for a model resource. Omitted fields use the model's default GPU type and one generator replica (D); num_generator_replicas must be positive.

lora_enabled
boolean
default:true

Whether to estimate a resource that hosts LoRA sessions (true, default) or a single full-weight session (false), as in a model resource. The model and GPU type must support rollout estimates in this mode.

Example:

true

Response

Estimated rollout time and price

Estimated rollout elapsed time, tokens, throughput, GPU-hours, and price

wall_seconds
number<double>
required

Estimated time until all rollout episodes finish, in seconds.

allocated_gpu_hours
number<double>
required

Generator GPUs across all replicas times wall time, in GPU-hours.

submitted_prompt_tokens
required

Prompt tokens submitted across all turns. JSON encodes this count as a decimal string.

generated_tokens
required

Generated output tokens across all turns. JSON encodes this count as a decimal string.

effective_decode_tokens_per_second
number<double>
required

generated_tokens divided by wall_seconds.

effective_distinct_prompts_per_replica
integer
required

Effective distinct_prompts_per_replica (Q) for the largest concurrent batch of one replica.

effective_episodes_per_prompt
integer
required

Effective episodes_per_prompt (G) for the largest concurrent batch of one replica.

effective_shared_prefix_tokens
integer
required

Effective shared_prefix_tokens (H).

lora_enabled
boolean
required

Whether the estimate is for a LoRA resource (true) or a full-weight resource (false).

billed_gpu_hours
number<double>
required

GPUs of the whole model resource, trainer and generator, times wall time, in GPU-hours. A model resource is billed for all of its GPUs while the rollouts run. An approximation measured on single-wave rollouts (concurrency equal to episodes) of up to four turns on one generator replica: actual usage can be up to about 45% higher when episodes share a prompt, and about 20% higher or lower otherwise. Longer multi-turn rollouts with distinct prompts can take longer than estimated.

estimated_price
number<double>
required

Estimated on-demand price of the rollouts: billed_gpu_hours times the GPU type's list price per GPU-hour, the rate the model resource cost estimate uses. An approximation that inherits the uncertainty of billed_gpu_hours; it is not a quote.

currency
string
required

Currency of estimated_price.

Example:

"USD"

effective_prefill_tokens_per_second
number<double>
required

submitted_prompt_tokens divided by wall_seconds, counting prompt tokens served from the prompt cache.