Estimate rollout time and price
Estimates rollout-only time, generator GPU-hours, billed GPU-hours of the whole model resource, and price for a supported model and GPU type. Each generator replica may contain several inference servers or span multiple nodes. Assumes identical episodes, exact reuse of generated tokens between turns, turns that start together within a replica, no environment delay, and no concurrent computation of the same initial prompt. Prompt-cache reuse and capacity misses are approximate, especially for multi-turn episodes. Rates are measured in the training mode reported by lora_enabled. Cache-transfer and request-preemption delays are not included, so long multi-turn rollouts with distinct prompts can take longer than estimated.
Authorizations
Body
Episode shape, model, GPU type, and concurrency for a rollout-only estimate
Number of distinct initial prompts among the concurrent episodes of one replica (Q). When episodes_per_prompt is given, defaults to ceil(concurrent episodes / G). Otherwise defaults to 1 when shared_prefix_tokens equals prompt_tokens, and to the concurrent episodes per replica when it does not. Must be 1 when shared_prefix_tokens equals prompt_tokens.
x >= 1Maximum sibling episodes per initial prompt (G). When distinct_prompts_per_replica is given, defaults to ceil(concurrent episodes / Q). Otherwise defaults to the concurrent episodes per replica when shared_prefix_tokens equals prompt_tokens, and to 1 when it does not. When shared_prefix_tokens equals prompt_tokens, must be at least the concurrent episodes per replica.
x >= 1Leading tokens shared by all distinct initial prompts (H). Defaults to 0; cannot exceed prompt_tokens.
Response
Estimated rollout time and price
Estimated rollout elapsed time, tokens, throughput, GPU-hours, and price
Effective shared_prefix_tokens (H).
GPUs of the whole model resource, trainer and generator, times wall time, in GPU-hours. A model resource is billed for all of its GPUs while the rollouts run. An approximation measured on single-wave rollouts (concurrency equal to episodes) of up to four turns on one generator replica: actual usage can be up to about 45% higher when episodes share a prompt, and about 20% higher or lower otherwise. Longer multi-turn rollouts with distinct prompts can take longer than estimated.