Clusters

GPU clusters are a beta feature. Behavior, flags, and supported hardware can change. Reach out to your Together AI contact or contact sales with feedback.

Create a cluster

tg beta clusters create

Parameters

Flag	Description
`--name [string]`	Name for the cluster.
`--num-gpus [integer]`	Number of GPUs to allocate in the cluster.
`--region [string]`	Region to create the cluster in. Valid regions can be found with `tg beta clusters list-regions`.
`--billing-type [ON_DEMAND\|RESERVED]`	Cluster reservation approach. `ON_DEMAND` begins billing the moment the cluster is created. Billing continues until you delete the cluster. `RESERVED` starts billing immediately. The cluster is automatically torn down after the `--duration-days` length elapses.
`--nvidia-driver-version [string]`	NVIDIA driver version. Valid versions can be found with `tg beta clusters list-regions`.
`--cuda-version [string]`	CUDA version. Valid versions can be found with `tg beta clusters list-regions`.
`--duration-days [number]`	Only used with `RESERVED` billing. Specifies how many days the cluster is reserved for.
`--gpu-type [string]`	GPU type to use for the cluster. One of `H100_SXM`, `H200_SXM`, `RTX_6000_PCI`, `L40_PCIE`, `B200_SXM`, `H100_SXM_INF`. Available types vary by region; see `tg beta clusters list-regions`.
`--cluster-type [KUBERNETES\|SLURM]`	Cluster workload manager or orchestrator.
`--volume [string]`	Storage volume ID to attach to the cluster. List existing volumes with `tg beta clusters storage list`.

Run tg beta clusters create with no flags to launch an interactive prompt that walks through the required fields. Pass --non-interactive (or --json) to skip prompts in CI.

Update a cluster

tg beta clusters update [CLUSTER_ID]

Parameters

Flag	Description
`--num-gpus [integer]`	Number of GPUs to allocate in the cluster.
`--cluster-type [KUBERNETES\|SLURM]`	Cluster workload manager or orchestrator.

Retrieve a cluster

tg beta clusters retrieve [CLUSTER_ID]

Delete a cluster

tg beta clusters delete [CLUSTER_ID]

List clusters

tg beta clusters list

List regions

Get configuration information per region for creating a GPU cluster.

tg beta clusters list-regions

Example output

{
  "regions": [
    {
      "driver_versions": [
        {
          "cuda_version": "12.9",
          "nvidia_driver_version": "575"
        },
        {
          "cuda_version": "12.8",
          "nvidia_driver_version": "570"
        },
        {
          "cuda_version": "13.1",
          "nvidia_driver_version": "590"
        },
        {
          "cuda_version": "13.1",
          "nvidia_driver_version": "580"
        }
      ],
      "name": "us-central-8",
      "supported_instance_types": [
        "H100_SXM",
        "H200_SXM"
      ]
    }
  ]
}

SSH into a cluster

SSH into a Slurm cluster using a short-lived OIDC-signed certificate. The command opens your browser to sign in, requests a certificate from the cluster’s certificate authority, and connects through the bastion host. No API key or long-lived SSH key is required.

Requires Together CLI 2.20+ and Python 3.10+. Check your version with tg --version. To install or upgrade, see Get started.

tg beta clusters ssh https://dex.<REGION>.cloud.together.ai/<CLUSTER_ID> --login <LOGIN>

Copy the Dex issuer URL and login name from the cluster UI when OIDC is selected as the SSH access method.

Parameters

Flag	Description
`DEX_URL` (positional)	Cluster Dex issuer URL: `https://dex.<BASE>/<CLUSTER_ID>`. required
`--login`, `-l`	POSIX login or SSH username on the cluster. required
`--host`	Target host reachable through the bastion. Default: `slurm-login`. For compute nodes, pass the node hostname shown in the UI (for example, `worker-1.slurm-compute.slurm`).
`--client-id`	Dex public client ID. Default: `together-cli`.
`--scope`	OIDC scopes. Default: `openid email`.
`--key-type`	Ephemeral key type (`ecdsa` or `ed25519`). Default: `ecdsa`.
`--ca-root`	step-ca root certificate (PEM) for TLS verification.
`--cache`	Cache SSH keys and certificates while valid. Default: `true`.
`--refresh`	Force refresh the cached SSH certificate.
`--cache-dir`	Directory for cached SSH keys and certificates.
`--print-ssh-command`	Print the underlying `ssh` command instead of executing it.
`--ssh-config-alias`	Print an `ssh_config` Host entry for this alias instead of executing `ssh`.
`--write-ssh-config`	Write or update the alias in `~/.together/ssh/config` and include it from `~/.ssh/config`. Requires `--ssh-config-alias`.

Any arguments after DEX_URL are passed through to ssh as a remote command.

Get cluster credentials

Download the cluster’s configuration and credentials to your local .kube/config file to manage Kubernetes resources.

tg beta clusters get-credentials [CLUSTER_ID]

Parameters

Flag	Description
`--file [Path\|-]`	Override the path to write the kubeconfig to. Pass `-` to print the config to stdout instead of writing to a file. Default: `~/.kube/config`.
`--context-name [string]`	Name of the context to add to the kubeconfig. Defaults to the cluster name.
`--overwrite-existing`	If there is a conflict with the existing kubeconfig, overwrite it instead of raising an error.
`--set-default-context`	Change the current context for `kubectl` to the new context.

Create cluster storage

tg beta clusters storage create

Parameters

Flag	Description
`--region [string]`	Region to create the storage volume in. required
`--size-tib [integer]`	Size of the storage volume in TiB. required
`--volume-name [string]`	Name for the storage volume. required

Retrieve cluster storage

tg beta clusters storage retrieve [VOLUME_ID]

List cluster storage

tg beta clusters storage list

Delete cluster storage

tg beta clusters storage delete [VOLUME_ID]

TOGETHER CLI

COMMANDS

Create a cluster

Parameters

Update a cluster

Parameters

Retrieve a cluster

Delete a cluster

List clusters

List regions

Example output

SSH into a cluster

Parameters

Get cluster credentials

Parameters

Create cluster storage

Parameters

Retrieve cluster storage

List cluster storage

Delete cluster storage

​Create a cluster

​Parameters

​Update a cluster

​Parameters

​Retrieve a cluster

​Delete a cluster

​List clusters

​List regions

​Example output

​SSH into a cluster

​Parameters

​Get cluster credentials

​Parameters

​Create cluster storage

​Parameters

​Retrieve cluster storage

​List cluster storage

​Delete cluster storage

Create a cluster

Parameters

Update a cluster

Parameters

Retrieve a cluster

Delete a cluster

List clusters

List regions

Example output

SSH into a cluster

Parameters

Get cluster credentials

Parameters

Create cluster storage

Parameters

Retrieve cluster storage

List cluster storage

Delete cluster storage