GPU clusters quickstart

Using a coding agent? Install the together-ai skill to let your agent write correct GPU cluster code automatically. See agent skills for details.

A GPU cluster is a set of GPU nodes dedicated to you, where you run workloads with Kubernetes or Slurm. Use one when you need full control over your training or inference stack, such as for multi-node training or for custom code that doesn’t fit a managed service like fine-tuning or dedicated model inference. In this quickstart, you’ll create a GPU cluster, connect to it with kubectl or SSH, run a test job that confirms its GPUs are ready, and delete the cluster when you’re done.

Requirements

Before you begin, make sure you have:

A Together AI account with sufficient credits.
The Together CLI installed. To install or upgrade it, run:
uv tool install "together[cli]" --upgrade
kubectl installed, if you plan to create a Kubernetes cluster.
An SSH key added at api.together.ai/settings/ssh-key before you create the cluster, if you plan to use key-based SSH on a Slurm cluster.

Step 1: Create a cluster

Console
CLI

Open the create form

Go to the GPU clusters page in the Together AI console and select Create cluster.

Choose the capacity type

Select On-demand to pay hourly with no commitment. To pay a fixed rate for 3 to 180 days instead, select Reserved.

Choose GPUs

Select a GPU type card (for example, H100-SXM), then set the number of GPUs under Select GPU Count. GPUs come in whole nodes of 8. For a reserved cluster, also set Duration below the GPU count.

Choose a region

Leave Region at its default, Any region, to target the region with the most available capacity for your selected GPU type. If you need the cluster in a particular location, select that region instead.

Configure the cluster

Under Cluster Details, enter a descriptive Cluster name.Under Orchestration, select Kubernetes for containerized workloads or Slurm for high-performance computing (HPC) batch scheduling.Leave Driver version at its default unless you need a specific NVIDIA driver and CUDA combination.

Create a shared volume

Under Storage, open Storage Selection and select + Create new storage, then set Size (TiB) and a Storage Name. The minimum size is 1 TiB, and you can resize the volume later. To reuse a volume you already have, select it instead.

Changing the GPU type resets the region to Any region and deselects any existing volume you chose, because volumes are region-specific. Select them again after you switch GPU types.

Create the cluster

Select Create Cluster, review the details, then select Confirm & Create. The console shows the cluster’s status as it provisions. Wait for the status to reach Ready, which usually takes 10 minutes to an hour, depending on cluster size and region.

List the regions to find one that supports your GPU type. The output also shows each region’s NVIDIA driver and CUDA versions, which you need when you create the cluster:

tg clusters list-regions

Create a shared storage volume in that region, and note the volume ID in the output. This example uses us-central-8:

tg clusters storage create \
  --volume-name my-training-data \
  --size-tib 1 \
  --region us-central-8

Create the cluster in the same region with the volume attached. With --non-interactive, you must choose an NVIDIA driver, so copy a driver version’s id from the list-regions output and pass it as --nvidia-version-id:

tg clusters create \
  --name my-cluster \
  --cluster-type KUBERNETES \
  --gpu-type H100_SXM \
  --num-gpus 8 \
  --region us-central-8 \
  --nvidia-version-id <NVIDIA_VERSION_ID> \
  --billing-type ON_DEMAND \
  --volume <VOLUME_ID> \
  --non-interactive

To be prompted for each required field instead, run tg clusters create with no flags. For all flags, including reserved billing with --billing-type RESERVED --duration-days, see the CLI reference.

A volume you create on its own goes to the region’s datacenter with the most free GPUs, which might not host your GPU type. If it doesn’t, cluster creation fails with Shared volume you selected does not exist in the datacenter in region <REGION> providing <GPU_TYPE>. To fix this, create the volume with the cluster instead:

Run tg clusters create without --volume and --non-interactive, and answer y to “Create a new storage volume?”. You can also create the cluster in the console.
Delete the unused volume with tg clusters storage delete <VOLUME_ID>.

To check provisioning progress, list your clusters. The output shows each cluster’s ID and status:

tg clusters list

Wait for the status to reach Ready. Provisioning usually takes 10 minutes to an hour, depending on cluster size and region.

If the GPU type you want has no available capacity, try a different region, or request capacity by selecting Need a different configuration? in the console create form.

Step 2: Connect to the cluster

How you connect depends on the cluster type you chose in Step 1:

Kubernetes
Slurm

Find your cluster ID, then download the cluster’s credentials. get-credentials merges them into your local ~/.kube/config (pass --file to write them elsewhere), and --set-default-context points kubectl at the new cluster:

tg clusters list
tg clusters get-credentials <CLUSTER_ID> --set-default-context

Only project admins can download credentials. For anyone else, get-credentials reports that no kubeconfig was found. Editors connect with an OpenID Connect (OIDC) kubeconfig that an admin shares instead.When the CLI detects an AI agent, it defaults to JSON output. In JSON mode, get-credentials reports the target path without writing the file, so pass --no-json to write it.Confirm that kubectl can reach the cluster:

kubectl get nodes

You should see the cluster’s nodes listed with status Ready. Nodes appear as they finish provisioning, so if any are missing, wait for the cluster status to reach Ready in the console.You can also view or download the kubeconfig from the cluster details page in the console.

Open the cluster’s details page in the console. If your cluster supports OpenID Connect (OIDC) SSH, the page shows an SSH access method selector. Choose OIDC or Key-based:

OIDC: Connects through the Together CLI with a short-lived certificate. Choose a login name when prompted. No SSH key is required.
Key-based: Uses the SSH key you added before you created the cluster.

Select Copy head node SSH command in the cluster sidebar. The copied command matches your selected access method and fills in <LOCATION>, the datacenter part of the node hostnames:

# OIDC
tg clusters ssh https://dex.<LOCATION>.cloud.together.ai/t-<CLUSTER_ID> --login <LOGIN>

# Key-based
ssh -J <LOGIN>@ssh.t-<CLUSTER_ID>.<LOCATION>.cloud.together.ai <LOGIN>@slurm-login

The console copies the OIDC command as tg beta clusters ssh. Together CLI 3.0 removed the beta path, so delete beta from the copied command.Run the command in your terminal. It opens a shell on the cluster’s login node, where you run Slurm commands.Verify that Slurm sees your nodes:

sinfo          # View node and partition status
squeue         # View the job queue

Slurm clusters run on Kubernetes. If you also need kubectl access to the underlying cluster API, a project admin can download the kubeconfig from the cluster details page or with tg clusters get-credentials. SSH to the login node is still the primary way to submit jobs.

Step 3: Run a test job

To confirm the cluster is ready for work, run a job that prints a node’s GPU status.

Kubernetes
Slurm

Save this pod definition to a file named gpu-test.yaml on your local machine, in the directory you run kubectl from. A pod is the unit of work Kubernetes schedules onto a node. This pod requests one GPU and runs nvidia-smi:

gpu-test.yaml

apiVersion: v1
kind: Pod
metadata:
  name: gpu-test
spec:
  restartPolicy: Never
  containers:
    - name: gpu-test
      image: nvidia/cuda:12.4.1-base-ubuntu22.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1

Create the pod, wait for it to finish, then print its output:

kubectl apply -f gpu-test.yaml
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/gpu-test --timeout=180s
kubectl logs gpu-test

In the login node shell you opened in Step 2, start an interactive session. The srun command tells Slurm to allocate one GPU on a compute node and opens a shell there. Run nvidia-smi in that shell to confirm the GPU is ready:

srun --gpus=1 --pty bash
nvidia-smi

You should see the nvidia-smi table listing the allocated GPU, its driver version, and memory.

Congrats! You created a GPU cluster and ran your first job on it.

Clean up to stop billing

On-demand clusters bill hourly until you delete them. When you’re done, delete the cluster:

tg clusters delete <CLUSTER_ID>

Your shared storage volume persists after cluster deletion and continues to bill until you delete it. Keep it to reuse your data on a future cluster, or delete it:

tg clusters storage delete <VOLUME_ID>

Next steps

Run workloads on Kubernetes

Deploy pods with persistent storage and run GPU workloads.

Submit Slurm jobs

Run and manage batch jobs with the preconfigured Slurm scheduler.

Cluster storage

Choose the right storage tier for datasets and checkpoints.

Billing and pricing

Understand how compute and storage charges work.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Requirements

Step 1: Create a cluster

Step 2: Connect to the cluster

Step 3: Run a test job

Clean up to stop billing

Next steps

Run workloads on Kubernetes

Submit Slurm jobs

Cluster storage

Billing and pricing