Overview

Together AI post-training lets you define a custom training loop for fine-tuning models on remote GPUs. You provision a model resource, attach a training session to it, and call the service at each step to sample completions, compute a loss, and update the model weights. When the training session ends, you can download the checkpoint to your machine, fork it to a new training session, or deploy it to a dedicated endpoint for production inference.

The post-training workflow. Load a model on GPUs, attach a training session, run your training loop, and save a checkpoint. From a checkpoint, you can fork or resume a new training session, download the weights to your machine, or deploy them to a dedicated endpoint.

Post-training supports several training methods, including:

Key features

Train on outcomes: With reinforcement learning, your code scores the model’s outputs, so you can train on any task where you can check the answer, even if you can’t provide training examples.
Train agents: Fine-tune models in multi-turn environments, where the model acts over several turns and can call tools.
Control every step: Your code chooses each batch, learning rate, and loss function. For any training method, you can write your own loss function that computes the loss in PyTorch from the model’s log-probabilities.
No infrastructure management: Your loop runs on any machine, even a laptop, while Together AI runs the sharded trainer, the sampling engines, and the weight sync between them. See Concepts for more details.
Iterate cheaply, then deploy: Train several adapters on one base model for the GPU cost of one, then deploy the best to dedicated inference.

Get started

SFT quickstart

Run your first supervised fine-tuning session from a cookbook recipe.

RL quickstart

Train a model on grade-school math with GRPO and an exact-match reward.

Supported models

Pick a base model by training mode, context length, and GPU type.

How it works

The main functionality for post-training is contained in just a few SDK methods. First, you create a model resource and attach a training session to it:

create: Provision a model resource, which represents a set of GPUs with the base model loaded on them. This is what you’re billed for.
create_session: Attach a training session to the model resource.

Then, at each step of the training loop, you call these methods:

sample_batch: Generate completions for your code to score.
forward_backward: Compute the loss and backpropagate the gradients.
optim_step: Update the model weights.
weights_sync: Publish the new weights to the generator replicas, so the next step samples from them.

SFT and DPO don’t sample completions, so they skip sample_batch and weights_sync.

Example

Here’s a minimal example of supervised fine-tuning with the Python SDK:

from transformers import AutoTokenizer
from together.post_training import ModelResourcesClient

MODEL = "Qwen/Qwen3.8-27B"

# Build a training sample
tokens = AutoTokenizer.from_pretrained(MODEL).encode("Q: What is 2 + 2?\nA: 4")

sample = {
    "model_input": {"chunks": [{"encoded_text": {"tokens": tokens[:-1]}}]},
    "loss_fn_inputs": {
        "target_tokens": tokens[1:],
        "weights": [1.0] * (len(tokens) - 1),
    },
}

# Provision a model resource
with ModelResourcesClient.create(
    base_model=MODEL,
    compute_config={"gpu_type": "B200-SXM", "num_generator_replicas": 0},
) as resources:

    # Start a training session
    with resources.create_session() as session:

        # Your machine runs the training loop
        # session.trainer calls run on Together GPUs
        for step in range(10):
            result = session.trainer.forward_backward(
                samples=[sample],
                loss={"type": "LOSS_TYPE_CROSS_ENTROPY"},
            )

            # Update the adapter's weights
            session.trainer.optim_step(adam_params={"learning_rate": 1e-4})

            # The loss falls toward 0 as the model memorizes the example.
            print(step, result.loss)

        # Save the adapter as an inference checkpoint you can deploy
        checkpoint = session.create_inference_checkpoint()
        print(checkpoint.id)

When you run the example:

Together AI provisions GPUs and loads the base model onto them.
Your code runs the training loop on your machine. At each step, it sends a batch to the GPUs, which compute the loss, update the model’s weights, and return the loss to your code.
When the program exits, Together AI releases the GPUs and billing stops.

Reinforcement learning works the same way, except each step starts with the GPUs generating completions for your code to score. The scored completions become the training batch. For more on how this works, see Concepts.

Cookbook

Instead of writing a training loop from scratch, you can use a recipe from the post-training cookbook, a Python library of ready-made loops built on the Together SDK. Each recipe handles dataset loading, tokenization, reward scoring, and the training loop itself, so you only need to work with the SDK directly when no recipe fits your task. The cookbook also includes notebooks. The notebooks/ directory walks step by step through GRPO, SFT, and training several low-rank adapters (LoRAs) on one model resource, using the SDK’s async calls. To add a notebook kernel, run uv sync --locked --group notebooks in your checkout.

Post-training vs. standard fine-tuning

Together AI also offers a job-based fine-tuning service for standard workloads. If you only need SFT or DPO and have well-formed data, it’s much simpler to use: upload a dataset, submit a job, and wait for the model to train. Post-training is more powerful but more complex. It uses a session-based workflow in which your own code runs the training loop against remote GPUs. The table summarizes the main differences:

	Standard fine-tuning	Post-training
Where it runs	Together AI runs the whole loop: tokenization, batching, loss, learning-rate schedule, checkpoints.	Your code drives the training loop, and remote GPUs run the heavy steps: forward and backward passes, optimizer steps, and sampling.
Billing	Per token processed.	Per GPU-minute.
Data	JSONL files in fixed formats, validated at upload, or pre-tokenized Parquet files.	Token sequences and loss masks you build, in any shape, changeable mid-run.
Mid-run control	None. The job runs to completion unless you cancel it or configure early stopping.	Full. Inspect per-step metrics, change the learning rate, reshuffle data, or stop on your own logic.
Loss functions	Built-in SFT and DPO objectives.	Built-in cross-entropy and policy-gradient loss functions, plus custom loss functions that run in your code.
Reinforcement learning	Not available.	GRPO and related policy-gradient methods, for both single-turn and multi-turn tasks.
Use when	You want standard SFT or DPO on well-formed data, with minimal code.	You need step-level control, custom loss functions or curricula, or reinforcement learning.

Train, evaluate, and productionize adapters

The three stages of a post-training project. Train: one model resource trains up to five adapters, A, B, and C, while you pay for only one base model. Evaluate: deploy the adapters to a dedicated endpoint and compare them on live traffic, where adapter B wins. Productionize: merge and quantize the winner, B, then scale it across several replicas for production inference.

Each training session produces a single adapter, but a training project usually produces several. Post-training lets you train several adapters on one base model, compare them, and promote the winner:

Train: Attach several training sessions to one model resource. Each session trains an independent adapter against the same frozen base model, so a hyperparameter sweep costs one base model’s worth of GPU time, not one per adapter.
Evaluate: Each adapter is registered as a model when its session saves an inference checkpoint. Attach the candidates to one deployment of the base model on dedicated GPUs, then compare them by naming a different adapter in each request. Multi-LoRA deployments support only some base models.
Productionize: Deploy the winner on its own deployment, which merges it into its base model when it starts. For GLM-5.3 and Kimi-K3, first prepare the adapter for FP4 inference.

Learn more

Concepts

How work splits between your code and Together AI GPUs, and the model resources, sessions, and checkpoints you work with.

Training methods

Choose a method and a training mode, compare built-in and custom loss functions, and combine methods.

Model resources

Size, create, attach to, and stop the GPUs your sessions train on.

Pricing

Per GPU-minute billing, cost estimates, and refunds on failure.

Manage checkpoints

Save, download, and resume from what you trained.

Deploy a checkpoint

Serve a trained model on dedicated model inference.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Key features

Get started

SFT quickstart

RL quickstart

Supported models

How it works

Example

Cookbook

Post-training vs. standard fine-tuning

Train, evaluate, and productionize adapters

Learn more

Concepts

Training methods

Model resources

Pricing

Manage checkpoints

Deploy a checkpoint

​Key features

​Get started

SFT quickstart

RL quickstart

Supported models

​How it works

​Example

​Cookbook

​Post-training vs. standard fine-tuning

​Train, evaluate, and productionize adapters

​Learn more

Concepts

Training methods

Model resources

Pricing

Manage checkpoints

Deploy a checkpoint

Key features

Get started

How it works

Example

Cookbook

Post-training vs. standard fine-tuning

Train, evaluate, and productionize adapters

Learn more