Skip to main content
Together AI post-training lets you fine-tune models on Together AI GPUs with your own training loop. You provision a model resource, attach a training session to it, and call the service at each step to sample completions, compute a loss, and update the model weights. You can write the loop yourself with the Together Python SDK, or start from a ready-made recipe in the post-training cookbook. From the checkpoints saved by a session, you can download the trained weights to your machine, fork the run into a new training session, or deploy the model to a dedicated endpoint for production inference.
The post-training workflow. Load a model on GPUs, attach a training session, run your training loop, and save a checkpoint. From a checkpoint, you can fork or resume a new training session, download the weights to your machine, or deploy them to a dedicated endpoint.
Post-training supports these training methods: If you only need SFT or DPO, you can alternatively use standard fine-tuning, which runs the whole training loop for you.

Key features

  • Train on outcomes: With reinforcement learning, your code scores the model’s outputs, so you can train on any task where you can check the answer, even if you can’t provide training examples.
  • Train agents: Fine-tune models in multi-turn environments, where the model acts over several turns and can call tools.
  • Control every step: Your code chooses each batch, learning rate, and loss function. For any training method, you can write your own loss function that computes the loss in PyTorch from the model’s log-probabilities.
  • No infrastructure management: Your loop runs on your laptop, while Together AI runs the sharded trainer, the generator replicas, and the weight sync between them. See Concepts for more details.
  • Iterate cheaply, then deploy: Train several low-rank adapters (LoRAs) on one base model while paying for only that base model’s GPU time, then deploy the best one for dedicated model inference.

Get started

SFT quickstart

Run a cookbook SFT recipe that trains a model on instruction-following examples.

GRPO quickstart

Run a cookbook GRPO recipe that trains a model on grade-school math with an exact-match reward.

Write a training loop

Run a minimal training loop with the Python SDK, then add batching and sampling.

Supported models

Pick a base model by training mode, context length, and GPU type.

Migrate from Tinker

Run a Tinker training script by changing its import line.

Cookbook recipes vs. writing your own loop

There are two main ways to use post-training, both of which use the same underlying APIs and infrastructure:
  • Run a cookbook recipe: Choose this if a ready-made training loop fits your task. The post-training cookbook is a Python library of recipes that handle dataset loading, tokenization, the loop itself, and, for GRPO, reward scoring. You can tune a run by editing its config.
  • Write your own loop with the SDK: Choose this if you need more control over data, sampling, rewards, or the loss function.

Post-training vs. standard fine-tuning

Together AI also offers a job-based fine-tuning service for standard workloads. If you only need SFT or DPO, and your data is well formed, it’s much simpler to use: just upload a dataset, submit a job, and wait for the model to train. Post-training is more powerful but more complex. It uses a session-based workflow in which your own code runs the training loop against remote GPUs. The table summarizes the main differences:

Train, evaluate, and productionize adapters

A training project usually produces several candidate adapters. Post-training lets you train them on one base model, compare them, and promote the winner:
The three stages of a post-training project. Train: one model resource trains adapters A, B, and C (up to five per resource) while you pay for only one base model. Evaluate: deploy the adapters to a dedicated endpoint and compare them on live traffic, where adapter B wins. Productionize: merge and quantize the winner, B, then scale it across several replicas for production inference.
  1. Train: Attach several training sessions to one model resource. Each session trains an independent adapter against the same frozen base model.
  2. Evaluate: If the base model supports multi-LoRA deployment, attach the candidates to one deployment of that model on dedicated GPUs, then compare them by naming a different adapter in each request. Each candidate is ready to attach once its session saves an inference checkpoint, which registers the adapter as a model.
  3. Productionize: Deploy the winner on its own deployment, where each replica merges the adapter into the base model’s weights when it starts. If the base model is served only in FP4, prepare the adapter for FP4 inference instead, then deploy the prepared model.

Learn more

Concepts

Learn how work splits between your code and Together AI GPUs, and what model resources, sessions, and checkpoints do.

Choose a training method

Compare SFT, DPO, and GRPO by the training data each one needs, and combine methods.

Model resources

Size, create, attach to, and stop the GPUs your sessions train on.

Pricing

Learn how per-GPU-minute billing and refunds on failure work, and estimate what a run costs.

Manage checkpoints

Save, download, and resume from what you trained.

Deploy a checkpoint

Serve a trained model on dedicated model inference.