Post-training cookbook overview

The post-training cookbook is a public Python library of ready-made training recipes built on the Together Python SDK, covering supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). Each recipe is a complete training loop that handles dataset loading, tokenization, validation, checkpoints, and logging, plus reward scoring for GRPO, so you start a run by editing a config instead of writing a loop. The cookbook’s chat-template and sample helpers also work on their own, inside a loop you write with the SDK.

How the cookbook relates to the SDK

Post-training code works at three levels: recipes, helpers, and the SDK. The cookbook supplies the recipes and helpers, and both call the SDK to train:

Level	Modules	What it does
Recipes	`post_training_cookbook.sft`, `.dpo`, `.grpo`, `.datasets`, `.envs`	Complete training loops configured by `SFTConfig`, `DPOConfig`, and `GRPOConfig`.
Helpers	`post_training_cookbook.codecs`, `.samples`, `.grpo.samples`, `.dpo.samples`, `.custom_loss`	Chat-template rendering, sample builders, group advantages, and the passes behind a custom loss function.
SDK	`together.post_training`	Model resources, sessions, training operations, and checkpoints.

Some features exist only in the cookbook’s recipes, not in the post-training service, so a loop you write with the SDK has them only if you implement them:

Dynamic sampling, overlong handling, and length budgets.
Async rollout collection.
The dataset registry and its built-in rewards.
The Env abstraction and the multi-turn episode loop.
Run directories and Weights & Biases (W&B) logging.
Continuing a run where its data and learning-rate schedule stopped.

Install

Clone to run recipes

Recipes and notebooks run from a checkout of the repository. Clone it and install its dependencies, including the Together Python SDK, with uv:

git clone https://github.com/togethercomputer/post-training-cookbook.git
cd post-training-cookbook
uv sync --locked

The notebooks in notebooks/ walk step by step through GRPO, SFT, and training several low-rank adapters (LoRAs) on one model resource, using the SDK’s async calls. To add a notebook kernel, run uv sync --locked --group notebooks in your checkout.

Install as a library

To import the helpers into your own project, install the cookbook from GitHub, since it isn’t published to PyPI. It requires Python 3.11 or later and pulls in PyTorch, Transformers, and version 3 of the Together Python SDK as dependencies:

uv add "post-training-cookbook @ git+https://github.com/togethercomputer/post-training-cookbook"

Import the cookbook as post_training_cookbook.

Recipes

Run a recipe with uv run python <path> from a cookbook checkout. Each recipe is a single script with its config at the top. To change a run, edit the config. To start a new recipe, copy an existing one.

Recipe	What it trains	Covered in
`recipes/sft/no_robots.py`	SFT on No Robots instruction-following conversations.	SFT quickstart
`recipes/sft/no_robots_full.py`	The same SFT run with full-weight training instead of a LoRA.	What changes with full-weight training
`recipes/dpo/helpsteer3.py`	DPO on HelpSteer3 preference pairs.	DPO recipes
`recipes/sft_then_grpo.py`	SFT on GSM8K to teach the answer format, then GRPO from the same adapter with a fresh optimizer.	Chain SFT and GRPO
`recipes/grpo/gsm8k.py`	Single-turn GRPO on GSM8K grade-school math.	GRPO quickstart
`recipes/grpo/gsm8k_muon.py`	GRPO on GSM8K with Qwen3.8 27B, the Muon optimizer, and full-weight training.	Optimizer
`recipes/grpo/deepmath.py`	Single-turn GRPO on DeepMath competition math, with a 16K-token budget.	Built-in datasets
`recipes/grpo/deepmath-lv5.py`	Single-turn GRPO on the DeepMath problems at difficulty 5 and below, with Qwen3.8 27B and thinking on.	Built-in datasets
`recipes/grpo/game24.py`	Single-turn GRPO on Game of 24 arithmetic puzzles, training on a partial-credit reward while validation reports strict accuracy.	Report a separate metric
`recipes/grpo/tool_use_game.py`	Multi-turn GRPO on the bundled tool-use environment.	Run multi-turn training
`recipes/grpo/tool_use_game_async.py`	The tool-use run with async collection and two generator replicas.	Async training
`recipes/grpo/deepmath_sweep.py`	A four-arm learning-rate sweep on DeepMath, each arm a session on one shared model resource.	Run a sweep
`recipes/grpo/deepmath_shared.py`	DeepMath GRPO on a model resource that’s already running. Set `MODEL_RESOURCES_ID` first.	Attach to a running resource
`recipes/serverless_eval/*.py`	No training. Scores a hosted model on GSM8K, DeepMath, Game of 24, or the tool-use environment.	Check a baseline before training

Run outputs

Run artifacts

Every cookbook recipe writes a record of the run to a run directory, runs/<run_name>/:

runs/<run_name>/
  config.yaml        the config, round-trippable through <Config>.from_yaml
  manifest.json      W&B URL, session ID, model, start time, git SHA
  metrics.jsonl      one JSON line per step, mirroring what goes to W&B
  checkpoints.jsonl  training and inference checkpoint IDs
  step_NNNN_*.json   per-step completions or episodes, when export_samples=True
  val_NNNN_*.json    validation completions, when export_samples=True

The cookbook always writes metrics.jsonl, so you can analyze a run offline without W&B. To see what the model is producing, read the per-step step_NNNN_*.json files.

Weights & Biases

When a config sets wandb_project, the cookbook logs every step to that project. GRPOConfig defaults it to post-training-cookbook. SFTConfig and DPOConfig leave it unset.

Set WANDB_API_KEY before you launch a run that logs to W&B. The cookbook calls wandb.init only after it provisions the model resource, so a missing key fails the run after provisioning. In an interactive terminal, wandb.init waits at a login prompt instead, and the resource bills while it waits.

To run without a W&B account, set WANDB_MODE=offline to keep the W&B logs on disk, or set wandb_project=None to turn W&B off. To compare related runs side by side, such as the arms of a sweep, give them the same wandb_group. The cookbook forwards wandb_project, wandb_entity, wandb_group, and wandb_run_name to the session as metadata, and the console shows them on the session’s page. It doesn’t send the run’s ID or URL, so the page doesn’t link to the run. To open the run, use the W&B URL in manifest.json.

Next steps

Run GRPO recipes

Train a GRPO recipe on a built-in dataset or your own.

Run multi-turn GRPO recipes

Train a tool-use agent on an environment you write.

Monitor training runs

Watch a run in the console and read the metrics each recipe logs.

Cookbook config reference

Look up every config field and its default.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Post-training cookbook overview

How the cookbook relates to the SDK

Install

Clone to run recipes

Install as a library

Recipes

Run outputs

Run artifacts

Weights & Biases

Next steps

Run GRPO recipes

Run multi-turn GRPO recipes

Monitor training runs

Cookbook config reference

​How the cookbook relates to the SDK

​Install

​Clone to run recipes

​Install as a library

​Recipes

​Run outputs

​Run artifacts

​Weights & Biases

​Next steps

Run GRPO recipes

Run multi-turn GRPO recipes

Monitor training runs

Cookbook config reference

How the cookbook relates to the SDK

Install

Clone to run recipes

Install as a library

Recipes

Run outputs

Run artifacts

Weights & Biases

Next steps