> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Post-training cookbook overview

> Run the post-training cookbook's ready-made recipes, or reuse its chat-template and sample helpers in your own training loop.

The [post-training cookbook](https://github.com/togethercomputer/post-training-cookbook) is a public Python library of ready-made training recipes built on the Together Python SDK, covering supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). Each recipe is a complete training loop that handles dataset loading, tokenization, validation, checkpoints, and logging, plus reward scoring for GRPO, so you start a run by editing a config instead of writing a loop. The cookbook's chat-template and sample helpers also work on their own, inside a loop you [write with the SDK](https://docs.together.ai/docs/post-training/training-loop.md).

## How the cookbook relates to the SDK

Post-training code works at three levels: recipes, helpers, and the SDK. The cookbook supplies the recipes and helpers, and both call the SDK to train:

| Level | Modules | What it does |
| - | - | - |
| Recipes | `post_training_cookbook.sft`, `.dpo`, `.grpo`, `.datasets`, `.envs` | Complete training loops configured by `SFTConfig`, `DPOConfig`, and `GRPOConfig`. |
| Helpers | `post_training_cookbook.codecs`, `.samples`, `.grpo.samples`, `.dpo.samples`, `.custom_loss` | Chat-template rendering, sample builders, group advantages, and the passes behind a custom loss function. |
| SDK | `together.post_training` | Model resources, sessions, training operations, and checkpoints. |

Some features exist only in the cookbook's recipes, not in the post-training service, so a loop you write with the SDK has them only if you implement them:

* [Dynamic sampling, overlong handling, and length budgets](https://docs.together.ai/docs/post-training/grpo-recipes.md#tune-a-grpo-recipe).
* [Async rollout collection](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training).
* [The dataset registry and its built-in rewards](https://docs.together.ai/docs/post-training/grpo-recipes.md#built-in-datasets).
* [The `Env` abstraction and the multi-turn episode loop](https://docs.together.ai/docs/post-training/multi-turn-recipes#write-an-environment).
* [Run directories and Weights & Biases (W\&B) logging](#run-outputs).
* [Continuing a run](https://docs.together.ai/docs/post-training/grpo-recipes.md#continue-a-run) where its data and learning-rate schedule stopped.

## Install

### Clone to run recipes

Recipes and notebooks run from a checkout of the repository. Clone it and install its dependencies, including the Together Python SDK, with [uv](https://docs.astral.sh/uv/):

```bash theme={null}
git clone https://github.com/togethercomputer/post-training-cookbook.git
cd post-training-cookbook
uv sync --locked
```

The notebooks in `notebooks/` walk step by step through GRPO, SFT, and training several low-rank adapters (LoRAs) on one model resource, using the SDK's async calls. To add a notebook kernel, run `uv sync --locked --group notebooks` in your checkout.

### Install as a library

To import the helpers into your own project, install the cookbook from GitHub, since it isn't published to PyPI. It requires Python 3.11 or later and pulls in PyTorch, Transformers, and version 3 of the Together Python SDK as dependencies:

```bash theme={null}
uv add "post-training-cookbook @ git+https://github.com/togethercomputer/post-training-cookbook"
```

Import the cookbook as `post_training_cookbook`.

## Recipes

Run a recipe with `uv run python <path>` from a [cookbook checkout](#clone-to-run-recipes). Each recipe is a single script with its config at the top. To change a run, edit the config. To start a new recipe, copy an existing one.

| Recipe | What it trains | Covered in |
| - | - | - |
| `recipes/sft/no_robots.py` | SFT on No Robots instruction-following conversations. | [SFT quickstart](https://docs.together.ai/docs/post-training/quickstart-sft.md) |
| `recipes/sft/no_robots_full.py` | The same SFT run with full-weight training instead of a LoRA. | [What changes with full-weight training](https://docs.together.ai/docs/post-training/lora-vs-full-weight.md#what-changes-with-full-weight-training) |
| `recipes/dpo/helpsteer3.py` | DPO on HelpSteer3 preference pairs. | [DPO recipes](https://docs.together.ai/docs/post-training/dpo-recipes.md) |
| `recipes/sft_then_grpo.py` | SFT on GSM8K to teach the answer format, then GRPO from the same adapter with a fresh optimizer. | [Chain SFT and GRPO](https://docs.together.ai/docs/post-training/sft-recipes.md#chain-sft-and-grpo) |
| `recipes/grpo/gsm8k.py` | Single-turn GRPO on GSM8K grade-school math. | [GRPO quickstart](https://docs.together.ai/docs/post-training/quickstart-rl.md) |
| `recipes/grpo/gsm8k_muon.py` | GRPO on GSM8K with Qwen3.8 27B, the Muon optimizer, and full-weight training. | [Optimizer](https://docs.together.ai/docs/post-training/cookbook.md#optimizer) |
| `recipes/grpo/deepmath.py` | Single-turn GRPO on DeepMath competition math, with a 16K-token budget. | [Built-in datasets](https://docs.together.ai/docs/post-training/grpo-recipes.md#built-in-datasets) |
| `recipes/grpo/deepmath-lv5.py` | Single-turn GRPO on the DeepMath problems at difficulty 5 and below, with Qwen3.8 27B and thinking on. | [Built-in datasets](https://docs.together.ai/docs/post-training/grpo-recipes.md#built-in-datasets) |
| `recipes/grpo/game24.py` | Single-turn GRPO on Game of 24 arithmetic puzzles, training on a partial-credit reward while validation reports strict accuracy. | [Report a separate metric](https://docs.together.ai/docs/post-training/grpo-recipes.md#report-a-separate-metric) |
| `recipes/grpo/tool_use_game.py` | Multi-turn GRPO on the bundled tool-use environment. | [Run multi-turn training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#run-multi-turn-training) |
| `recipes/grpo/tool_use_game_async.py` | The tool-use run with async collection and two generator replicas. | [Async training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training) |
| `recipes/grpo/deepmath_sweep.py` | A four-arm learning-rate sweep on DeepMath, each arm a session on one shared model resource. | [Run a sweep](https://docs.together.ai/docs/post-training/multi-lora.md#run-a-sweep) |
| `recipes/grpo/deepmath_shared.py` | DeepMath GRPO on a model resource that's already running. Set `MODEL_RESOURCES_ID` first. | [Attach to a running resource](https://docs.together.ai/docs/post-training/multi-lora.md#attach-to-a-running-resource) |
| `recipes/serverless_eval/*.py` | No training. Scores a hosted model on GSM8K, DeepMath, Game of 24, or the tool-use environment. | [Check a baseline before training](https://docs.together.ai/docs/post-training/evaluate.md#check-a-baseline-before-training) |

## Run outputs

### Run artifacts

Every cookbook recipe writes a record of the run to a run directory, `runs/<run_name>/`:

```text theme={null}
runs/<run_name>/
  config.yaml        the config, round-trippable through <Config>.from_yaml
  manifest.json      W&B URL, session ID, model, start time, git SHA
  metrics.jsonl      one JSON line per step, mirroring what goes to W&B
  checkpoints.jsonl  training and inference checkpoint IDs
  step_NNNN_*.json   per-step completions or episodes, when export_samples=True
  val_NNNN_*.json    validation completions, when export_samples=True
```

The cookbook always writes `metrics.jsonl`, so you can analyze a run offline without W\&B. To see what the model is producing, read the per-step `step_NNNN_*.json` files.

### Weights & Biases

When a config sets `wandb_project`, the cookbook logs every step to that project. `GRPOConfig` defaults it to `post-training-cookbook`. `SFTConfig` and `DPOConfig` leave it unset.

> [!WARNING]
>
> Set `WANDB_API_KEY` before you launch a run that logs to W\&B. The cookbook calls `wandb.init` only after it provisions the model resource, so a missing key fails the run after provisioning. In an interactive terminal, `wandb.init` waits at a login prompt instead, and the resource bills while it waits.

To run without a W\&B account, set `WANDB_MODE=offline` to keep the W\&B logs on disk, or set `wandb_project=None` to turn W\&B off.

To compare related runs side by side, such as the arms of a sweep, give them the same `wandb_group`.

The cookbook forwards `wandb_project`, `wandb_entity`, `wandb_group`, and `wandb_run_name` to the session as metadata, and the console shows them on the session's page. It doesn't send the run's ID or URL, so the page doesn't link to the run. To open the run, use the W\&B URL in `manifest.json`.

## Next steps

> [card] [Run GRPO recipes](https://docs.together.ai/docs/post-training/grpo-recipes.md)
>
> Train a GRPO recipe on a built-in dataset or your own.

> [card] [Run multi-turn GRPO recipes](https://docs.together.ai/docs/post-training/multi-turn-recipes.md)
>
> Train a tool-use agent on an environment you write.

> [card] [Monitor training runs](https://docs.together.ai/docs/post-training/monitoring.md)
>
> Watch a run in the console and read the metrics each recipe logs.

> [card] [Cookbook config reference](https://docs.together.ai/docs/post-training/cookbook.md)
>
> Look up every config field and its default.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.