> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Cookbook reference

> The post-training cookbook's runnable recipes, and every field on its GRPO, SFT, and DPO config classes.

Every cookbook recipe passes a config object to a training function. This page lists the recipes the cookbook ships and every field on its three config classes: `GRPOConfig`, `SFTConfig`, and `DPOConfig`. For the API fields behind these settings, see the [post-training API reference](/reference/post-training/model-resources-create). For the limits that apply to them, see [Limits](/docs/post-training/model-resources#limits).

The cookbook renders prompts with a codec for each model family and loads each model's tokenizer from Hugging Face by its catalog ID. It has codecs for the Qwen models, `moonshotai/Kimi-K3`, `MiniMaxAI/MiniMax-M3`, and `zai-org/GLM-5.3`.

## Recipes

Run a recipe from a cookbook checkout with `uv run python <path>`. To set up a checkout, see the [RL quickstart](/docs/post-training/quickstart-rl). Each recipe is a single script with its config at the top. Edit the values there to change a run, or copy the file to start a new recipe.

| Recipe | What it trains | Covered in |
| - | - | - |
| `recipes/sft/no_robots.py` | SFT on No Robots instruction-following conversations. | [SFT quickstart](/docs/post-training/quickstart-sft) |
| `recipes/sft/no_robots_full.py` | The same SFT run with full-weight training instead of a low-rank adapter (LoRA). | [LoRA vs. full-weight training](/docs/post-training/training-methods#lora-vs-full-weight-training) |
| `recipes/dpo/helpsteer3.py` | DPO on HelpSteer3 preference pairs. | [Direct preference optimization](/docs/post-training/dpo) |
| `recipes/sft_then_grpo.py` | SFT on GSM8K to teach the answer format, then GRPO from the same adapter with a fresh optimizer. | [Chain SFT and GRPO](/docs/post-training/sft#chain-sft-and-grpo) |
| `recipes/grpo/gsm8k.py` | Single-turn GRPO on GSM8K grade-school math. | [GRPO quickstart](/docs/post-training/quickstart-rl) |
| `recipes/grpo/gsm8k_muon.py` | GRPO on GSM8K with Qwen3.8 27B, the Muon optimizer, and full-weight training. | [Optimizer](#optimizer) |
| `recipes/grpo/deepmath.py` | Single-turn GRPO on DeepMath competition math, with a 16K-token budget. | [Built-in datasets](/docs/post-training/reward-functions#built-in-datasets) |
| `recipes/grpo/tool_use_game.py` | Multi-turn GRPO on the bundled tool-use environment. | [Multi-turn environments](/docs/post-training/multi-turn-environments) |
| `recipes/grpo/tool_use_game_async.py` | The tool-use run with async collection and two generator replicas. | [Async training](/docs/post-training/grpo#async-training) |
| `recipes/grpo/deepmath_sweep.py` | A four-arm learning-rate sweep on DeepMath, each arm a session on one shared model resource. | [Run a sweep](/docs/post-training/multi-lora#run-a-sweep) |
| `recipes/grpo/deepmath_shared.py` | DeepMath GRPO on a model resource that's already running. Set `MODEL_RESOURCES_ID` first. | [Attach to a running resource](/docs/post-training/multi-lora#attach-to-a-running-resource) |
| `recipes/serverless_eval/*.py` | No training. Scores a hosted model on GSM8K, DeepMath, Game of 24, or the tool-use environment. | [Check a baseline before training](/docs/post-training/reward-functions#check-a-baseline-before-training) |

## Load a config from YAML

Each config class is a Python dataclass, so recipes set fields as keyword arguments. Every class also loads from a YAML file with the same field names:

```python theme={null}
from post_training_cookbook.grpo.config import GRPOConfig

config = GRPOConfig.from_yaml("config.yaml")
```

`from_yaml` raises `ValueError` on any key that isn't a field. Every run writes its config to `runs/<run_name>/config.yaml`, which loads back the same way.

## Shared fields

`GRPOConfig`, `SFTConfig`, and `DPOConfig` all take these fields. Where a default differs between the configs, the table lists each one.

| Field | Type | Default | Description |
| - | - | - | - |
| `use_lora` | `bool` | `True` | `True` trains a low-rank adapter (LoRA). `False` trains the full weights on a resource created with `lora_enabled=False`. Not every model supports full-weight training. See [Supported models](/docs/post-training/supported-models). |
| `lora_rank` | `int \| None` | `None` | Adapter rank. `None` uses the server default of `32`. |
| `lora_alpha` | `int \| None` | `None` | Adapter scaling factor. `None` uses the server default of `64`. |
| `lora_dropout` | `float \| None` | `None` | Dropout on the adapter layers. `None` uses the server default of `0.0`. |
| `adam_beta1` | `float` | `0.9` | AdamW first-moment decay. |
| `adam_beta2` | `float` | `0.95` | AdamW second-moment decay. |
| `adam_eps` | `float` | `1e-8` | AdamW epsilon. |
| `weight_decay` | `float` | `0.01` (GRPO), `0.1` (SFT, DPO) | Weight decay coefficient. Under Muon, it applies to both parameter groups. |
| `training_checkpoint_interval` | `int` | `0` | Steps between training checkpoints. `0` saves none. To save one at the end, set it to the step count. |
| `inference_checkpoint_interval` | `int` | `0` | Steps between intermediate inference checkpoints. The final step always saves one. |
| `resume_from_checkpoint_id` | `str \| None` | `None` | Training checkpoint to start the session from. |
| `load_optimizer` | `bool \| None` | `None` | `None` or `True` restores the optimizer state. `False` loads the weights only and starts a new run. To continue a GRPO run, also set `resume_run_dir`. SFT and DPO restart the learning-rate schedule at step zero either way. |
| `wandb_project` | `str \| None` | `"post-training-cookbook"` (GRPO), `None` (SFT, DPO) | Weights & Biases project. `None` turns off W\&B logging. Without a W\&B API key, a run that logs fails right after it provisions the resource, unless `WANDB_MODE=offline` is set. |
| `wandb_entity` | `str \| None` | `None` | W\&B team or username. |
| `wandb_run_name` | `str \| None` | `None` | W\&B run name. It also names the session and the run directory. `None` generates a name. |
| `wandb_group` | `str \| None` | `None` | W\&B group, for comparing related runs such as the arms of a sweep. |
| `timeout` | `float` | `7200.0` | Timeout in seconds for each SDK call the loop makes. |

## `GRPOConfig`

Configures `train_single_turn` and `train_multi_turn` in `post_training_cookbook.grpo.train`. Set exactly one of `model` and `model_resources_id`. It also takes the [fields every config shares](#shared-fields).

### Model and resource

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str \| None` | `None` | Catalog model ID to provision a new model resource for. The run stops the resource when it exits. |
| `model_resources_id` | `str \| None` | `None` | ID of an existing model resource to attach a session to. The resource defines the base model and keeps running after the run exits. |
| `num_generator_replicas` | `int` | `1` | Generator replicas to provision, at least `1`. Ignored when attaching to an existing resource. |

### Schedule

| Field | Type | Default | Description |
| - | - | - | - |
| `batch_size` | `int` | `16` | Prompts per step for single-turn runs, or tasks per step for multi-turn runs. |
| `num_steps` | `int \| None` | `None` | Total training steps, including any steps a continued run already completed. `None` runs `num_epochs` passes over the training examples, or one pass over the tasks for multi-turn runs. |
| `learning_rate` | `float` | `1e-5` | Peak learning rate. |
| `warmup_ratio` | `float` | `0.05` | Fraction of steps over which the learning rate rises linearly to its peak. |
| `min_lr_ratio` | `float` | `0.1` | Final learning rate as a fraction of the peak, at the end of the cosine decay. |

### Sampling and loss

| Field | Type | Default | Description |
| - | - | - | - |
| `temperature` | `float` | `0.7` | Sampling temperature for training completions. |
| `max_sampling_attempts` | `int` | `1` | Batches a step may draw to collect `batch_size` groups with varied rewards. `1` turns dynamic sampling off. |
| `loss` | `str` | `"grpo"` | Built-in loss function for each step: `"grpo"`, `"ppo"`, `"cispo"`, `"dppo"`, `"dro"`, or `"importance_sampling"`. See [Variants](/docs/post-training/grpo#variants). |
| `clip_low_threshold` | `float` | `0.8` | Lower bound on the importance ratio. Must be in `[0, 1]` for `"grpo"` and `"ppo"`, and at least `0` for `"cispo"`. Sent with `"grpo"`, `"ppo"`, and `"cispo"`. Leave it at the default with any other loss function, or the config raises. |
| `clip_high_threshold` | `float` | `1.28` | Upper bound on the importance ratio. Must be at least `1` for `"grpo"` and `"ppo"`, and at least `clip_low_threshold` for `"cispo"`. Sent with `"grpo"`, `"ppo"`, and `"cispo"`. Leave it at the default with any other loss function, or the config raises. |
| `loss_agg_type` | `str` | `"GRPO_LOSS_AGGREGATION_TYPE_FIXED_HORIZON"` | How per-token losses reduce to a scalar. Applies only to `"grpo"`. See [Variants](/docs/post-training/grpo#variants). |
| `ratio_type` | `str` | `"GRPO_LOSS_RATIO_TYPE_TOKEN"` | Importance-ratio granularity. `"GRPO_LOSS_RATIO_TYPE_SEQUENCE"` is GSPO. Applies only to `"grpo"`. |
| `dro_beta` | `float \| None` | `None` | Coefficient on DRO's log-ratio penalty. Required when `loss="dro"`, and rejected otherwise. |

### Token budgets

| Field | Type | Default | Description |
| - | - | - | - |
| `max_model_length` | `int` | `2048` | Maximum prompt plus completion length, in tokens. |
| `max_sample_tokens` | `int \| None` | `None` | Maximum tokens generated per completion. `None` uses `max_model_length`. |
| `min_response_tokens` | `int` | `1` | Minimum completion budget a prompt must leave room for. |
| `overlong_penalty_factor` | `float` | `1.0` | Scale of the soft penalty on completions that approach the budget. `0` turns the penalty off. |
| `overlong_buffer_len` | `int \| None` | `None` | Tokens before the budget at which the penalty starts. `None` uses `max_sample_tokens // 8`. |
| `overlong_masking` | `bool` | `False` | Skip truncated completions when building training samples. Works independently of the penalty. |

For single-turn runs, set `max_sample_tokens` or `max_prompt_length`. If both are `None`, the prompt budget is `0` and every prompt is dropped.

### Thinking

| Field | Type | Default | Description |
| - | - | - | - |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. `None` uses the chat template's default, which is on for the Qwen models and Kimi-K3, and adaptive on MiniMax-M3. GLM-5.3 always thinks, so `False` raises `ValueError`. |
| `thinking_effort` | `str \| None` | `None` | Reasoning budget: `"low"`, `"high"`, or `"max"`. Only GLM-5.3 and Kimi-K3 support it, and any other model raises `ValueError` when it's set. `None` uses the template default, which is `max` on GLM-5.3. Not on `DPOConfig`. |
| `preserve_thinking` | `bool \| None` | `None` | Controls whether earlier turns keep their reasoning when history is rendered. See the values below. |

`preserve_thinking` values:

| Value | History rendering | Training sequences |
| - | - | - |
| `None` | The chat template's own behavior. The Qwen3.5 and Qwen3.6 templates strip earlier reasoning. The Qwen3.8, GLM-5.3, Kimi-K3, and MiniMax-M3 templates keep it. | One sequence per turn wherever stripping changes the history. |
| `False` | Strips earlier reasoning on every model. Same as `None` on Qwen3.5 and Qwen3.6. | Same as `None`. |
| `True` | Keeps reasoning in every turn. Requires `enable_thinking` to be `None` or `True`. | Usually one sequence per conversation, which is cheaper. |

Set `preserve_thinking` to match inference. Use `True` only when the model also sees its earlier reasoning at inference. Otherwise, the run trains on history that inference never reproduces. [Reasoning in conversation history](/docs/post-training/concepts#reasoning-in-conversation-history) explains why stripping splits a conversation.

### Optimizer

| Field | Type | Default | Description |
| - | - | - | - |
| `optimizer` | `"adam" \| "muon"` | `"adam"` | Optimizer for the weight matrices. `"muon"` requires `use_lora=False`. |
| `muon_scaling_strategy` | `str \| None` | `None` | `"MUON_SCALING_STRATEGY_MATCH_ADAM"` or `"MUON_SCALING_STRATEGY_ORIGINAL"`. `None` uses the server default. |
| `muon_momentum` | `float \| None` | `None` | Muon momentum coefficient. `None` uses the server default. |
| `muon_newton_schulz_steps` | `int \| None` | `None` | Newton-Schulz iterations per Muon update. `None` uses the server default. |

Under Muon, the [shared](#shared-fields) AdamW fields still apply to the parameters Muon doesn't optimize: embeddings, norm layers, and the output head.

### Weight sync

| Field | Type | Default | Description |
| - | - | - | - |
| `weight_sync_type` | `str` | `"WEIGHT_SYNC_TYPE_SYNCHRONOUS"` | How new weights reach the generators after each step. The deferred modes require `max_steps_off_policy`. See [Async training](/docs/post-training/grpo#async-training). |

### Validation

| Field | Type | Default | Description |
| - | - | - | - |
| `validation_interval` | `int` | `50` | Steps between validation runs. The loop also validates once before the first step. |
| `val_temperature` | `float \| None` | `None` | Sampling temperature for validation. `None` samples greedily. |

### Checkpoints and resume

| Field | Type | Default | Description |
| - | - | - | - |
| `resume_run_dir` | `str \| None` | `None` | Directory of the run being continued. Required when continuing, and rejected otherwise. See [Continue a cookbook run](/docs/post-training/checkpoints#continue-a-cookbook-run). |

### Logging

| Field | Type | Default | Description |
| - | - | - | - |
| `export_samples` | `bool` | `True` | Write each step's completions or episodes to JSON files in the run directory. |

### Single-turn only

Multi-turn runs ignore these fields.

| Field | Type | Default | Description |
| - | - | - | - |
| `dataset` | `str` | `"gsm8k"` | Name of a dataset in the cookbook's registry. See [Built-in datasets](/docs/post-training/reward-functions#built-in-datasets). |
| `num_examples` | `int` | `7000` | Training examples drawn from the dataset. |
| `num_val_examples` | `int` | `200` | Validation examples, drawn from the rows after the training examples. `0` turns validation off. |
| `group_size` | `int` | `8` | Completions sampled per prompt. |
| `num_epochs` | `int` | `1` | Passes over the training examples when `num_steps` is `None`. |
| `max_prompt_length` | `int \| None` | `None` | Maximum prompt length, in tokens. Longer prompts are dropped before training. `None` uses `max_model_length - max_sample_tokens`. |

### Multi-turn only

Single-turn runs ignore these fields, except `max_steps_off_policy`, which `train_single_turn` rejects. They take effect when `max_turns` is set.

| Field | Type | Default | Description |
| - | - | - | - |
| `max_turns` | `int \| None` | `None` | Maximum assistant turns per episode. Required by `train_multi_turn`, which raises when it's `None`. `train_single_turn` ignores it. |
| `episodes_per_task` | `int` | `1` | Episodes per task per step. Set it to `2` or more, because a single episode has no group to compare against. |
| `failed_parse_reward` | `float` | `-0.1` | Reward for an episode whose sampled message can't be parsed. |
| `val_max_turns` | `int \| None` | `None` | Turn limit during validation. `None` uses `max_turns`. |
| `validation_iterations` | `int` | `1` | Validation passes per validation run, averaged together. |
| `sequential_validation` | `bool` | `False` | Run validation episodes one at a time instead of concurrently. |
| `max_steps_off_policy` | `int \| None` | `None` | `None` collects in lockstep. An integer turns on async collection and sets how many steps old a group may be and still train. |
| `groups_in_flight` | `int \| None` | `None` | Rollout groups collected concurrently under async collection. `None` uses `batch_size`. |

## `SFTConfig`

Configures `train` in `post_training_cookbook.sft.train`. SFT runs on a trainer-only resource, so it has no generator settings. Set exactly one of `model` and `model_resources_id`. It also takes the [fields every config shares](#shared-fields).

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str \| None` | `None` | Catalog model ID to provision a new trainer-only resource for. |
| `model_resources_id` | `str \| None` | `None` | ID of an existing model resource to attach a session to. |
| `dataset_name` | `str` | `"HuggingFaceH4/no_robots"` | Hugging Face dataset repo ID, or the path to a local directory of `.jsonl` files. Rows need a `messages` column unless you pass `to_messages` to `train`. See [Prepare your data](/docs/post-training/sft#prepare-your-data). |
| `dataset_config` | `str \| None` | `None` | Hugging Face dataset config name. |
| `dataset_split` | `str` | `"train"` | Dataset split to train on. |
| `batch_size` | `int` | `8` | Conversations per step. |
| `num_epochs` | `int` | `1` | Passes over the split. Each pass reshuffles the rows and drops the final partial batch. |
| `max_steps` | `int \| None` | `None` | Cap on total training steps. `None` trains on every full batch of every epoch. |
| `learning_rate` | `float` | `1e-4` | Starting learning rate, which decays linearly toward zero over the run. |
| `max_length` | `int` | `4096` | Maximum tokens per training sequence. |
| `train_on_what` | `str` | `"all_assistant_messages"` | `"all_assistant_messages"` (`TRAIN_ON_ALL`) or `"last_assistant_message"` (`TRAIN_ON_LAST`). Any other value raises a `ValueError` when you build the config. |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. Set it to `False` on Qwen models for data without reasoning. |
| `preserve_thinking` | `bool \| None` | `None` | Controls whether earlier turns keep their reasoning when history is rendered. See [Thinking](#thinking). |
| `thinking_effort` | `str \| None` | `None` | Reasoning budget on GLM-5.3 and Kimi-K3. See [Thinking](#thinking). |

`train` also takes a keyword-only `to_messages` function, which maps a dataset row to its list of messages. The default reads the `messages` column. `train` returns the ID of the latest training checkpoint, or `None` if the run saved none.

## `DPOConfig`

Configures `train` in `post_training_cookbook.dpo.train`. `DPOConfig` has no `model_resources_id` field, so every DPO run provisions its own trainer-only resource. It also takes the [fields every config shares](#shared-fields).

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str` | `"Qwen/Qwen3.8-27B"` | Catalog model ID to provision a trainer-only resource for. |
| `dataset_name` | `str` | `"nvidia/HelpSteer3"` | Hugging Face dataset repo ID. With the default `to_pair`, rows need HelpSteer3's `context`, `response1`, `response2`, and `overall_preference` columns. See [Direct preference optimization](/docs/post-training/dpo). |
| `dataset_config` | `str \| None` | `None` | Hugging Face dataset config name. |
| `dataset_split` | `str` | `"train"` | Dataset split to train on. |
| `dataset_subsample` | `int \| None` | `2000` | Maximum dataset rows to load, after a seeded shuffle. Rows that `to_pair` skips, such as HelpSteer3 ties, are dropped afterward. Pairs longer than `max_length` are truncated, and dropped only if a side has no trained token left. So training gets fewer pairs than this. `None` uses the whole split. |
| `val_split` | `str` | `"validation"` | Dataset split to validate on. |
| `val_subsample` | `int \| None` | `100` | Maximum validation rows to load, before rows that `to_pair` skips are dropped. `None` uses the whole split. `0` turns validation off. |
| `val_interval` | `int` | `100` | Steps between validation runs. `0` or less turns validation off. |
| `batch_size` | `int` | `8` | Preference pairs per step. |
| `max_steps` | `int` | `1000` | Total optimizer steps. |
| `learning_rate` | `float` | `5e-6` | Peak learning rate. |
| `warmup_ratio` | `float` | `0.03` | Fraction of steps over which the learning rate rises linearly to its peak. |
| `min_lr_ratio` | `float` | `0.1` | Final learning rate as a fraction of the peak, at the end of the cosine decay. |
| `max_length` | `int` | `2048` | Maximum tokens per chosen or rejected sequence. |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. Set it to `False` on Qwen models for preference data without reasoning, such as HelpSteer3. |
| `beta` | `float` | `0.1` | Temperature on the implicit reward. Higher values keep the policy closer to the reference. |
| `label_smoothing` | `float` | `0.0` | Conservative DPO label smoothing. |
| `precompute_ref_batch_size` | `int` | `128` | Batch size for the reference log-probability passes before training and for each validation pass. |

`train` also takes a keyword-only `to_pair` function, which maps a dataset row to a `(chosen_messages, rejected_messages)` tuple, or to `None` to skip the row. The default, `helpsteer3_pair`, reads HelpSteer3's columns. See [Use your own preference dataset](/docs/post-training/dpo#use-your-own-preference-dataset).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.