Cookbook reference

Every cookbook recipe passes a config object to a training function. This page lists the recipes the cookbook ships and every field on its three config classes: GRPOConfig, SFTConfig, and DPOConfig. For the API fields behind these settings, see the post-training API reference. For the limits that apply to them, see Limits. The cookbook renders prompts with a codec for each model family and loads each model’s tokenizer from Hugging Face by its catalog ID. It has codecs for the Qwen models, moonshotai/Kimi-K3, MiniMaxAI/MiniMax-M3, and zai-org/GLM-5.3.

Recipes

Run a recipe from a cookbook checkout with uv run python <path>. To set up a checkout, see the RL quickstart. Each recipe is a single script with its config at the top. Edit the values there to change a run, or copy the file to start a new recipe.

Recipe	What it trains	Covered in
`recipes/sft/no_robots.py`	SFT on No Robots instruction-following conversations.	SFT quickstart
`recipes/sft/no_robots_full.py`	The same SFT run with full-weight training instead of a low-rank adapter (LoRA).	LoRA vs. full-weight training
`recipes/dpo/helpsteer3.py`	DPO on HelpSteer3 preference pairs.	Direct preference optimization
`recipes/sft_then_grpo.py`	SFT on GSM8K to teach the answer format, then GRPO from the same adapter with a fresh optimizer.	Chain SFT and GRPO
`recipes/grpo/gsm8k.py`	Single-turn GRPO on GSM8K grade-school math.	GRPO quickstart
`recipes/grpo/gsm8k_muon.py`	GRPO on GSM8K with Qwen3.8 27B, the Muon optimizer, and full-weight training.	Optimizer
`recipes/grpo/deepmath.py`	Single-turn GRPO on DeepMath competition math, with a 16K-token budget.	Built-in datasets
`recipes/grpo/tool_use_game.py`	Multi-turn GRPO on the bundled tool-use environment.	Multi-turn environments
`recipes/grpo/tool_use_game_async.py`	The tool-use run with async collection and two generator replicas.	Async training
`recipes/grpo/deepmath_sweep.py`	A four-arm learning-rate sweep on DeepMath, each arm a session on one shared model resource.	Run a sweep
`recipes/grpo/deepmath_shared.py`	DeepMath GRPO on a model resource that’s already running. Set `MODEL_RESOURCES_ID` first.	Attach to a running resource
`recipes/serverless_eval/*.py`	No training. Scores a hosted model on GSM8K, DeepMath, Game of 24, or the tool-use environment.	Check a baseline before training

Load a config from YAML

Each config class is a Python dataclass, so recipes set fields as keyword arguments. Every class also loads from a YAML file with the same field names:

from post_training_cookbook.grpo.config import GRPOConfig

config = GRPOConfig.from_yaml("config.yaml")

from_yaml raises ValueError on any key that isn’t a field. Every run writes its config to runs/<run_name>/config.yaml, which loads back the same way.

Shared fields

GRPOConfig, SFTConfig, and DPOConfig all take these fields. Where a default differs between the configs, the table lists each one.

Field	Type	Default	Description
`use_lora`	`bool`	`True`	`True` trains a low-rank adapter (LoRA). `False` trains the full weights on a resource created with `lora_enabled=False`. Not every model supports full-weight training. See Supported models.
`lora_rank`	`int \| None`	`None`	Adapter rank. `None` uses the server default of `32`.
`lora_alpha`	`int \| None`	`None`	Adapter scaling factor. `None` uses the server default of `64`.
`lora_dropout`	`float \| None`	`None`	Dropout on the adapter layers. `None` uses the server default of `0.0`.
`adam_beta1`	`float`	`0.9`	AdamW first-moment decay.
`adam_beta2`	`float`	`0.95`	AdamW second-moment decay.
`adam_eps`	`float`	`1e-8`	AdamW epsilon.
`weight_decay`	`float`	`0.01` (GRPO), `0.1` (SFT, DPO)	Weight decay coefficient. Under Muon, it applies to both parameter groups.
`training_checkpoint_interval`	`int`	`0`	Steps between training checkpoints. `0` saves none. To save one at the end, set it to the step count.
`inference_checkpoint_interval`	`int`	`0`	Steps between intermediate inference checkpoints. The final step always saves one.
`resume_from_checkpoint_id`	`str \| None`	`None`	Training checkpoint to start the session from.
`load_optimizer`	`bool \| None`	`None`	`None` or `True` restores the optimizer state. `False` loads the weights only and starts a new run. To continue a GRPO run, also set `resume_run_dir`. SFT and DPO restart the learning-rate schedule at step zero either way.
`wandb_project`	`str \| None`	`"post-training-cookbook"` (GRPO), `None` (SFT, DPO)	Weights & Biases project. `None` turns off W&B logging. Without a W&B API key, a run that logs fails right after it provisions the resource, unless `WANDB_MODE=offline` is set.
`wandb_entity`	`str \| None`	`None`	W&B team or username.
`wandb_run_name`	`str \| None`	`None`	W&B run name. It also names the session and the run directory. `None` generates a name.
`wandb_group`	`str \| None`	`None`	W&B group, for comparing related runs such as the arms of a sweep.
`timeout`	`float`	`7200.0`	Timeout in seconds for each SDK call the loop makes.

`GRPOConfig`

Configures train_single_turn and train_multi_turn in post_training_cookbook.grpo.train. Set exactly one of model and model_resources_id. It also takes the fields every config shares.

Model and resource

Field	Type	Default	Description
`model`	`str \| None`	`None`	Catalog model ID to provision a new model resource for. The run stops the resource when it exits.
`model_resources_id`	`str \| None`	`None`	ID of an existing model resource to attach a session to. The resource defines the base model and keeps running after the run exits.
`num_generator_replicas`	`int`	`1`	Generator replicas to provision, at least `1`. Ignored when attaching to an existing resource.

Schedule

Field	Type	Default	Description
`batch_size`	`int`	`16`	Prompts per step for single-turn runs, or tasks per step for multi-turn runs.
`num_steps`	`int \| None`	`None`	Total training steps, including any steps a continued run already completed. `None` runs `num_epochs` passes over the training examples, or one pass over the tasks for multi-turn runs.
`learning_rate`	`float`	`1e-5`	Peak learning rate.
`warmup_ratio`	`float`	`0.05`	Fraction of steps over which the learning rate rises linearly to its peak.
`min_lr_ratio`	`float`	`0.1`	Final learning rate as a fraction of the peak, at the end of the cosine decay.

Sampling and loss

Field	Type	Default	Description
`temperature`	`float`	`0.7`	Sampling temperature for training completions.
`max_sampling_attempts`	`int`	`1`	Batches a step may draw to collect `batch_size` groups with varied rewards. `1` turns dynamic sampling off.
`loss`	`str`	`"grpo"`	Built-in loss function for each step: `"grpo"`, `"ppo"`, `"cispo"`, `"dppo"`, `"dro"`, or `"importance_sampling"`. See Variants.
`clip_low_threshold`	`float`	`0.8`	Lower bound on the importance ratio. Must be in `[0, 1]` for `"grpo"` and `"ppo"`, and at least `0` for `"cispo"`. Sent with `"grpo"`, `"ppo"`, and `"cispo"`. Leave it at the default with any other loss function, or the config raises.
`clip_high_threshold`	`float`	`1.28`	Upper bound on the importance ratio. Must be at least `1` for `"grpo"` and `"ppo"`, and at least `clip_low_threshold` for `"cispo"`. Sent with `"grpo"`, `"ppo"`, and `"cispo"`. Leave it at the default with any other loss function, or the config raises.
`loss_agg_type`	`str`	`"GRPO_LOSS_AGGREGATION_TYPE_FIXED_HORIZON"`	How per-token losses reduce to a scalar. Applies only to `"grpo"`. See Variants.
`ratio_type`	`str`	`"GRPO_LOSS_RATIO_TYPE_TOKEN"`	Importance-ratio granularity. `"GRPO_LOSS_RATIO_TYPE_SEQUENCE"` is GSPO. Applies only to `"grpo"`.
`dro_beta`	`float \| None`	`None`	Coefficient on DRO’s log-ratio penalty. Required when `loss="dro"`, and rejected otherwise.

Token budgets

Field	Type	Default	Description
`max_model_length`	`int`	`2048`	Maximum prompt plus completion length, in tokens.
`max_sample_tokens`	`int \| None`	`None`	Maximum tokens generated per completion. `None` uses `max_model_length`.
`min_response_tokens`	`int`	`1`	Minimum completion budget a prompt must leave room for.
`overlong_penalty_factor`	`float`	`1.0`	Scale of the soft penalty on completions that approach the budget. `0` turns the penalty off.
`overlong_buffer_len`	`int \| None`	`None`	Tokens before the budget at which the penalty starts. `None` uses `max_sample_tokens // 8`.
`overlong_masking`	`bool`	`False`	Skip truncated completions when building training samples. Works independently of the penalty.

For single-turn runs, set max_sample_tokens or max_prompt_length. If both are None, the prompt budget is 0 and every prompt is dropped.

Thinking

Field	Type	Default	Description
`enable_thinking`	`bool \| None`	`None`	Turns the model’s `<think>` output on or off. `None` uses the chat template’s default, which is on for the Qwen models and Kimi-K3, and adaptive on MiniMax-M3. GLM-5.3 always thinks, so `False` raises `ValueError`.
`thinking_effort`	`str \| None`	`None`	Reasoning budget: `"low"`, `"high"`, or `"max"`. Only GLM-5.3 and Kimi-K3 support it, and any other model raises `ValueError` when it’s set. `None` uses the template default, which is `max` on GLM-5.3. Not on `DPOConfig`.
`preserve_thinking`	`bool \| None`	`None`	Controls whether earlier turns keep their reasoning when history is rendered. See the values below.

preserve_thinking values:

Value	History rendering	Training sequences
`None`	The chat template’s own behavior. The Qwen3.5 and Qwen3.6 templates strip earlier reasoning. The Qwen3.8, GLM-5.3, Kimi-K3, and MiniMax-M3 templates keep it.	One sequence per turn wherever stripping changes the history.
`False`	Strips earlier reasoning on every model. Same as `None` on Qwen3.5 and Qwen3.6.	Same as `None`.
`True`	Keeps reasoning in every turn. Requires `enable_thinking` to be `None` or `True`.	Usually one sequence per conversation, which is cheaper.

Set preserve_thinking to match inference. Use True only when the model also sees its earlier reasoning at inference. Otherwise, the run trains on history that inference never reproduces. Reasoning in conversation history explains why stripping splits a conversation.

Optimizer

Field	Type	Default	Description
`optimizer`	`"adam" \| "muon"`	`"adam"`	Optimizer for the weight matrices. `"muon"` requires `use_lora=False`.
`muon_scaling_strategy`	`str \| None`	`None`	`"MUON_SCALING_STRATEGY_MATCH_ADAM"` or `"MUON_SCALING_STRATEGY_ORIGINAL"`. `None` uses the server default.
`muon_momentum`	`float \| None`	`None`	Muon momentum coefficient. `None` uses the server default.
`muon_newton_schulz_steps`	`int \| None`	`None`	Newton-Schulz iterations per Muon update. `None` uses the server default.

Under Muon, the shared AdamW fields still apply to the parameters Muon doesn’t optimize: embeddings, norm layers, and the output head.

Weight sync

Field	Type	Default	Description
`weight_sync_type`	`str`	`"WEIGHT_SYNC_TYPE_SYNCHRONOUS"`	How new weights reach the generators after each step. The deferred modes require `max_steps_off_policy`. See Async training.

Validation

Field	Type	Default	Description
`validation_interval`	`int`	`50`	Steps between validation runs. The loop also validates once before the first step.
`val_temperature`	`float \| None`	`None`	Sampling temperature for validation. `None` samples greedily.

Checkpoints and resume

Field	Type	Default	Description
`resume_run_dir`	`str \| None`	`None`	Directory of the run being continued. Required when continuing, and rejected otherwise. See Continue a cookbook run.

Logging

Field	Type	Default	Description
`export_samples`	`bool`	`True`	Write each step’s completions or episodes to JSON files in the run directory.

Single-turn only

Multi-turn runs ignore these fields.

Field	Type	Default	Description
`dataset`	`str`	`"gsm8k"`	Name of a dataset in the cookbook’s registry. See Built-in datasets.
`num_examples`	`int`	`7000`	Training examples drawn from the dataset.
`num_val_examples`	`int`	`200`	Validation examples, drawn from the rows after the training examples. `0` turns validation off.
`group_size`	`int`	`8`	Completions sampled per prompt.
`num_epochs`	`int`	`1`	Passes over the training examples when `num_steps` is `None`.
`max_prompt_length`	`int \| None`	`None`	Maximum prompt length, in tokens. Longer prompts are dropped before training. `None` uses `max_model_length - max_sample_tokens`.

Multi-turn only

Single-turn runs ignore these fields, except max_steps_off_policy, which train_single_turn rejects. They take effect when max_turns is set.

Field	Type	Default	Description
`max_turns`	`int \| None`	`None`	Maximum assistant turns per episode. Required by `train_multi_turn`, which raises when it’s `None`. `train_single_turn` ignores it.
`episodes_per_task`	`int`	`1`	Episodes per task per step. Set it to `2` or more, because a single episode has no group to compare against.
`failed_parse_reward`	`float`	`-0.1`	Reward for an episode whose sampled message can’t be parsed.
`val_max_turns`	`int \| None`	`None`	Turn limit during validation. `None` uses `max_turns`.
`validation_iterations`	`int`	`1`	Validation passes per validation run, averaged together.
`sequential_validation`	`bool`	`False`	Run validation episodes one at a time instead of concurrently.
`max_steps_off_policy`	`int \| None`	`None`	`None` collects in lockstep. An integer turns on async collection and sets how many steps old a group may be and still train.
`groups_in_flight`	`int \| None`	`None`	Rollout groups collected concurrently under async collection. `None` uses `batch_size`.

`SFTConfig`

Configures train in post_training_cookbook.sft.train. SFT runs on a trainer-only resource, so it has no generator settings. Set exactly one of model and model_resources_id. It also takes the fields every config shares.

Field	Type	Default	Description
`model`	`str \| None`	`None`	Catalog model ID to provision a new trainer-only resource for.
`model_resources_id`	`str \| None`	`None`	ID of an existing model resource to attach a session to.
`dataset_name`	`str`	`"HuggingFaceH4/no_robots"`	Hugging Face dataset repo ID, or the path to a local directory of `.jsonl` files. Rows need a `messages` column unless you pass `to_messages` to `train`. See Prepare your data.
`dataset_config`	`str \| None`	`None`	Hugging Face dataset config name.
`dataset_split`	`str`	`"train"`	Dataset split to train on.
`batch_size`	`int`	`8`	Conversations per step.
`num_epochs`	`int`	`1`	Passes over the split. Each pass reshuffles the rows and drops the final partial batch.
`max_steps`	`int \| None`	`None`	Cap on total training steps. `None` trains on every full batch of every epoch.
`learning_rate`	`float`	`1e-4`	Starting learning rate, which decays linearly toward zero over the run.
`max_length`	`int`	`4096`	Maximum tokens per training sequence.
`train_on_what`	`str`	`"all_assistant_messages"`	`"all_assistant_messages"` (`TRAIN_ON_ALL`) or `"last_assistant_message"` (`TRAIN_ON_LAST`). Any other value raises a `ValueError` when you build the config.
`enable_thinking`	`bool \| None`	`None`	Turns the model’s `<think>` output on or off. Set it to `False` on Qwen models for data without reasoning.
`preserve_thinking`	`bool \| None`	`None`	Controls whether earlier turns keep their reasoning when history is rendered. See Thinking.
`thinking_effort`	`str \| None`	`None`	Reasoning budget on GLM-5.3 and Kimi-K3. See Thinking.

train also takes a keyword-only to_messages function, which maps a dataset row to its list of messages. The default reads the messages column. train returns the ID of the latest training checkpoint, or None if the run saved none.

`DPOConfig`

Configures train in post_training_cookbook.dpo.train. DPOConfig has no model_resources_id field, so every DPO run provisions its own trainer-only resource. It also takes the fields every config shares.

Field	Type	Default	Description
`model`	`str`	`"Qwen/Qwen3.8-27B"`	Catalog model ID to provision a trainer-only resource for.
`dataset_name`	`str`	`"nvidia/HelpSteer3"`	Hugging Face dataset repo ID. With the default `to_pair`, rows need HelpSteer3’s `context`, `response1`, `response2`, and `overall_preference` columns. See Direct preference optimization.
`dataset_config`	`str \| None`	`None`	Hugging Face dataset config name.
`dataset_split`	`str`	`"train"`	Dataset split to train on.
`dataset_subsample`	`int \| None`	`2000`	Maximum dataset rows to load, after a seeded shuffle. Rows that `to_pair` skips, such as HelpSteer3 ties, are dropped afterward. Pairs longer than `max_length` are truncated, and dropped only if a side has no trained token left. So training gets fewer pairs than this. `None` uses the whole split.
`val_split`	`str`	`"validation"`	Dataset split to validate on.
`val_subsample`	`int \| None`	`100`	Maximum validation rows to load, before rows that `to_pair` skips are dropped. `None` uses the whole split. `0` turns validation off.
`val_interval`	`int`	`100`	Steps between validation runs. `0` or less turns validation off.
`batch_size`	`int`	`8`	Preference pairs per step.
`max_steps`	`int`	`1000`	Total optimizer steps.
`learning_rate`	`float`	`5e-6`	Peak learning rate.
`warmup_ratio`	`float`	`0.03`	Fraction of steps over which the learning rate rises linearly to its peak.
`min_lr_ratio`	`float`	`0.1`	Final learning rate as a fraction of the peak, at the end of the cosine decay.
`max_length`	`int`	`2048`	Maximum tokens per chosen or rejected sequence.
`enable_thinking`	`bool \| None`	`None`	Turns the model’s `<think>` output on or off. Set it to `False` on Qwen models for preference data without reasoning, such as HelpSteer3.
`beta`	`float`	`0.1`	Temperature on the implicit reward. Higher values keep the policy closer to the reference.
`label_smoothing`	`float`	`0.0`	Conservative DPO label smoothing.
`precompute_ref_batch_size`	`int`	`128`	Batch size for the reference log-probability passes before training and for each validation pass.

train also takes a keyword-only to_pair function, which maps a dataset row to a (chosen_messages, rejected_messages) tuple, or to None to skip the row. The default, helpsteer3_pair, reads HelpSteer3’s columns. See Use your own preference dataset.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Recipes

Load a config from YAML

Shared fields

`GRPOConfig`

Model and resource

Schedule

Sampling and loss

Token budgets

Thinking

Optimizer

Weight sync

Validation

Checkpoints and resume

Logging

Single-turn only

Multi-turn only

`SFTConfig`

`DPOConfig`

​Recipes

​Load a config from YAML

​Shared fields

​GRPOConfig

​Model and resource

​Schedule

​Sampling and loss

​Token budgets

​Thinking

​Optimizer

​Weight sync

​Validation

​Checkpoints and resume

​Logging

​Single-turn only

​Multi-turn only

​SFTConfig

​DPOConfig

Recipes

Load a config from YAML

Shared fields

`GRPOConfig`

Model and resource

Schedule

Sampling and loss

Token budgets

Thinking

Optimizer

Weight sync

Validation

Checkpoints and resume

Logging

Single-turn only

Multi-turn only

`SFTConfig`

`DPOConfig`