Skip to main content
Run supervised fine-tuning (SFT) with the recipes from the post-training cookbook, on the No Robots dataset or on your own conversations. You can also continue an SFT adapter with group relative policy optimization (GRPO).

​
Requirements

  • A cookbook checkout.
  • A supported model. SFT doesn’t need generator support.
  • Your own dataset, unless you train on No Robots. Each row needs a conversation with at least one assistant turn.
  • Weights & Biases (W&B) logging set up, if your config sets wandb_project, as the No Robots example does. Without a W&B setup, the run fails after it provisions the model resource. In an interactive terminal, it waits at a W&B login prompt instead, and the resource bills while it waits. Export WANDB_API_KEY, or set WANDB_MODE=offline to log locally.

​
Run an SFT recipe

Pass an SFTConfig to the cookbook’s train function, which handles dataset loading, tokenization, loss masks, the training loop, and checkpoints. To train on your own data, use your own dataset. This example trains Qwen3.8 27B on No Robots, the dataset the SFT quickstart runs end to end, and leaves every other setting at its default:
train provisions a trainer-only model resource for model, attaches one session to it, and stops both when it returns. To train on a model resource that’s already running, set model_resources_id instead of model. The config sets enable_thinking=False because No Robots conversations contain no reasoning. For other data and models, match your data to the chat template. Settings worth adjusting first:
  • num_epochs and max_steps: train makes num_epochs passes over the dataset, one by default, and reshuffles the rows before each pass. max_steps caps the total number of steps, which keeps a test run short. Its default, None, sets no cap, so the run trains on every full batch, about 1,187 steps for one epoch of No Robots at a batch_size of 8.
  • batch_size: Larger batches (16 to 32) produce smoother loss curves.
  • learning_rate: For a low-rank adapter (LoRA) run, 1e-4 to 5e-4 is typical, but the right rate depends on the model and the task. Full-weight training needs a lower rate. For example, the cookbook’s No Robots recipes use 2e-5 for full-weight training and 1e-4 for LoRA on the same model. The rate decays linearly toward zero over the run.
  • use_lora: Set it to False to train the full weights instead of an adapter. Not every model supports full-weight training, so check supported models first.
  • train_on_what: TRAIN_ON_ALL trains on every assistant turn. TRAIN_ON_LAST trains on only the final one, which suits multi-turn datasets where only the last response matters.
The cookbook config reference lists every SFTConfig field.

​
Use your own dataset

Select your data with dataset_name, dataset_config, and dataset_split. The recipe loads data with Hugging Face datasets, so it also reads local files. For example, save one row per line in ./my_data/train.jsonl, then set dataset_name="./my_data" and dataset_split="train". Relative paths resolve from your working directory. By default, the recipe reads each row’s messages column, a list of {"role": ..., "content": ...} dicts. For any other schema, pass a to_messages function that maps each row to its messages:
Put any instruction you want the model to learn, such as an answer format, in the messages that to_messages returns. train runs to_messages on the first row before it provisions a model resource or session, so a schema mismatch fails before billing starts.

​
Match your data to the chat template

The cookbook renders each conversation with the model’s chat template. Check your model’s template behavior, then match your data and thinking settings to it:
  • Data without reasoning: Set enable_thinking as your model’s template requires. Qwen3.8 27B, the model in this page’s examples, needs False. Otherwise, the recipe skips those rows, so the run fails after it provisions the model resource.
  • Data with reasoning: Put assistant reasoning in reasoning_content. The recipe skips rows with <think> tags inside content. preserve_thinking controls whether earlier turns keep their reasoning.
  • Consecutive assistant turns: If your model’s template ends each turn with the next message’s role marker, merge consecutive assistant turns or split them before training. The recipe skips any row that has them and logs a warning.

​
Customize the loss mask

The recipe builds loss masks for the assistant turns that train_on_what selects. To weight tokens any other way, build the samples in your own training loop instead of calling train:
  1. Render each conversation into token sequences with the cookbook’s render_conversation.
  2. Change each sequence’s loss_mask.
  3. Pass each sequence’s tokens and mask to build_cross_entropy_sample.

​
Chain SFT and GRPO

To continue training an SFT adapter with GRPO, hand it off through a training checkpoint. The cookbook’s recipes/sft_then_grpo.py runs both phases on GSM8K. Its SFT phase teaches the #### <answer> format that the GRPO reward reads. To set up the same handoff in your own configs:
  1. Set training_checkpoint_interval so the SFT run writes a training checkpoint at its last step. The cookbook writes one only on steps that are a multiple of the interval.
  2. Set the GRPO config’s resume_from_checkpoint_id to the ID that train returns, and set load_optimizer=False. train returns the ID of the SFT run’s latest training checkpoint, or None if it wrote none.
  3. Set the same lora_rank and lora_alpha in both configs.
SFTConfig logs to W&B only when you set wandb_project, but GRPOConfig logs by default. Before you run both phases, set up W&B or pass wandb_project=None to GRPOConfig.

​
Resume from a checkpoint

To start an SFT recipe from a training checkpoint, set resume_from_checkpoint_id. The recipe restores the adapter and optimizer from the checkpoint, but its learning-rate schedule and data order restart from step zero. SFTConfig has no resume_run_dir, the setting a GRPO recipe uses to pick up its schedule and data order where the original run stopped.

​
Troubleshooting

  • The run fails before provisioning with Nothing to train: The split has fewer rows than batch_size. Use a larger split or a smaller batch_size.
  • The run fails after provisioning with nothing was renderable: The recipe skipped every row. Check the log for Skipping row warnings, then match your data to the chat template.
  • Out of memory or timeouts: Reduce batch_size or max_length, or raise timeout on the config.
For problems that aren’t specific to recipes, see Lifecycle, errors, and limits.

​
Next steps

GRPO recipes

Continue an SFT adapter with reinforcement learning.

Monitor training runs

Read the metrics an SFT recipe logs.

Cookbook config reference

Look up every SFTConfig field and its default.

Deploy a checkpoint

Deploy the trained model on dedicated model inference.