Overview

Post-training offers three training methods, which learn from different training signals: example responses, preference pairs, or scores from a reward function you write. Each method can train a low-rank adapter (LoRA) or the full model weights. This page explains how to choose a method and a training mode, and how to combine methods in one project. Fine-tuning jobs also run supervised fine-tuning and preference optimization, without a training loop of your own. To choose between the two services, see Post-training vs. standard fine-tuning.

​

Choose a training method

Choose a method based on the training data you can produce:

​

Supervised fine-tuning (SFT)

SFT trains the model to reproduce your example responses token by token, so it picks up their content, format, and style. It’s the simplest method to run, because it needs no reward function. But SFT can teach only what your demonstrations show. To push accuracy on a measurable task past them, continue training with GRPO, as described in Combine methods. SFT runs on a trainer-only model resource, with no generator replicas. A trainer-only resource costs less per minute than one that samples. Supervised fine-tuning covers data preparation, loss masks, and writing your own loop.

​

Direct preference optimization (DPO)

DPO trains on pairs of responses to the same prompt. Each step raises the likelihood of the preferred response relative to the rejected one, measured against a frozen reference copy of the model. It fits behavior that’s easier to judge than to demonstrate: you can say which of two responses is better but can’t write the ideal one. The pairs are fixed before the run starts, so you can collect them offline from human raters, a reward model, or comparisons between two systems. DPO doesn’t generate responses while it trains, so, like SFT, it runs on a trainer-only resource. Direct preference optimization covers preparing preference pairs and running the recipe.

​

Group relative policy optimization (GRPO)

GRPO is a reinforcement learning method that trains the model on its own responses. For each prompt, the model samples a group of completions, a reward function you write scores each one, and the update makes the completions that scored above the group’s average more likely. Because you write the reward function, GRPO fits tasks where your code can check an answer, such as math with a known solution or code with tests. It works even when you can’t write the ideal answer yourself. GRPO also trains agents that act over several turns and call tools, in multi-turn environments. GRPO is the only method that samples while it trains, so its model resource needs at least one generator replica, which adds GPUs and cost. Models marked trainer-only in Supported models can’t run it. Reinforcement learning with GRPO covers the training loop and its settings, and Write a reward function covers designing the reward. The service also ships related policy-gradient loss functions, listed in Built-in loss functions, and the cookbook’s GRPO loop can train with any of them.

​

Built-in and custom loss functions

Each method trains with a loss function, which defines what the model learns. For each batch, the loss function returns a single number, the loss, that measures how far the model’s outputs are from what you want it to learn. Each training step updates the weights to lower the loss. For how a step turns the loss into a weight update, see What training actually does. Every loss function in post-training works on the model’s log-probabilities. A token’s log-probability is the logarithm of the probability the model assigns to that token, given the tokens before it. A loss function that rewards a higher log-probability makes the model more likely to produce that token. For SFT and GRPO, the service runs a built-in loss function. For DPO, your code runs a custom loss function and sends the gradients of the loss to the service. Any method can use a custom loss function, not only DPO. Write one when no built-in loss function matches your objective. For example, a custom SFT loss function can add a regularization term to cross-entropy, such as a penalty for drifting from the starting model. Write a custom loss function lists more use cases and explains how, with examples for supervised and reinforcement learning loops.

​

LoRA vs. full-weight training

Every method trains in one of two modes:

• LoRA: Adds small trainable matrices alongside the frozen base weights and trains only those. The result is an adapter that you can download on its own or merge into the base model. LoRA is the default.
• Full-weight training (full fine-tuning): Updates every parameter in the model. The result is a full set of model weights.

Set the mode when you create the model resource, with lora_enabled in the SDK or use_lora in a cookbook config. Every session on the resource trains in that mode, and the mode can’t change after creation. Manage model resources covers the other settings that are fixed at creation.

​

When to use LoRA

Start with LoRA. Use it when:

• You’re starting a new project: LoRA runs on every catalog model and is the default in every cookbook config, so it’s the shortest path to a working training loop.
• You want to compare several adapters: Several LoRA sessions can share one model resource, so a hyperparameter sweep costs one base model’s GPU time. On a multi-LoRA deployment, you can then compare the trained adapters on one deployment.
• You’re tuning style, format, or domain vocabulary: These are the kinds of updates LoRA handles best.

​

When to use full-weight training

Not every catalog model supports full-weight training. Before you set lora_enabled=False or use_lora=False, check that the model’s Training modes on Supported models include full. When you list models programmatically, look for a full entry in trainer_config. Use full-weight training when:

• The base behavior needs a substantial change: A model that doesn’t know the task you’re training for may need every weight updated, not only an adapter.
• LoRA results plateau below your target: First raise lora_rank (rank in the SDK’s LoraConfig), up to the model’s maximum in Supported models. Raise lora_alpha with it, up to its maximum of 128, to keep the alpha-to-rank ratio the same, because the adapter’s update scales with lora_alpha / lora_rank. Raising lora_alpha alone acts like a higher learning rate. If quality still falls short, switch to full-weight training.
• You need the Muon optimizer or pipeline weight sync: Both require full-weight training.

​

What changes with full-weight training

To compare the hourly rate of both modes for your model, call estimate_cost once with lora_enabled=True and once with lora_enabled=False.

​

Combine methods

To run methods in sequence, save a training checkpoint at the end of one run and start the next run from it. Each phase runs in its own session. Keep the base model and the training mode the same in every phase. A full-weight SFT run can hand off to a full-weight GRPO run, but a training checkpoint from a LoRA session can’t start a full-weight session, or the reverse. A common combination teaches the output format with SFT, then optimizes accuracy with GRPO. The format matters because GRPO learns only from groups whose completions score differently. If the base model rarely produces correct answers, or always produces only them, there will be no training signal. The cookbook skips any group where every completion scores the same, so those groups train nothing. A short SFT run on correctly formatted examples gets the rewards to vary, so GRPO has a signal from its first steps. Chain SFT and GRPO shows the full handoff. The same handoff works from SFT into DPO. DPO measures each update against the model as it was when the DPO run started, so after an SFT handoff, the SFT model is the reference.