> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Supervised fine-tuning

> Train a model on example conversations with supervised fine-tuning.

Supervised fine-tuning (SFT) trains a model to imitate example responses. Use it when you have high-quality demonstrations of the behavior you want: human-written answers, a target output format, or conversations from a system that already works. The [SFT quickstart](/docs/post-training/quickstart-sft) walks through a complete run. This page explains how SFT runs on post-training, how to bring your own data or write your own loop, and how to hand the trained adapter to a GRPO run.

<Note>
  [Fine-tuning jobs](/docs/fine-tuning/supervised) also run SFT, with a fixed data format and no training loop to write. For help choosing between the two, see [Post-training vs. standard fine-tuning](/docs/post-training/overview#post-training-vs-standard-fine-tuning).
</Note>

## How post-training runs SFT

SFT runs on a trainer-only model resource. Nothing samples from the model, so the resource runs with `num_generator_replicas=0`. Each step runs two operations:

1. `forward_backward` with `LOSS_TYPE_CROSS_ENTROPY`, which computes the weighted next-token cross-entropy over the batch and accumulates gradients.
2. `optim_step`, which applies the accumulated gradients with the step's learning rate.

The next step trains directly on the updated weights.

## Run an SFT recipe

The cookbook's `SFTConfig` and `train` function handle dataset loading, tokenization, loss masks, the training loop, and checkpoints. This example trains `Qwen/Qwen3.8-27B` on the No Robots dataset, with every other setting at its default:

```python theme={null}
from post_training_cookbook.sft.config import SFTConfig
from post_training_cookbook.sft.train import train

config = SFTConfig(
    model="Qwen/Qwen3.8-27B",
    enable_thinking=False,
    dataset_name="HuggingFaceH4/no_robots",
    max_steps=200,
    wandb_project="sft-no-robots",
)

train(config)
```

`train` provisions a trainer-only model resource for `model`, attaches one session to it, and stops both when it returns. To train on a resource that's already running, set `model_resources_id` instead of `model`. See [Manage model resources](/docs/post-training/model-resources).

The config sets `enable_thinking=False` because No Robots conversations contain no reasoning, as [Data for reasoning models](#data-for-reasoning-models) explains.

The config also sets `wandb_project`, so `train` logs to Weights & Biases. Export `WANDB_API_KEY` before you run it, or set `WANDB_MODE=offline` to log locally. Without either, the run fails after it provisions the model resource, or, in an interactive terminal, waits at a W\&B login prompt while the resource bills.

Settings worth adjusting first:

* **`num_epochs` and `max_steps`:** `train` makes `num_epochs` passes over the dataset, one by default, and reshuffles the rows before each pass. `max_steps` caps the total number of steps, which keeps a test run short. Its default, `None`, sets no cap, so a config without it trains on every full batch: about 1,187 steps for No Robots at a `batch_size` of 8. The learning rate decays linearly toward zero over the run.
* **`learning_rate`:** The right rate depends on the model and the task. For a low-rank adapter (LoRA) run, `1e-4` to `5e-4` is typical. Full-weight training needs a lower rate. The cookbook's No Robots recipes use `2e-5` for full-weight training and `1e-4` for LoRA on the same model.
* **`train_on_what`:** `TRAIN_ON_ALL` trains on every assistant turn. `TRAIN_ON_LAST` trains on only the final one, which suits multi-turn datasets where only the last response matters.
* **`batch_size`:** Larger batches (16 to 32) produce smoother loss curves.
* **`use_lora=False`:** Trains the full weights instead of an adapter. Not every model supports full-weight training, so check [Supported models](/docs/post-training/supported-models) first. See [LoRA vs. full-weight training](/docs/post-training/training-methods#lora-vs-full-weight-training).

The [cookbook reference](/docs/post-training/cookbook#sftconfig) lists every `SFTConfig` field.

## Prepare your data

`train` accepts any Hugging Face dataset with a `messages` column. In each row, `messages` is a list of `{"role": ..., "content": ...}` dicts:

```json theme={null}
{
  "messages": [
    {"role": "user", "content": "Write a haiku about programming."},
    {
      "role": "assistant",
      "content": "Code flows like water\nBugs hide in the morning light\nTests bring peace of mind"
    }
  ]
}
```

For a dataset with other columns, set `dataset_name` to it and pass a `to_messages` function to `train`. It maps one row to its list of messages. Put any instruction you want the model to learn, such as an answer format, in that mapping:

```python theme={null}
def qa_messages(row):
    return [
        {"role": "user", "content": row["question"]},
        {"role": "assistant", "content": row["answer"]},
    ]


train(config, to_messages=qa_messages)
```

Before it provisions a model resource or opens a session, `train` calls `to_messages` on the first row, so a schema mismatch, such as a missing `messages` column, fails before you're billed.

For data that isn't on Hugging Face, save the rows as a JSON Lines file named `train.jsonl`, one row per line, in a folder of its own. Then set `dataset_name` to the folder's path, such as `"./my_data"`, and keep `dataset_split="train"`. `train` loads the folder with the Hugging Face `datasets` library, which reads `train.jsonl` into the `train` split. A relative path resolves from the directory you run the recipe in.

### Data for reasoning models

Two chat-template behaviors decide whether your rows train:

* **Turn thinking off for data without reasoning:** Every Qwen model in the catalog turns thinking on by default. For conversations without `<think>` blocks, such as No Robots, set `enable_thinking=False`. With thinking on, the template renders each prompt differently from the full conversation, so `train` skips every row. The run then fails with nothing trained, after it has already provisioned the model resource. GLM-5.3 doesn't need the setting: its template always thinks and renders these rows as they are, and the cookbook rejects `enable_thinking=False` for it.
* **Avoid consecutive assistant turns on GLM models:** The GLM template ends an assistant turn with the role marker of the message after it, so an assistant turn followed by another assistant turn has no stop token. `train` skips those rows with a warning. Merge or split the turns before training.

For conversations that do contain reasoning, put each assistant turn's reasoning in a `reasoning_content` field next to `content`. Qwen3.8-27B's template reads reasoning only from that field, so `train` skips rows that carry it as `<think>` tags inside `content`. `preserve_thinking` controls whether earlier turns keep their reasoning when the history is rendered. See [Thinking settings](/docs/post-training/cookbook#thinking).

## Loss masks

SFT doesn't train on every token equally. The model's chat template wraps each message in role headers and special tokens, and a loss mask gives each position a weight that sets how much it counts toward the loss:

* Role headers, including the assistant's (`<|im_start|>assistant` in Qwen's chat template), and user content get weight `0`.
* Assistant content and the assistant's end-of-turn token (`<|im_end|>` in Qwen's template) get weight `1`. Training on the end-of-turn token teaches the model when to stop.

The cookbook's `render_conversation` builds the mask from the chat template and your `train_on_what` choice. Weights are `float32`, so you can down-weight some turns with fractional values instead of masking them out. [Concepts](/docs/post-training/concepts#data-and-samples) explains how the mask lines up with the input and target tokens.

## Write your own SFT loop

When none of the cookbook recipes fit your use case, you can write your own training loop. This loop trains on an in-memory list of conversations:

```python theme={null}
from post_training_cookbook.codecs import get_codec
from post_training_cookbook.samples import (
    build_cross_entropy_sample,
    is_trainable,
)
from post_training_cookbook.sft.train import TRAIN_ON_ALL, render_conversation
from together.post_training import (
    AdamParams,
    ComputeConfig,
    LoraConfig,
    LossConfig,
    ModelResourcesClient,
)

MODEL = "Qwen/Qwen3.8-27B"
BATCH_SIZE = 8
conversations = load_conversations()  # a list of message lists

# Load the model's chat template, with thinking off for data without reasoning
codec = get_codec(MODEL, enable_thinking=False)

# Provision a trainer-only model resource (SFT never samples)
with ModelResourcesClient.create(
    base_model=MODEL,
    compute_config=ComputeConfig(
        gpu_type="B200-SXM", num_generator_replicas=0
    ),
) as resources:

    # Start a training session that trains a LoRA adapter
    with resources.create_session(
        lora_config=LoraConfig(rank=16, alpha=16)
    ) as session:

        # Each step trains on the next BATCH_SIZE conversations
        for step, start in enumerate(range(0, len(conversations), BATCH_SIZE)):

            # Render each conversation into tokens and a loss mask
            # that trains only the assistant turns
            samples = []
            for messages in conversations[start : start + BATCH_SIZE]:
                for seq in render_conversation(
                    messages,
                    codec=codec,
                    train_on_what=TRAIN_ON_ALL,
                    max_length=4096,
                ):
                    # Skip sequences that truncation left with no trained token
                    if not is_trainable(seq.loss_mask):
                        continue
                    samples.append(
                        build_cross_entropy_sample(
                            tokens=seq.tokens, loss_mask=seq.loss_mask
                        )
                    )

            # Compute the cross-entropy loss and accumulate gradients
            result = session.trainer.forward_backward(
                samples=samples,
                loss=LossConfig(type="LOSS_TYPE_CROSS_ENTROPY"),
            )

            # Update the adapter's weights
            session.trainer.optim_step(
                adam_params=AdamParams(learning_rate=1e-4, weight_decay=0.1),
            )

            # The loss is summed over the trained tokens, not averaged
            print(f"[step {step}] loss_sum={result.loss:.1f}")

        # Save the adapter as an inference checkpoint you can deploy
        checkpoint = session.create_inference_checkpoint()
        print(checkpoint.id)
```

`render_conversation` returns a list of sequences, not a single one, because it splits a conversation wherever the chat template [strips earlier reasoning](/docs/post-training/concepts#reasoning-in-conversation-history).

`build_cross_entropy_sample` shifts the tokens into input and target positions for you. For cross-entropy, `result.loss` is the sum over all trained positions, not a mean. Divide it by the batch's total weight to get the per-token mean the recipes log.

To train with a variant of cross-entropy, such as one with an added regularization term, [write a custom loss function](/docs/post-training/custom-loss-functions). Each step then uses `forward` and `custom_forward_backward` instead of `forward_backward`.

## Chain SFT and GRPO

To continue an SFT adapter with [GRPO](/docs/post-training/grpo), hand it over with a training checkpoint. [Combine methods](/docs/post-training/training-methods#combine-methods) explains when this helps. The handoff takes three settings:

1. Set `training_checkpoint_interval` so the SFT run writes a training checkpoint at its last step. The cookbook writes one only when the interval fires, and `train` returns the latest checkpoint ID (or `None`).
2. Start the GRPO run with `resume_from_checkpoint_id` set to that ID and `load_optimizer=False`. The new session loads the adapter weights with a fresh optimizer and its step counter at zero.
3. Use the same LoRA rank and alpha in both phases. A checkpoint with a different rank fails to load, and a different alpha silently changes how strongly the adapter applies.

Because SFT is trainer-only and GRPO needs a generator, the two phases run in separate sessions, typically on a new model resource for the same base model:

```python theme={null}
from dataclasses import replace

from post_training_cookbook.datasets.registry import DATASETS
from post_training_cookbook.grpo.config import GRPOConfig
from post_training_cookbook.grpo.train import train_single_turn
from post_training_cookbook.sft.config import SFTConfig
from post_training_cookbook.sft.train import train as sft_train

MODEL = "Qwen/Qwen3.8-27B"


def gsm8k_messages(row):
    # Pose each question the way the GRPO phase does, so SFT teaches
    # the "#### <answer>" format that the GRPO reward reads.
    return [
        {"role": "user", "content": DATASETS["gsm8k"].format_question(row)},
        {"role": "assistant", "content": row["answer"]},
    ]


sft_config = SFTConfig(
    model=MODEL,
    dataset_name="openai/gsm8k",
    dataset_config="main",
    max_steps=20,
    enable_thinking=False,
    lora_rank=16,
    lora_alpha=16,
    training_checkpoint_interval=20,
)
grpo_config = GRPOConfig(
    model=MODEL,
    dataset="gsm8k",
    enable_thinking=False,
    num_steps=20,
    max_model_length=1536,
    max_prompt_length=768,
    max_sample_tokens=768,
    lora_rank=16,
    lora_alpha=16,
)

checkpoint_id = sft_train(sft_config, to_messages=gsm8k_messages)
train_single_turn(
    replace(
        grpo_config,
        resume_from_checkpoint_id=checkpoint_id,
        load_optimizer=False,
    )
)
```

GSM8K rows have `question` and `answer` columns, not `messages`, so the SFT phase maps each row with `to_messages`. The mapping poses each question in the GRPO prompt format, so SFT teaches the answer format that GRPO then scores. The cookbook's `recipes/sft_then_grpo.py` is the complete worked example.

Unlike `SFTConfig`, `GRPOConfig` logs to Weights & Biases by default, so export `WANDB_API_KEY` or `WANDB_MODE=offline`, or pass `wandb_project=None` to `GRPOConfig`, before you run this example.

## Next steps

<CardGroup cols={2}>
  <Card title="Direct preference optimization" icon="scale" href="/docs/post-training/dpo">
    Train on preference pairs with a custom loss function.
  </Card>

  <Card title="Reinforcement learning with GRPO" icon="target" href="/docs/post-training/grpo">
    Continue the adapter with reinforcement learning.
  </Card>

  <Card title="Cookbook reference" icon="adjustments-horizontal" href="/docs/post-training/cookbook#sftconfig">
    Every `SFTConfig` field and its default.
  </Card>

  <Card title="Deploy a checkpoint" icon="rocket" href="/docs/post-training/deployment">
    Deploy the trained model on dedicated model inference.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.