> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Build samples and loss masks

> Align a sample's arrays with its target tokens, build samples by hand, and mask a chat conversation's prompt.

When you write your own training loop with the Together Python SDK, you build the [samples](https://docs.together.ai/docs/post-training/concepts.md#data-and-samples) it trains on. Each sample pairs a token sequence with the per-token values that the loss function reads. This page shows how a sample's arrays line up, how to build them by hand, and how to mask a chat conversation so only the reply trains.

## How a sample's arrays line up

A `Sample` holds its input tokens in `model_input` and its per-token arrays in a `loss_fn_inputs` mapping. The arrays line up so that each position is scored on predicting the next token:

* **Input tokens:** `model_input` holds the prompt and response, minus the final token.
* **Target tokens:** `target_tokens` holds the same sequence minus the first token, so position `i` is scored on predicting `target_tokens[i]`, the token that follows `model_input[i]`.
* **Weights:** `weights` is the loss mask. A position with `1.0` trains on its target token, and a position with `0.0` doesn't. To train on the response only, use `0.0` where the target token is part of the prompt and `1.0` where it's part of the response.

For example, a six-token sequence, `t0` through `t5`, with a three-token prompt gives five positions:

| Position | `model_input` | `target_tokens` | `weights` |
| - | - | - | - |
| 0 | `t0` | `t1` | `0.0` |
| 1 | `t1` | `t2` | `0.0` |
| 2 | `t2` | `t3` | `1.0` |
| 3 | `t3` | `t4` | `1.0` |
| 4 | `t4` | `t5` | `1.0` |

Position 2 reads the last prompt token, but its target is the first response token, so its weight is `1.0`. The last position's target is `t5`, so `t5` trains like any other response token, even though it isn't in `model_input`.

Group relative policy optimization (GRPO) samples add two more arrays to `loss_fn_inputs`:

* **Advantages:** `advantages` holds the completion's advantage at every trained position and `0.0` elsewhere.
* **Log-probabilities:** `logprobs` holds the log-probability the generator assigned to each completion token when it sampled the completion, and `0.0` on the prompt.

## Build samples by hand

Every array in `loss_fn_inputs` needs one value per position. To build each array, start with one value per token in the sequence, then drop the first value, as `target_tokens` drops the first token. Use `0.0` for positions you don't train on rather than leaving them out.

> [!WARNING]
>
> The service doesn't shift arrays or check that they line up. An array trimmed from the wrong end still has the right length, so the run trains on the wrong tokens without an error. Trim every array in one function, as `build_sample` in the next example does.

The `build_sample` function builds a cross-entropy sample from a token sequence and a loss mask with one value per token:

```python theme={null}
from together.post_training import Sample


def build_sample(tokens: list[int], loss_mask: list[float]) -> Sample:
    # Drop the last token from the input, and the first value
    # from every per-token array
    return {
        "model_input": {"chunks": [{"encoded_text": {"tokens": tokens[:-1]}}]},
        "loss_fn_inputs": {
            "target_tokens": {"data": tokens[1:], "dtype": "int64"},
            "weights": {"data": loss_mask[1:], "dtype": "float32"},
        },
    }
```

Each array is a tensor that holds its values in `data` and declares its `dtype`, either `int64` or `float32`.

To build a GRPO sample from a [sampled completion](https://docs.together.ai/docs/post-training/training-loop.md#sample-from-the-generator), also add the `advantages` and `logprobs` arrays. Build each one the same way as `weights`, from a per-token list with its first value dropped.

### Build a loss mask from a chat template

To train on only the final reply in a chat conversation, build the loss mask from the model's chat template. Render the conversation twice with the model's tokenizer: once without the final reply, ready for the model to respond, and once in full. When the full rendering starts with the prompt rendering, every token after the prompt belongs to the reply. This example passes the resulting mask to `build_sample`:

```python theme={null}
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")


def render(messages: list[dict], **kwargs) -> list[int]:
    # Apply the chat template, with thinking off for data without reasoning
    return tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        return_dict=True,
        enable_thinking=False,
        **kwargs,
    )["input_ids"]


messages = [
    {"role": "user", "content": "What is 2 + 2?"},
    {"role": "assistant", "content": "4"},
]

# Render the prompt as the model sees it before it replies,
# then the full conversation
prompt = render(messages[:-1], add_generation_prompt=True)
tokens = render(messages)
if tokens[: len(prompt)] != prompt:
    raise ValueError("The template renders the prompt differently")

# Train on the reply and its stop token, not on the prompt
loss_mask = [0.0] * len(prompt) + [1.0] * (len(tokens) - len(prompt))
sample = build_sample(tokens, loss_mask)
```

For a single-turn Qwen3.8 27B conversation, the example builds the same tokens and loss mask as the [cookbook](https://docs.together.ai/docs/post-training/cookbook-overview.md). To adapt it to other models or conversations, handle these [chat template behaviors](https://docs.together.ai/docs/post-training/concepts.md#chat-templates-and-thinking):

* **Thinking:** Templates differ in whether they think by default and whether you can turn thinking off. For data without reasoning, set `enable_thinking` to fit [the model's thinking behavior](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates). The `enable_thinking=False` in `render` turns thinking off for Qwen3.8 27B.
* **Stop tokens:** Some templates don't close the final reply with a stop token. For those, append [the model's stop token](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates) to `tokens` and `1.0` to `loss_mask`, or the model never learns to stop.
* **Multi-turn conversations:** To train on every assistant turn, build one sample per turn from the conversation up to and including that turn. Some templates strip reasoning from earlier turns, so a later turn's prompt doesn't always start with the earlier turns as the model produced them. A sample per turn avoids the mismatch, because each turn trains against the prompt the template renders for it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.