> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Write a training loop

> Write a custom training loop with the Together Python SDK, from a single step to batch training and sampling.

Write your own loop when you need control over the data, sampling, rewards, or loss function that isn't in a [cookbook recipe](https://docs.together.ai/docs/post-training/cookbook-overview.md).

Your training loop calls the Together Python SDK to create a [model resource and a session](https://docs.together.ai/docs/post-training/concepts.md#model-resources-and-sessions), then sends each step's operations to the resource.

> [!TIP]
>
> If you already have a Tinker training script, you can [migrate it](https://docs.together.ai/docs/post-training/migrate-from-tinker.md) by changing its import line.

## Install the SDK

Install the Together Python SDK, version 3.0.0 or later:

```bash theme={null}
uv add "together>=3.0.0"
```

## Run a minimal loop

This loop trains a low-rank adapter (LoRA) for Qwen3.8 27B on one sample for one step, then saves the adapter as an inference checkpoint. The model resource it creates bills [per GPU-minute](https://docs.together.ai/docs/post-training/pricing.md) from when it's ready until the loop exits:

```python theme={null}
from together.post_training import ComputeConfig, ModelResourcesClient
from together.post_training import AdamParams, LossConfig, Sample

# "The capital of France is" and " Paris." as Qwen3.8 token IDs
tokens = [760, 6511, 314, 9338, 369, 11751, 13]
mask = [0.0] * 5 + [1.0] * 2  # train only on the reply
sample: Sample = {
    "model_input": {"chunks": [{"encoded_text": {"tokens": tokens[:-1]}}]},
    "loss_fn_inputs": {
        "target_tokens": {"data": tokens[1:], "dtype": "int64"},
        "weights": {"data": mask[1:], "dtype": "float32"},
    },
}

# Create a model resource
with ModelResourcesClient.create(
    base_model="Qwen/Qwen3.8-27B",
    compute_config=ComputeConfig(num_generator_replicas=0),
) as resources:

    # Create a session
    with resources.create_session() as session:

        # Train on the sample
        session.trainer.forward_backward(
            samples=[sample], loss=LossConfig(type="LOSS_TYPE_CROSS_ENTROPY")
        )
        session.trainer.optim_step(adam_params=AdamParams(learning_rate=1e-4))

        # Save the checkpoint
        print(session.create_inference_checkpoint().id)
```

To build on this loop, learn how to:

* [Build samples from text](https://docs.together.ai/docs/post-training/samples.md).
* [Configure the model resource](https://docs.together.ai/docs/post-training/model-resources.md#create-a-resource).
* [Download and resume from checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md).

## Train on a batch

This loop trains on eight of your `samples` per step, for `NUM_STEPS` steps. You pass the learning rate on every `optim_step`, so it can follow a schedule, such as this loop's linear decay:

```python theme={null}
for step in range(NUM_STEPS):
    batch = samples[step * 8 : (step + 1) * 8]
    result = session.trainer.forward_backward(
        samples=batch, loss=LossConfig(type="LOSS_TYPE_CROSS_ENTROPY")
    )
    learning_rate = 1e-4 * (1 - step / NUM_STEPS)
    session.trainer.optim_step(
        adam_params=AdamParams(learning_rate=learning_rate),
    )

    # Report the loss per trained token
    trained = sum(sum(s["loss_fn_inputs"]["weights"]["data"]) for s in batch)
    print(f"step {step + 1}: loss={result.loss / trained:.4f}")
```

For cross-entropy, `result.loss` is a sum over the batch's trained tokens, not a mean. Divide it by the number of trained tokens for a per-token loss that stays comparable as the batch's token count changes.

To train with group relative policy optimization (GRPO), build [GRPO samples](https://docs.together.ai/docs/post-training/samples.md) and pass a policy-gradient loss. `GrpoLossParams` sets the [clip band](https://docs.together.ai/docs/post-training/loss-functions.md#clipping) and the `beta` weight of a [Kullback-Leibler (KL) penalty](https://docs.together.ai/docs/post-training/loss-functions.md#kl-penalty):

```python theme={null}
from together.post_training import GrpoLossParams, LossConfig

loss = LossConfig(
    type="LOSS_TYPE_GRPO",
    grpo_params=GrpoLossParams(
        clip_low_threshold=0.8,
        clip_high_threshold=1.28,
        beta=0.01,  # Needs reference_logprobs in every sample
    ),
)
session.trainer.forward_backward(samples=batch, loss=loss)
```

## Sample from the generator

Pass all your prompts to `sample_batch` in one call, and set `num_samples` to request a group of completions for each prompt. In this example, `prompts` holds each prompt's token IDs:

```python theme={null}
from together.post_training import SamplingParams

results = session.generator.sample_batch(
    [{"chunks": [{"encoded_text": {"tokens": p}}]} for p in prompts],
    num_samples=8,
    sampling_params=SamplingParams(max_tokens=768, temperature=0.7),
)

# One result per prompt, in order
for result in results:  
    for sequence in result.sequences:

        # Token IDs can come back as strings, so convert them
        tokens = [int(t) for t in sequence.tokens]
        logprobs = sequence.logprobs or []

# After each optim_step, publish the new weights to the generators
session.trainer.weights_sync()
```

Build each completion's [GRPO sample](https://docs.together.ai/docs/post-training/samples.md#build-samples-by-hand) from its `tokens` and `logprobs`.

Sampling can run at the same time as other sampling and trainer operations, but a synchronous `weights_sync` waits for in-flight sampling to finish.

One `SamplingParams` applies to every prompt in a `sample_batch` call. The [API reference](https://docs.together.ai/reference/post-training/operations-sample-create.md) lists its fields and defaults. Two fields need care:

* **`stop`:** For models without stop-string support, omit `stop` and truncate completions in your code. A request with an unsupported stop string fails as invalid input.
* **`seed`:** Leave it unset. It's unsupported and causes an internal error when the generator uses multiple inference engines.

### Compute log-probabilities for fixed text

Call `session.generator.compute_logprobs(prompt)` for one prompt, or `compute_logprobs_batch(prompts)` for several. If the trainer has stepped since the last sync, call `weights_sync` first, so the log-probabilities come from the updated weights. Entry `0` of the result is a `0` placeholder, because the first token has no preceding context. `sample` with `prompt_logprobs=True` returns the same placeholder.

## Session operations

Every operation targets a [session](https://docs.together.ai/docs/post-training/concepts.md#model-resources-and-sessions). Calls on `session.trainer` run on the model resource's [trainer](https://docs.together.ai/docs/post-training/concepts.md#trainer-and-generators). Calls on `session.generator` run on its generator replicas, which exist only on a resource created with `num_generator_replicas` above `0`.

| Call | What it does |
| - | - |
| `session.trainer.forward_backward` | Computes a [built-in loss function](https://docs.together.ai/docs/post-training/loss-functions.md#built-in-loss-functions) over a batch of samples and accumulates gradients. |
| `session.trainer.forward` | Returns each target token's log-probability without accumulating gradients, as the first pass of a [custom loss function](https://docs.together.ai/docs/post-training/custom-loss-functions.md). |
| `session.trainer.custom_forward_backward` | Backpropagates the gradients your code computed for each log-probability, as the second pass of a custom loss function. |
| `session.trainer.optim_step` | Applies the accumulated gradients with the learning rate and optimizer parameters you pass. |
| `session.trainer.weights_sync` | Publishes the trainer's current weights to the generators, so the next sample comes from the updated policy. |
| `session.generator.sample`, `sample_batch` | Samples completions from the generators' current weights, with each token's log-probability. |
| `session.generator.compute_logprobs`, `compute_logprobs_batch` | Returns each token's log-probability for a fixed input under the generators' current weights. |
| `session.create_inference_checkpoint` | Saves the weights so you can [download or deploy](https://docs.together.ai/docs/post-training/checkpoints.md#inference-checkpoints) them. |
| `session.create_training_checkpoint` | Saves the weights, optimizer state, and step counter, so a later session can [resume the run](https://docs.together.ai/docs/post-training/checkpoints.md#resume-or-fork-a-run). |

The [API reference](https://docs.together.ai/reference/post-training/operations-forward-backward-create.md) lists every operation's fields and defaults. `weights_sync` takes a `weight_sync_type`, which defaults to `WEIGHT_SYNC_TYPE_SYNCHRONOUS` and also accepts the deferred modes used for [async training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training).

Each call submits an operation, then waits until the operation finishes or the call's `timeout` runs out. When an operation fails on the resource, the SDK raises an [operation error](https://docs.together.ai/docs/post-training/limits-and-errors.md#operation-errors).

### Call operations asynchronously

Every call has an `_async` variant, such as `forward_backward_async`, `sample_batch_async`, or `ModelResourcesClient.create_async`. Use the `_async` variants in async code. The synchronous calls block the calling thread, so they raise `RuntimeError` when called from code that's already running an event loop, such as an async application.

Jupyter notebooks are the exception: a synchronous call works in a notebook cell but blocks the cell until it returns. The `_async` variants work in notebooks too.

The `_async` variants also let independent work overlap. For example, one session can sample while another trains, or you can [train several adapters at once](https://docs.together.ai/docs/post-training/multi-lora.md#train-adapters-with-the-sdk).

## Next steps

For complete loops, see the cookbook's notebooks and direct preference optimization (DPO) loop, which build samples with the cookbook's helpers.

> [card] [SFT notebook](https://github.com/togethercomputer/post-training-cookbook/blob/main/notebooks/sft_example.ipynb)
>
> Supervised fine-tuning (SFT) on the No Robots dataset, step by step.

> [card] [GRPO notebook](https://github.com/togethercomputer/post-training-cookbook/blob/main/notebooks/grpo_example.ipynb)
>
> GRPO on GSM8K math problems, step by step.

> [card] [DPO training loop](https://github.com/togethercomputer/post-training-cookbook/blob/main/post_training_cookbook/dpo/train.py)
>
> DPO on chosen and rejected pairs, built on a custom loss function.

> [card] [Samples and loss masks](https://docs.together.ai/docs/post-training/samples.md)
>
> Line up a sample's arrays and mask a chat prompt.

> [card] [Custom loss functions](https://docs.together.ai/docs/post-training/custom-loss-functions.md)
>
> Train toward an objective you compute in PyTorch, such as DPO.

> [card] [Model resources](https://docs.together.ai/docs/post-training/model-resources.md)
>
> Choose GPUs, attach to a running resource, and stop it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.