> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LoRA vs. full-weight training

> Choose between training a LoRA adapter and updating every model weight, and compare cost, limits, and deployment for each mode.

Every training method runs in one of two modes:

* **LoRA:** The service trains only a low-rank adapter (LoRA), a set of small matrices added alongside the frozen base weights. You can download the adapter on its own or merge it into the base model. LoRA is the default.
* **Full-weight training (full fine-tuning):** The service updates every parameter in the model. The result is a full set of model weights.

Set the mode when you [create the model resource](https://docs.together.ai/docs/post-training/model-resources.md#training-mode), with `lora_enabled` in the SDK or `use_lora` in a cookbook config. Every session on the resource trains in that mode, and you can't change the mode after creation.

## When to use LoRA

Start with LoRA. Use it when:

* **You're starting a new project:** LoRA runs on every catalog model and is the default in every cookbook config, so it's the shortest path to a working training loop.
* **You want to compare several adapters:** Several LoRA sessions can [share one model resource](https://docs.together.ai/docs/post-training/multi-lora.md), so you pay for only one resource across a hyperparameter sweep. You can then compare the trained adapters on a single [multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md).
* **You're tuning style, format, or domain vocabulary:** These are the kinds of updates LoRA handles best.

## When to use full-weight training

Use full-weight training when:

* **The base behavior needs a substantial change:** A model that doesn't know the task you're training for may need every weight updated, not only an adapter.
* **LoRA results plateau below your target:** First raise `lora_rank` (`rank` in the SDK's `LoraConfig`), up to the model's maximum in [Supported models](https://docs.together.ai/docs/post-training/supported-models.md). Raise `lora_alpha` (`alpha` in the SDK) with it, up to its maximum of `128`, to keep the alpha-to-rank ratio the same. The adapter's update scales with `lora_alpha / lora_rank`, so raising `lora_alpha` alone acts like a higher learning rate. If quality still falls short, switch to full-weight training.
* **You need the [Muon optimizer](https://docs.together.ai/docs/post-training/model-resources.md#optimizer) or [pipeline weight sync](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training):** Both require full-weight training.

Not every model in the catalog supports full-weight training. Before you set `lora_enabled=False` or `use_lora=False`, check that the model's **Training modes** on [Supported models](https://docs.together.ai/docs/post-training/supported-models.md) include **full**. When you [list models programmatically](https://docs.together.ai/docs/post-training/supported-models.md#list-supported-models-programmatically), look for a `full` entry in `trainer_config`.

## What changes with full-weight training

| | LoRA | Full-weight |
| - | - | - |
| Models | Every catalog model | Models whose **Training modes** on [Supported models](https://docs.together.ai/docs/post-training/supported-models.md) include **full** |
| Sessions per resource | Several, up to the model's limit in [Supported models](https://docs.together.ai/docs/post-training/supported-models.md) | One, so a hyperparameter sweep needs one resource per configuration |
| Hourly rate | Per model, in [Pricing](https://docs.together.ai/docs/post-training/pricing.md#estimate-the-cost-of-a-resource) | The same as LoRA or higher, depending on the model |
| Max context | Per model, in [Supported models](https://docs.together.ai/docs/post-training/supported-models.md) | The same as LoRA or lower, depending on the model |
| Learning rate (supervised fine-tuning) | Higher, such as `1e-4` in `recipes/sft/no_robots.py` | Lower, such as `2e-5` in `recipes/sft/no_robots_full.py` |
| Optimizer | Adam | Adam or Muon |
| Weight sync modes (group relative policy optimization) | Synchronous, background publish | Synchronous, background publish, pipeline |
| Inference checkpoint | Adapter, plus merged weights on Qwen models | Merged weights |
| Deployment | As an adapter on a [multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md), or merged into the base model at startup [on its own deployment](https://docs.together.ai/docs/post-training/deployment.md#deploy-a-lora-adapter-in-merged-mode) | [Directly](https://docs.together.ai/docs/post-training/deployment.md#deploy-a-merged-model), with no merge step |
| [Tinker client](https://docs.together.ai/docs/post-training/migrate-from-tinker.md) | Supported | Not supported |

To compare the hourly rate of both modes for your model, call `estimate_cost` once with `lora_enabled=True` and once with `lora_enabled=False`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.