LoRA vs. full fine-tuning

• LoRA: Trains a small set of adapter weights on top of the frozen base model.
• Full fine-tuning: Updates every weight in the base model.

​

Choose a method

• You’re starting a new fine-tune: LoRA gets you a working model fastest and at the lowest cost.
• You want to ship multiple adapters from the same base: Adapters are small and can be swapped on a single hosted base model.
• You’re tuning style, format, or domain vocabulary: These are the kinds of updates that LoRA handles best.

• The base behavior needs a substantial change: A model that doesn’t know the task you’re training for may need every weight updated, not just an adapter.
• LoRA results plateau below your target: Try increasing lora_r and lora_alpha first, and if quality still falls short, switch to full fine-tuning.

​

Set the method on your job

from together import Together

client = Together()

# LoRA (default) — lora=True is optional
job = client.fine_tuning.create(
    training_file="<FILE_ID>",
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
    lora=True,
)

# Full fine-tuning
job = client.fine_tuning.create(
    training_file="<FILE_ID>",
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
    lora=False,
)

import Together from "together-ai";

const client = new Together();

// LoRA (default) — lora: true is optional
const loraJob = await client.fineTuning.create({
  training_file: "<FILE_ID>",
  model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
  lora: true,
});

// Full fine-tuning
const fullJob = await client.fineTuning.create({
  training_file: "<FILE_ID>",
  model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
  lora: false,
});

# LoRA (default)
tg fine-tuning create \
  --training-file "<FILE_ID>" \
  --model "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference" \
  --lora

# Full fine-tuning
tg fine-tuning create \
  --training-file "<FILE_ID>" \
  --model "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference" \
  --no-lora

​

LoRA settings

​

Default target modules

​

What to expect from full fine-tuning

• Supported models: Full fine-tuning is available for a subset of the models that support LoRA. Large mixture-of-experts models, long-context variants, and some vision-language models are LoRA-only. See supported models for the per-model breakdown.
• Smaller batch sizes: Because full fine-tuning updates every weight, it carries a larger memory footprint, so the maximum batch size for a given model is generally smaller than the LoRA equivalent.
• Higher cost: Full fine-tuning trains every parameter rather than the 0.1% to 1% a LoRA job touches, so it consumes more compute and costs more. See pricing for details.

from together import Together

client = Together()

limits = client.fine_tuning.model_limits(model_name="<MODEL_ID>")
print(limits.supports_full_training)

​

Serve your model

• LoRA: After the job completes, deploy the merged model on a dedicated endpoint. See deployment.
• Full fine-tuning: The job produces a complete model rather than a compact adapter. Deploy it on a dedicated endpoint, or download the weights for local use. See deployment.