> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LoRA fine-tuning

> Fine-tune and run dedicated inference for a model with LoRA adapters

## Overview

LoRA (Low-Rank Adaptation) enables efficient fine-tuning of large language models by training only a small set of additional parameters while keeping the original model weights frozen. This approach delivers several key advantages:

* **Reduced training costs**: Trains fewer parameters than full fine-tuning, using less GPU memory
* **Faster deployment**: Produces compact adapter files that can be quickly shared and deployed

Together AI handles the LoRA fine-tuning workflow. Once training is complete, you can deploy your fine-tuned model using a [dedicated endpoint](/docs/dedicated-endpoints/overview) for inference.

## Quick start

This guide demonstrates how to fine-tune a model using LoRA. For a general introduction to fine-tuning on Together AI, see the [fine-tuning overview](/docs/fine-tuning-overview).

### Prerequisites

* Together AI API key
* Training data in the JSONL format

### Step 1: Upload Training Data

First, upload your training dataset to Together AI:

<CodeGroup>
  ```bash CLI theme={null}
  tg files upload "your-datafile.jsonl"
  ```

  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

  files_response = client.files.upload(file="your-datafile.jsonl")

  print(files_response.model_dump())
  ```
</CodeGroup>

### Step 2: Create Fine-tuning Job

Launch a LoRA fine-tuning job using the uploaded file ID:

<CodeGroup>
  ```curl CLI theme={null}
  tg fine-tuning create \
    --training-file "file-629e58b4-ff73-438c-b2cc-f69542b27980" \
    --model "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference" \
    --lora
  ```

  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

  fine_tuning_response = client.fine_tuning.create(
      training_file=files_response.id,
      model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
      lora=True,
  )

  print(fine_tuning_response.model_dump())
  ```
</CodeGroup>

> **Note**: If you plan to use a validation set, make sure to set the `--validation-file` and `--n-evals` (the number of evaluations over the entire job) parameters. `--n-evals` needs to be set as a number above 0 in order for your validation set to be used.

Once you submit the fine-tuning job you should be able to see the model `output_name` and `job_id` in the response:

<CodeGroup>
  ```json Json theme={null}
  {
    "id": "ft-44129430-ac08-4136-9774-aed81e0164a4",
    "training_file": "file-629e58b4-ff73-438c-b2cc-f69542b27980",
    "validation_file": "",
    "model": "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
    "output_name": "zainhas/Meta-Llama-3.1-8B-Instruct-Reference-my-demo-finetune-4224205a",
    ...
  }
  ```
</CodeGroup>

### Step 3: Getting the output model

Once you submit the fine-tuning job you should be able to see the model `output_name` and `job_id` in the response:

<CodeGroup>
  ```json Json theme={null}
  {
    "id": "ft-44129430-ac08-4136-9774-aed81e0164a4",
    "training_file": "file-629e58b4-ff73-438c-b2cc-f69542b27980",
    "validation_file": "",
    "model": "meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",
    "output_name": "zainhas/Meta-Llama-3.1-8B-Instruct-Reference-my-demo-finetune-4224205a",
    ...
  }
  ```
</CodeGroup>

You can also see the status of the job and get the model name if you navigate to your fine-tuned model in the 'Model' or 'Jobs' tab in the Together dashboard.

<Frame>
  <img src="https://mintcdn.com/togetherai-52386018/msWWavplJrEZR36N/images/docs/a9c3fb15e77e1df27b72195dc80dea4d0748fcf5f958a15d5884bdb982d3d9b9-image.png?fit=max&auto=format&n=msWWavplJrEZR36N&q=85&s=7e332c714e184d6d4d9554b761a6e350" alt="" width="1920" height="1080" data-path="images/docs/a9c3fb15e77e1df27b72195dc80dea4d0748fcf5f958a15d5884bdb982d3d9b9-image.png" />
</Frame>

### Step 4: Deploy for inference

Once the fine-tuning job is completed, you can deploy your model for inference using a [dedicated endpoint](/docs/dedicated-endpoints/overview). See [Deploying a Fine-tuned Model](/docs/deploying-a-fine-tuned-model) for detailed instructions.

## Best Practices

1. **Data Preparation**: Ensure your training data follows the correct JSONL format for your chosen model
2. **Validation Sets**: Always include validation data to monitor training quality
3. **Model Naming**: Use descriptive names for easy identification in production
4. **Monitoring**: Track training metrics through the Together dashboard

## Frequently Asked Questions

### Which base models support LoRA fine-tuning?

Together AI supports LoRA fine-tuning on a curated selection of high-performance base models. See the [supported models list](/docs/fine-tuning-models) for current options.

### What's the difference between LoRA and full fine-tuning?

LoRA trains only a small set of additional parameters (typically 0.1-1% of model size), resulting in faster training, lower costs, and smaller output files, while full fine-tuning updates all model parameters for maximum customization at higher computational cost.

### How do I run inference on my LoRA fine-tuned model?

Once training is complete, deploy your model using a [dedicated endpoint](/docs/dedicated-endpoints/overview). See [Deploying a Fine-tuned Model](/docs/deploying-a-fine-tuned-model) for instructions.

## Next Steps

* Explore [advanced fine-tuning parameters](/docs/fine-tuning-quickstart#advanced-topics) for optimizing model performance
* Learn about [uploading custom adapters](/docs/dedicated-endpoints/adapter) trained outside Together AI
* Deploy your model with a [dedicated endpoint](/docs/dedicated-endpoints/overview)