Skip to main content
You can upload custom or fine-tuned models from Hugging Face or S3 and run inference on a dedicated endpoint through Together AI. This is a quick guide that shows you how to do this through our UI or CLI.

​
Requirements

Currently, we support models that meet the following criteria.
  • Source: We support uploads from Hugging Face or S3.
  • Type: We support text generation and embedding models.
  • Scale: We currently only support models that fit in a single node. Multi-node models are not supported when you upload a custom model.

​
Model upload via S3 presigned URL

When uploading a model to Together using an S3 presigned URL, the URL must point to a single archive file containing the model files.

​
Supported archive formats

The presigned URL must reference one of the following archive types:
  • .zip
  • .tar
  • .tar.gz

​
Required archive structure

The archive must contain the model files laid out in standard Hugging Face model repository format. The files should exist at the root of the archive, not nested inside an extra top-level directory. A valid archive will look like this when extracted: 
config.json
generation_config.json
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
special_tokens_map.json
tokenizer.json
tokenizer_config.json

​
Creating an archive from a Hugging Face model directory

If you already have a Hugging Face model directory on disk, make sure you archive the directory contents, not the individual files and not the directory itself. The key detail is to archive the current directory using . or / so that files are at the root of the archive.

​
Presigned URL requirements

  • The presigned URL must point to the archive file in S3.
  • The presigned URL expiration time must be at least 100 minutes.

​
Important notes

  • Do not upload individual files. Only a single archive file is supported.
  • Do not include additional wrapping directories inside the archive.
  • The structure must be compatible with Hugging Face from_pretrained loading.
If the archive does not follow this format, model upload or loading may fail.

​
Preparing and Uploading Your Model Archive

This section walks you through the complete process of downloading a model, creating an archive, uploading it to S3, and generating a presigned URL.

​
Prerequisites

  • AWS CLI installed and configured with appropriate credentials
  • An S3 bucket with write access
  • (Optional) Hugging Face CLI if downloading models from Hugging Face (hf auth login)

​
Step 1: Prepare your model directory

You can either use a model you already have locally, or download one from Hugging Face. Option A: Use an existing local directory If you already have a model directory on disk (e.g., from training or a previous download), skip to Step 2. Just ensure your directory contains the model files in standard Hugging Face format (e.g., config.json, *.safetensors, tokenizer.json, etc.). Option B: Download from Hugging Face Use the Hugging Face CLI to download a model to a local directory: 
hf download <repo_name> --local-dir <local_dir_path>
For example, to download a specific model: 
hf download meta-llama/Llama-2-7b-hf --local-dir ./llama-2-7b
If you only need specific files (e.g., safetensors weights): 
hf download adept/fuyu-8b model-00001-of-00002.safetensors --local-dir ./fuyu

​
Step 2: Create the archive

Navigate to your model directory and create a .tar.gz archive. The key is to archive the contents from within the directory so files are at the root level of the archive: 
cd <local_dir_path>
tar -czvf ../<archive_name>.tar.gz .
For example: 
cd ./llama-2-7b
tar -czvf ../llama-2-7b.tar.gz .
Using . ensures the files are placed at the root of the archive, not nested inside an extra directory. This is required for the model to load correctly.

​
Step 3: Upload the archive to S3

Upload the archive to your S3 bucket: 
aws s3 cp <archive_name>.tar.gz s3://<bucket>/<path>/<archive_name>.tar.gz
For example: 
aws s3 cp llama-2-7b.tar.gz s3://my-models-bucket/uploads/llama-2-7b.tar.gz

​
Step 4: Generate a presigned URL

Create a presigned URL with sufficient expiration time. The URL must be valid for at least 100 minutes: 
aws s3 presign s3://<bucket>/<path>/<archive_name>.tar.gz --expires-in 7200
For example (2 hours = 7200 seconds): 
aws s3 presign s3://my-models-bucket/uploads/llama-2-7b.tar.gz --expires-in 7200
This command outputs a presigned URL that you can use in the Together AI model upload flow via the UI or API.

​
Example: End-to-end workflow

Here’s a complete example downloading and uploading a model: 
# 1. Download model from Hugging Face
hf download openai-community/gpt2 --local-dir ~/.gpt2

# 2. Create archive from within the model directory
cd ~/.gpt2
tar -czvf ../gpt2.tar.gz .
cd ..

# 3. Upload to S3
aws s3 cp gpt2.tar.gz s3://my-bucket/models/gpt2.tar.gz

# 4. Generate presigned URL (valid for 2 hours)
aws s3 presign s3://my-bucket/models/gpt2.tar.gz --expires-in 7200
Use the generated presigned URL as the model source when uploading via the UI or API.

​
Getting Started

​
Upload the model

Model uploads can be done via the UI, API or the CLI. The API reference can be found here.

​
UI

To upload via the web, just log in and navigate to models > add custom model to reach this page:
Upload model
Then fill in the source URL (S3 or Hugging Face), the model name and how you would like it described in your Together account once uploaded.

​
CLI

Upload a model from Hugging Face or S3: 
together models upload \
  --model-name <your_model_name> \
  --model-source <path_to_model_or_repo> \
  --model-type <model_or_adapter> \
  --hf-token <your_HF_token_if_uploading_from_HF> \
  --description <description_of_your_model>

​
Checking the status of your upload

When an upload has been kicked off, it will return a job id. You can poll our API using the returned job id until the model has finished uploading. 
curl -X GET "https://api.together.ai/v1/jobs/{jobId}" \
     -H "Authorization: Bearer $TOGETHER_API_KEY" \
     -H "Content-Type: application/json" \
The output contains a “status” field. When the “status” is “Complete”, your model is ready to be deployed.

​
Deploy the model

Uploaded models are treated like any other dedicated endpoint models. Deploying a custom model can be done via the UI, API or the CLI. The API reference can be found here.

​
UI

All models, custom and finetuned models as well as any model that has a dedicated endpoint will be listed under My Models. To deploy a custom model: Select the model to open the model page.
My Models
The model page will display details from your uploaded model with an option to create a dedicated endpoint.
Create Dedicated Endpoint
When you select ‘Create Dedicated Endpoint’ you will see an option to configure the deployment.
Create Dedicated Endpoint
Once an endpoint has been deployed, you can interact with it on the playground or via the API.

​
CLI

After uploading your model, you can verify its registration and check available hardware options. List your uploaded models: 
bash CLI together models list
View available GPU SKUs for a specific model: 
together endpoints hardware --model <model-name>
Once your model is uploaded, create a dedicated inference endpoint: 
together endpoints create \
  --display-name <endpoint-name> \
  --model <model-name> \
  --gpu h100 \
  --no-speculative-decoding \
  --no-prompt-cache \
  --gpu-count 2
After deploying, you can view all your endpoints and retrieve connection details such as URL, scaling configuration, and status. List all endpoints: 
bash CLI together endpoints list
Get details for a specific endpoint: 
together endpoints get <endpoint-id>