Quickstart

Follow this guide to deploy a model for dedicated inference, send it a request, and scale it down when you’re done.

Requirements

Before you begin, make sure you have:

Created an account and generated an API key.
Set your API key as an environment variable in your terminal.
Installed the Together CLI:

# Install
uv tool install "together[cli]"

# Upgrade
uv tool upgrade "together[cli]"

# List commands
tg --help

The dedicated model inference commands require Together CLI version 2.24.0 or later. Check your version with tg --version.

In CI, agents, or other environments where the CLI cannot prompt for confirmation, select a project before deploying. Run tg whoami to find your project ID, then set TOGETHER_PROJECT_ID or pass --project <project_id> to the command.

Step 1: Deploy a model

Deploy google/gemma-4-E4B-it, one of the supported models Together hosts. The deployment provisions in the background: for a model this size, first-time provisioning usually takes about 5 to 10 minutes while the weights download and hardware is allocated, and larger models take longer.

CLI
Console

The endpoints deploy command creates an endpoint, attaches a deployment on the model’s default hardware, and routes all traffic to it:

tg beta endpoints deploy google/gemma-4-E4B-it \
  --endpoint quickstart-endpoint

The command returns as soon as the resources are created and prints the endpoint’s details:

√ Model deployed to endpoint your-project-slug/quickstart-endpoint.

╭─ Endpoint Details for quickstart-endpoint ───────────────────────────────────╮
│    Endpoint string  your-project-slug/quickstart-endpoint                    │
│       Endpoint ID  ep_abc123                                                 │
│        Created at  07/13/2026, 06:52 PM                                      │
│        Updated at  07/13/2026, 06:52 PM                                      │
│        Visibility  Private                                                   │
│           Web URL  https://api.together.ai/endpoints/ep_abc123               │
╰──────────────────────────────────────────────────────────────────────────────╯

Deployments
╭──────────────────────────────────────┬───────────────────┬───────────────────╮
│  Deployment                          │  Model            │                   │
├──────────────────────────────────────┼───────────────────┼───────────────────┤
│  Name:                               │  google/gemma-4…  │    Status:        │
│  google-gemma-4-E4B-it-BF16-abc123…  │                   │  Provisioning     │
│    ID: dep_abc123                    │                   │  Replicas: 0 / 1  │
╰──────────────────────────────────────┴───────────────────┴───────────────────╯

Note the endpoint string (your-project-slug/quickstart-endpoint): pass it as the model parameter when you send requests.Check the deployment’s status with the deployment ID from the output, and wait for DEPLOYMENT_STATE_READY:

tg beta endpoints get dep_abc123

Open the create form

Go to the Endpoints page and select New endpoint.

Name the endpoint and deployment

In Endpoint name, enter quickstart-endpoint. In Deployment name, enter quickstart-deployment. The console requires a deployment name, unlike the CLI, which generates one for you.

Choose the model

Under Model, select google/gemma-4-E4B-it, and leave the default Quantization.

Keep the defaults

Leave the default Hardware and Region, and keep Min replicas and Max replicas at 1.

Create the endpoint

Select Create endpoint. You land on the endpoint’s page, where the deployment starts in Provisioning and moves to Ready once it’s live.

Copy the endpoint string shown on the endpoint’s page (in the form your-project-slug/quickstart-endpoint): this is what you pass as the model parameter when you send requests.

The Create endpoint form in the Together AI console, configured with the model Gemma 4 E4B IT, BF16 quantization, 1x NVIDIA-H100-80GB hardware, Any region, and min and max replicas both set to 1. The Summary panel shows an estimated $5.49 per hour and a Create endpoint button.

Step 2: Send a request

Point the inference base URL at https://api-inference.together.ai/v1, pass the endpoint string as the model parameter, and use the same request shape as a serverless model:

from together import Together

client = Together(base_url="https://api-inference.together.ai/v1")

response = client.chat.completions.create(
    model="your-project-slug/quickstart-endpoint",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    max_tokens=512,
)

print(response.choices[0].message.content)

import Together from 'together-ai';

const client = new Together({
  baseURL: 'https://api-inference.together.ai/v1',
});

const response = await client.chat.completions.create({
  model: 'your-project-slug/quickstart-endpoint',
  messages: [{ role: 'user', content: 'What is 2+2?' }],
  max_tokens: 512,
});

console.log(response.choices[0].message.content);

curl -s -X POST https://api-inference.together.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-project-slug/quickstart-endpoint",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "max_tokens": 512
  }' | jq .

You should see output similar to this:

{
  "id": "8d43d9055fa845b2b1ad11b946191a7e",
  "object": "chat.completion",
  "created": 1782839730,
  "model": "your-project-slug/quickstart-endpoint",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "2 + 2 = 4."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 74,
    "completion_tokens": 13,
    "total_tokens": 87
  }
}

Congrats! You deployed and called your first dedicated model on Together AI.

Step 3: Clean up resources

Dedicated model inference bills per minute per running replica, so tear down what you deployed once you’re done.

CLI
Console

Pass the endpoint ID (ep_abc123) from the deploy output to rm with --force to delete the endpoint and its deployment in one step:

tg beta endpoints rm ep_abc123 --force

To stop charges without deleting anything (for example, to redeploy later), scale the deployment to zero instead, then re-run the status command above to confirm it reaches DEPLOYMENT_STATE_STOPPED.

Next steps

Concepts

Understand the resource model and development workflow.

Manage deployments

Create, scale, stop, and delete endpoints and deployments.

Configure autoscaling

Autoscale a deployment on the metric that fits your workload.

Upload a model

Deploy a model you fine-tuned from a supported base model.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

GPU CLUSTERS

FINE-TUNING

CODE EXECUTION

ADMINISTRATION

Requirements

Step 1: Deploy a model

Step 2: Send a request

Step 3: Clean up resources

Next steps

Concepts

Manage deployments

Configure autoscaling

Upload a model

​Requirements

​Step 1: Deploy a model

​Step 2: Send a request

​Step 3: Clean up resources

​Next steps

Concepts

Manage deployments

Configure autoscaling

Upload a model

Requirements

Step 1: Deploy a model

Step 2: Send a request

Step 3: Clean up resources

Next steps