> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Deploy a model for inference on dedicated GPUs.

> [!TIP]
>
> Using a coding agent? Install the [together-ai](https://github.com/togethercomputer/skills/tree/main/skills/together-ai) skill to let your agent deploy and manage dedicated endpoints automatically. See [agent skills](https://docs.together.ai/docs/agent-skills.md) for details.

Dedicated model inference (DMI) lets you serve a model on reserved hardware, providing several advantages over [serverless models](https://docs.together.ai/docs/serverless/overview.md):

* **Better performance:** Dedicated GPUs provide higher throughput, lower latency, and more predictable performance.
* **No hard rate limits:** You're only limited by the capacity of your selected hardware, plus the bounds of your autoscaling configuration.
* **Fine-tuned models:** Deploy a model you fine-tuned from a supported base model.
* **Cost-efficient at scale:** DMI bills per-GPU-minute, which is cheaper at high utilization than serverless models (which bill per-token).

Dedicated model inference uses the same [inference APIs](https://docs.together.ai/docs/inference/overview.md#shared-inference-api) as serverless models, so you can prototype on serverless, then deploy on DMI without changing your application code.

> [!TIP]
>
> If you're running a stock model in production and want a defined SLA without managing hardware, contact sales for [provisioned throughput](https://docs.together.ai/docs/inference/provisioned-throughput.md).

You can deploy an endpoint from the console, through the API, or with the [Together CLI](https://docs.together.ai/reference/cli/endpoints.md):

```bash theme={null}
# deploy a model to a new endpoint
tg endpoints deploy google/gemma-4-E4B-it --endpoint my-endpoint

# check the status until it's READY
tg endpoints get <ENDPOINT_ID>

# delete the endpoint when you're done
tg endpoints rm <ENDPOINT_ID> --force
```

The [quickstart](https://docs.together.ai/docs/dedicated-endpoints/quickstart.md) breaks down this flow and shows how to run inference against the endpoint.

## Get started

> [card] [Quickstart](https://docs.together.ai/docs/dedicated-endpoints/quickstart.md)
>
> Deploy and call your first endpoint in a few minutes.

> [card] [Concepts](https://docs.together.ai/docs/dedicated-endpoints/concepts.md)
>
> The DMI resource model and development workflow.

> [card] [Manage deployments](https://docs.together.ai/docs/dedicated-endpoints/manage.md)
>
> Create, scale, stop, and delete your deployments.

> [card] [Supported models](https://docs.together.ai/docs/dedicated-endpoints/models.md)
>
> Browse the list of Together-hosted models you can deploy.

> [card] [Upload a fine-tuned model](https://docs.together.ai/docs/dedicated-endpoints/custom-models.md)
>
> Deploy a model you fine-tuned from a supported base model.

> [card] [Upload a LoRA adapter](https://docs.together.ai/docs/dedicated-endpoints/adapter.md)
>
> Import a custom low-rank adapter (LoRA) from Hugging Face or S3 and serve it.

> [card] [Multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md)
>
> Attach post-training adapters to one deployment and choose one per request.

> [card] [Migrate from v1](https://docs.together.ai/docs/dedicated-endpoints/migrate-from-v1.md)
>
> Migrate a dedicated endpoint to the new DMI resource model.

## Project scope

The `tg endpoints` and `tg models` commands, the management API, and the SDKs' `client.endpoints` and `client.models` methods operate within a [Together AI project](https://docs.together.ai/docs/projects.md). The CLI reads the project from the `TOGETHER_PROJECT_ID` environment variable, or you can pass `--project` on any command. If neither is set, it uses the project you selected with `tg login` or `tg switch`, or the project associated with your API key.

In the SDKs, pass `project_id` to `Together()` in Python or `projectID` to `new Together()` in TypeScript, or set `TOGETHER_PROJECT_ID`. Otherwise, read the project ID from `client.whoami()` before project-scoped API calls. For a project API key, the response's `type` is `project_api_key` and the ID is in `project.id`.

```bash theme={null}
export TOGETHER_PROJECT_ID=your_project_id
```

## Development workflow

To create a deployment, run the `tg endpoints deploy` command, passing the model and endpoint name. This creates an endpoint, attaches a deployment, and routes all traffic to it:

```bash theme={null}
tg endpoints deploy google/gemma-4-E4B-it \
  --endpoint my-endpoint
```

The command prints the new endpoint's **endpoint string** (`<project_slug>/<endpoint_name>`): pass it as the `model` parameter on inference requests.

Once the deployment is ready, send inference requests to the endpoint using the same [inference API](https://docs.together.ai/docs/inference/overview.md#shared-inference-api) as serverless models. Pass the endpoint string as the `model` parameter:

```python Python {6} theme={null}
from together import Together

client = Together(base_url="https://api-inference.together.ai")

response = client.chat.completions.create(
    model="your-project-slug/my-endpoint",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    max_tokens=512,
)

print(response.choices[0].message.content)
```

```typescript TypeScript {8} theme={null}
import Together from 'together-ai';

const client = new Together({
  baseURL: 'https://api-inference.together.ai',
});

const response = await client.chat.completions.create({
  model: 'your-project-slug/my-endpoint',
  messages: [{ role: 'user', content: 'What is 2+2?' }],
  max_tokens: 512,
});

console.log(response.choices[0].message.content);
```

```bash cURL {5} theme={null}
curl -s -X POST https://api-inference.together.ai/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-project-slug/my-endpoint",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "max_tokens": 512
  }'
```

`deploy` bundles the endpoint, deployment, and traffic-routing steps for you. You can also [run those steps individually](https://docs.together.ai/docs/dedicated-endpoints/manage.md), or drive them from the Python and TypeScript SDKs.

For a step-by-step walkthrough, [follow the quickstart](https://docs.together.ai/docs/dedicated-endpoints/quickstart.md). For more details on the DMI resource model, see [Concepts](https://docs.together.ai/docs/dedicated-endpoints/concepts.md).

## Key features

* **Deploy any supported model:** Run a [Together-hosted model](https://docs.together.ai/docs/dedicated-endpoints/models.md), a [model you fine-tuned on Together](https://docs.together.ai/docs/fine-tuning/overview.md), or [a fine-tuned model you upload](https://docs.together.ai/docs/dedicated-endpoints/custom-models.md).
* **Autoscale on demand:** [Scale your deployments with replicas](https://docs.together.ai/docs/dedicated-endpoints/scaling.md) to meet demand, and stop them when you don't need them to reduce costs.
* **Split traffic across deployments:** Host multiple deployments behind one endpoint URL and [route requests](https://docs.together.ai/docs/dedicated-endpoints/split-traffic.md) between them by weight.
* **Compare deployments on live traffic:** Run an [A/B test](https://docs.together.ai/docs/dedicated-endpoints/ab-tests.md) with control and variant splits to measure a candidate against a baseline before you promote it.
* **Migrate deployments with rollouts:** Shift live traffic from one deployment to another in [controlled, metric-gated steps](https://docs.together.ai/docs/dedicated-endpoints/rollouts.md) without changing the endpoint URL.
* **Monitor endpoints:** Track latency, throughput, and utilization in [built-in dashboards](https://docs.together.ai/docs/dedicated-endpoints/monitoring.md), and trace lifecycle changes through the events feed.

## Pricing

Dedicated model inference [bills per minute by hardware](https://docs.together.ai/docs/dedicated-endpoints/pricing.md) while a deployment runs, regardless of model or request volume. Each running replica bills independently and stops billing as soon as it scales down.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.