Quickstart

Dedicated container inference runs your own Docker container on Together’s managed GPU infrastructure. You bring the container, and Together handles provisioning, autoscaling, networking, and observability. Use it when you need to run your own model or inference code, such as a custom architecture, a modified vLLM or SGLang engine, or an audio, image, or video model, without managing GPU servers yourself. In this tutorial, you’ll deploy a minimal hello-world worker built with the Sprocket SDK, send it a job through the queue, read the result, and delete the deployment.

Requirements

Dedicated containers access: Contact your account representative or [email protected] to enable dedicated containers for your organization.
Docker: For building and pushing container images. See the Docker installation guide.
uv: For installing the Together CLI. See uv’s installation guide.

Step 1: Install the Together CLI

uv tool install "together[cli]" --upgrade

Authenticate the CLI:

tg login

Step 2: Clone the Sprocket examples

git clone [email protected]:togethercomputer/sprocket.git
cd sprocket

The hello-world worker, included in sprocket/examples/hello_world, is a minimal Sprocket that returns a greeting:

import os
import sprocket


class HelloWorld(sprocket.Sprocket):
    def setup(self) -> None:
        self.greeting = "Hello"

    def predict(self, args: dict) -> dict:
        name = args.get("name", "world")
        return {"message": f"{self.greeting}, {name}!"}


if __name__ == "__main__":
    queue_name = os.environ.get("TOGETHER_DEPLOYMENT_NAME", "hello-world")
    sprocket.run(HelloWorld(), queue_name)

Step 3: Build and deploy

Deployments can be configured with a pyproject.toml file. The deployment name, set by the configuration, must be globally unique. The example worker uses this pyproject.toml configuration:

[project]
name = "hello-world"
version = "0.1.0"
dependencies = ["sprocket"]

[[tool.uv.index]]
name = "together-pypi"
url = "https://pypi.together.ai/"

[tool.uv.sources]
sprocket = { index = "together-pypi" }

[tool.jig.image]
python_version = "3.11"
cmd = "python3 hello_world.py --queue"
copy = ["hello_world.py"]

[tool.jig.deploy]
gpu_type = "none"
gpu_count = 0
cpu = 1
memory = 2
storage = 10
port = 8000
min_replicas = 1
max_replicas = 1

Change the project name in pyproject.toml and use this name for the rest of the tutorial.

Navigate to the example worker and deploy:

cd examples/hello-world
tg beta jig deploy

This command:

Builds the Docker image from the example.
Pushes it to Together’s private registry.
Creates a deployment on Together’s GPU infrastructure.

Step 4: Watch deployment status

watch 'tg beta jig status'

Wait until the deployment shows running and replicas are ready. Press Ctrl+C to stop watching. watch is not installed by default on macOS. Install it with brew install watch or your package manager of choice.

You can also view the status of your deployments from the Together AI web console.

Step 5: Test the health endpoint

curl https://api.together.ai/v1/deployment-request/hello-world/health \
  -H "Authorization: Bearer $TOGETHER_API_KEY"

Expected response:

{"status": "healthy"}

Step 6: Submit a job

curl -X POST "https://api.together.ai/v1/queue/submit" \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hello-world",
    "payload": {"name": "Together"},
    "priority": 1
  }'

Response:

{
  "request_id": "req_abc123"
}

Copy the request_id for the next step.

Step 7: Get the job result

curl "https://api.together.ai/v1/queue/status?model=hello-world&request_id=req_abc123" \
  -H "Authorization: Bearer $TOGETHER_API_KEY"

Real request IDs use UUIDv7 format (e.g., 019ba379-92da-71e4-ac40-d98059fd67c7). Replace req_abc123 with your actual request ID from the submit response.

Response (when complete):

{
  "request_id": "req_abc123",
  "model": "hello-world",
  "status": "done",
  "outputs": {"message": "Hello, Together!"}
}

Step 8: View logs

Stream logs from your deployment:

tg beta jig logs --follow

Step 9: Clean up

When you’re done, delete the deployment:

tg beta jig destroy

Next steps

Now that you’ve deployed your first container, explore the full platform:

Dedicated container inference overview

Read the architecture and core concepts.

Jig CLI

Build, push, and deploy containers, and manage secrets and volumes.

Sprocket SDK

Build queue-integrated inference workers.

API reference

Call the REST API for deployments, secrets, and queues.

Tutorials

Image generation with FLUX.2

Run single-GPU inference with 4-bit quantization.

Video generation with Wan 2.1

Run multi-GPU inference with torchrun.

Serve an OpenAI-compatible endpoint

Run in HTTP server mode behind the OpenAI Images API.

GET STARTED

SERVERLESS

INFERENCE APIS

DEDICATED MODEL INFERENCE

DEDICATED CONTAINER INFERENCE

POST-TRAINING

FINE-TUNING

GPU CLUSTERS

CODE EXECUTION

ADMINISTRATION

Requirements

Step 1: Install the Together CLI

Step 2: Clone the Sprocket examples

Step 3: Build and deploy

Step 4: Watch deployment status

Step 5: Test the health endpoint

Step 6: Submit a job

Step 7: Get the job result

Step 8: View logs

Step 9: Clean up

Next steps

Dedicated container inference overview

Jig CLI

Sprocket SDK

API reference

Tutorials

Image generation with FLUX.2

Video generation with Wan 2.1

Serve an OpenAI-compatible endpoint

​Requirements

​Step 1: Install the Together CLI

​Step 2: Clone the Sprocket examples

​Step 3: Build and deploy

​Step 4: Watch deployment status

​Step 5: Test the health endpoint

​Step 6: Submit a job

​Step 7: Get the job result

​Step 8: View logs

​Step 9: Clean up

​Next steps

Dedicated container inference overview

Jig CLI

Sprocket SDK

API reference

​Tutorials

Image generation with FLUX.2

Video generation with Wan 2.1

Serve an OpenAI-compatible endpoint

Requirements

Step 1: Install the Together CLI

Step 2: Clone the Sprocket examples

Step 3: Build and deploy

Step 4: Watch deployment status

Step 5: Test the health endpoint

Step 6: Submit a job

Step 7: Get the job result

Step 8: View logs

Step 9: Clean up

Next steps

Tutorials