Skip to main content
This guide walks you through deploying a sample inference worker to Together’s managed GPU infrastructure.

​
Prerequisites

  • Together API Key – Required for all operations. Get one from together.ai.
  • Dedicated Containers access – Contact your account representative or [email protected] to enable Dedicated Containers for your organization.
  • Docker – For building and pushing container images. Get it here.
  • uv (optional) – For Python/package management. Install from astral-sh/uv.

​
Step 1: Install the Together CLI

uv tool install together
Set your API key: 
export TOGETHER_API_KEY=your_key_here

​
Step 2: Clone the Sprocket Examples

git clone [email protected]:togethercomputer/sprocket.git
cd sprocket
The example worker is a simple Sprocket that multiplies an input value by 2: 
import asyncio
import logging
import os
import time

import sprocket

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


class ExampleSprocket(sprocket.Sprocket):
    def setup(self) -> None:
        self.multiplier = 2

        for i in range(5):
            time.sleep(1)
            logger.info("Setup in progress... %i", i)

    def predict(self, args: dict) -> dict:
        time.sleep(float(args.get("sleep", 0.5)))
        value = args["multiplicand"] * self.multiplier
        return {"value": value}


if __name__ == "__main__":
    queue_name = os.environ.get(
        "TOGETHER_DEPLOYMENT_NAME", "sprocket-queue-test"
    )
    sprocket.run(ExampleSprocket(), queue_name)

​
Step 3: Build and Deploy

Navigate to the example worker and deploy: 
cd examples/example-worker
together beta jig deploy
This command:
  1. Builds the Docker image from the example
  2. Pushes it to Together’s private registry
  3. Creates a deployment on Together’s GPU infrastructure
Note your deployment name in the pyproject.toml and from the output (you’ll need it for the next steps). The example worker uses this pyproject.toml configuration: 
[project]
name = "sprocket-queue-test"
version = "0.1.0"
dependencies = ["sprocket"]

[[tool.uv.index]]
name = "together-pypi"
url = "https://pypi.together.ai/"

[tool.uv.sources]
sprocket = { index = "together-pypi" }

[tool.jig.image]
python_version = "3.11"
cmd = "python3 example_worker.py --queue"
system_packages = ["git"]
copy = ["example_worker.py"]

[tool.jig.deploy]
description = ""
gpu_type = "none"
gpu_count = 0
cpu = 4
memory = 32  # in GB
storage = 100 # in GB
port = 8000
min_replicas = 1
max_replicas = 10

[tool.jig.deploy.environment_variables]
FOO="bar"

[tool.jig.autoscaling]
profile = "QueueBacklogPerWorker"
targetValue = "1.05" # underprovision by 5%

​
Step 4: Watch Deployment Status

watch 'together beta jig status'
Wait until the deployment shows running and replicas are ready. Press Ctrl+C to stop watching. Note that watch is not installed by default on MacOS, use brew install watch or your package manager of choice.

​
Step 5: Test the Health Endpoint

Replace <your-deployment> with your deployment name: 
curl https://api.together.ai/v1/deployments/<your-deployment>/health \
  -H "Authorization: Bearer $TOGETHER_API_KEY"
Expected response: 
{"status": "healthy"}

​
Step 6: Submit a Job

Replace <your-deployment> with your deployment name: 
curl -X POST "https://api.together.ai/v1/queue/submit" \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<your-deployment>",
    "payload": {"multiplicand": 5, "sleep": 1},
    "priority": 1
  }'
Response: 
{
  "request_id": "019ba379-92da-71e4-ac40-d98059fd67c7",
  "status": "pending"
}
Copy the request_id for the next step.

​
Step 7: Get the Job Result

Replace <your-deployment> and <request-id> with your values: 
curl "https://api.together.ai/v1/queue/status?model=<your-deployment>&request_id=<request-id>" \
  -H "Authorization: Bearer $TOGETHER_API_KEY"
Response (when complete): 
{
  "request_id": "019ba379-92da-71e4-ac40-d98059fd67c7",
  "model": "my-deployment",
  "status": "done",
  "outputs": {"value": 10}
}
The example worker multiplies the input by 2, so multiplicand: 5 returns value: 10.

​
Step 8: View Logs

Stream logs from your deployment: 
together beta jig logs --follow

​
Step 9: Clean Up

When you’re done, delete the deployment: 
together beta jig destroy

​
Next Steps

Now that you’ve deployed your first container, explore the full platform:

​
Example Guides