Create an evaluation job

# Docs for v1 can be found by changing the above selector ^
from together import Together
import os

client = Together(
    api_key=os.environ.get("TOGETHER_API_KEY"),
)

response = client.evals.create(
    type="classify",
    parameters=ParametersEvaluationClassifyParameters(
        judge=ParametersEvaluationClassifyParametersJudge(
            model="openai/gpt-oss-120b",
            model_source="serverless",
            system_template="You are an expert evaluator...",
        ),
        input_data_file_path="file-abc123",
        labels=["good", "bad"],
        pass_labels=["good"],
        model_to_evaluate="Qwen/Qwen3.5-9B"
    )
)

print(response.workflow_id)

{
  "workflow_id": "eval-1234-1244513",
  "status": "pending"
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "param": null,
    "code": null
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "param": null,
    "code": null
  }
}

POST

evaluation

# Docs for v1 can be found by changing the above selector ^
from together import Together
import os

client = Together(
    api_key=os.environ.get("TOGETHER_API_KEY"),
)

response = client.evals.create(
    type="classify",
    parameters=ParametersEvaluationClassifyParameters(
        judge=ParametersEvaluationClassifyParametersJudge(
            model="openai/gpt-oss-120b",
            model_source="serverless",
            system_template="You are an expert evaluator...",
        ),
        input_data_file_path="file-abc123",
        labels=["good", "bad"],
        pass_labels=["good"],
        model_to_evaluate="Qwen/Qwen3.5-9B"
    )
)

print(response.workflow_id)

{
  "workflow_id": "eval-1234-1244513",
  "status": "pending"
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "param": null,
    "code": null
  }
}

{
  "error": {
    "message": "<string>",
    "type": "<string>",
    "param": null,
    "code": null
  }
}

Authorizations

Authorization

string

header

default:default

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

type

enum<string>

required

The type of evaluation to perform

Available options:

classify,

score,

compare

Example:

"classify"

parameters

object

required

Type-specific parameters for the evaluation

Option 1
Option 2
Option 3

Show child attributes

Response

Evaluation job created successfully

workflow_id

string

The ID of the created evaluation job

Example:

"eval-1234-1244513"

status

enum<string>

Initial status of the job

Available options:

pending

Get model limits

Get all evaluation jobs

⌘I

TOGETHER APIS

SDK LIBRARIES

ERROR CODES

Authorizations

Body

Response