Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.together.ai/llms.txt

Use this file to discover all available pages before exploring further.

Reference for managing live batch jobs: checking status, downloading outputs and error files, cancelling, and listing. For an end-to-end walkthrough, see Run a batch job.
Python examples require together>=2.0.0

​
Check batch status

batches.retrieve() returns the batch object directly (no .job wrapper, unlike create()). 
from together import Together

client = Together()

batch = client.batches.retrieve("batch-xyz789")

print(batch.status)
print(batch.progress)
StatusDescription
VALIDATINGThe input file is being validated before the batch can begin.
IN_PROGRESSRequests are being processed.
COMPLETEDAll requests processed; results available.
FAILEDProcessing failed.
EXPIREDThe job exceeded its time limit.
CANCELLEDThe job was cancelled.
Poll every 30 to 60 seconds. Tighter loops will hit rate limits without giving the server time to make progress.

​
Retrieve results

When the batch reaches COMPLETED, download the output file referenced by output_file_id. Per-request failures are stored separately in error_file_id. Always download both: a COMPLETED batch can still contain individual request failures. 
from together import Together

client = Together()

batch = client.batches.retrieve("batch-xyz789")

if batch.output_file_id:
    with client.files.with_streaming_response.content(
        id=batch.output_file_id,
    ) as response:
        with open("batch_output.jsonl", "wb") as f:
            for chunk in response.iter_bytes():
                f.write(chunk)

if batch.error_file_id:
    with client.files.with_streaming_response.content(
        id=batch.error_file_id,
    ) as response:
        with open("batch_errors.jsonl", "wb") as f:
            for chunk in response.iter_bytes():
                f.write(chunk)
Output and error lines are both keyed by the custom_id from your input, so you can reconcile them with a single pass over each file. Line order does not match input order.

​
Cancel a batch

You can cancel a batch while it is VALIDATING or IN_PROGRESS. Requests that have already completed before the cancellation are still billed, and their responses are still returned in the output file. 
from together import Together

client = Together()

client.batches.cancel("batch-xyz789")

​
List batches

from together import Together

client = Together()

for batch in client.batches.list():
    print(batch.id, batch.status)

​
Errors

​
Error codes

CodeDescriptionSolution
400Invalid request formatCheck JSONL syntax and required fields.
401Authentication failedVerify your API key.
404Batch not foundCheck the batch ID.
429Rate limit exceededReduce request frequency.
500Server errorRetry with exponential backoff.

​
Error file format

Each line in the error file pairs a custom_id from your input with the failure reason:
batch_errors.jsonl
{"custom_id": "req-1", "error": {"message": "Invalid model specified", "code": "invalid_model"}}
{"custom_id": "req-5", "error": {"message": "Request timeout", "code": "timeout"}}