Python SDK

To switch to using the Together API, simply switch out the API key to your Together API key, base_url to https://api.together.xyz/v1, and model to one of our chat models.

import os
import openai

system_content = "You are a travel agent. Be descriptive and helpful."
user_content = "Tell me about San Francisco"

client = openai.OpenAI(
    api_key=os.environ.get("TOGETHER_API_KEY"),
    base_url="https://api.together.xyz/v1",
    )
chat_completion = client.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=[
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ],
    temperature=0.7,
    max_tokens=1024,
)
response = chat_completion.choices[0].message.content
print("Together response:\n", response)

Streaming

This example demonstrates how you can use the OpenAI Python SDK to stream tokens:

import os
import openai

system_content = "You are a travel agent. Be descriptive and helpful."
user_content = "Tell me about San Francisco"

client = openai.OpenAI(
    api_key=os.environ.get("TOGETHER_API_KEY"),
    base_url="https://api.together.xyz/v1",
    )

stream = client.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=[
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ],
    stream=True,
    max_tokens=1024,
    stop=['</s>']
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Node.js

This example shows how you can use the OpenAI Node.js SDK.

const OpenAI = require("openai");

const openai = new OpenAI({
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
});

async function run() {
  const chatCompletion = await openai.chat.completions.create({
    messages: [
      {role: "system", content: "You are an AI assistant"},
      {role: "user", content: "Who won the world series in 2020?"},
    ],
    model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
    max_tokens: 1024,
  });
  console.log(chatCompletion.choices[0].message.content);
}

run();

Streaming in Node.js

const OpenAI = require("openai");

const openai = new OpenAI({
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
});

async function run() {
    const stream = await openai.chat.completions.create({
        messages: [
                {role: "system", content: "You are an AI assistant"},
                {role: "user", content: "Who won the world series in 2020?"},
        ],
        model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
        max_tokens: 1024,
        stream: true,
    });
    for await (const chunk of stream) {
        console.log(chunk.choices[0].text);
      }
    }
    
run();

Response structure

An example Chat Completions API response looks as follows:

{
  "id": "83489fc92fe4faf8-SJC",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "The Los Angeles Dodgers won the World Series in 2020."
      }
    }
  ],
  "created": 1702411968,
  "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
  "object": "chat.completion"
}

The assistant’s response can be extracted with:

response['choices'][0]['message']['content']

Community Libraries

The Together API is also supported by most OpenAI libraries built by the community!

import openai

client = OpenAI(api_key=TOGETHER_API_KEY,
        base_url="https://api.together.xyz/v1")

def get_embeddings(texts, model="togethercomputer/m2-bert-80M-32k-retrieval"):
   texts = [text.replace("\n", " ") for text in texts]
   outputs = client.embeddings.create(input = texts, model=model)
   return [outputs.data[i].embedding for i in range(len(texts))]

return get_embeddings(texts)