CLI

Reference this guide to learn how to run inference using the command-line interface.

Suggest Edits

In this tutorial, we will teach you how to use the command-line interface to run models. We will be querying the RedPajama-INCITE-7B-Base model to complete the phrase "Space robots are".


For the full API reference go to API Reference.


Pre-requisites

  • Make sure you have Python installed in your CLI.
  • Obtain an API key from Together to access the Inference API by signing up.

Install the Library

Launch your terminal and install or update the Together CLI by executing the following command:

pip install --upgrade together

Authenticate

The API Key can be configured by setting the TOGETHER_API_KEY environment variable, like this:

export TOGETHER_API_KEY=xxxxx

Find your API token in your account settings.

Select your Model

The default model we are using for this guide is togethercomputer/RedPajama-INCITE-7B-Base. Browse all available models on this list or by executing this command: 

$ together models list

If your desired model isn't listed, feel free to request a model.

Run your First Completion

Once you've installed the Together CLI library, authenticated, and selected a model, let's run your first completion by execution the following command:

together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base

šŸŽ‰ Congrats! You've run a model using the CLI.

Commands for Inference

Below you can find all the available commands in the CLI for inference:

# list commands
together --help

# list available models
together models list

# start a model
together models start togethercomputer/RedPajama-INCITE-7B-Base

# create completion
together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base

# start a chat
together chat -m togethercomputer/RedPajama-INCITE-7B-Base

# generate an image from a prompt
together image "Space robots" --model SG161222/Realistic_Vision_V3.0_VAE

# check which models are running
together models instances

# stop a model
together models stop togethercomputer/RedPajama-INCITE-7B-Base

# list commands
together --help

# list available models
together models list

# start a fine-tuned model for inference
together models start <finetuned-model-name>

# create completion
together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base

# start a chat
together chat -m togethercomputer/RedPajama-INCITE-7B-Base

# generate an image from a prompt
together image "Space robots" --model SG161222/Realistic_Vision_V3.0_VAE

# check which models are running
together models instances

# stop a model
together models stop togethercomputer/RedPajama-INCITE-7B-Base

Updated 24 days ago