CLI
Reference this guide to learn how to run inference using the command-line interface.
In this tutorial, we will teach you how to use the command-line interface to run models. We will be querying the RedPajama-INCITE-7B-Base
model to complete the phrase "Space robots are".
For the full API reference go to API Reference.
Pre-requisites
- Make sure you have Python installed in your CLI.
- Obtain an API key from Together to access the Inference API by signing up.
Install the Library
Launch your terminal and install or update the Together CLI by executing the following command:
pip install --upgrade together
Authenticate
The API Key can be configured by setting the TOGETHER_API_KEY
environment variable, like this:
export TOGETHER_API_KEY=xxxxx
Find your API token in your account settings.
Select your Model
The default model we are using for this guide is togethercomputer/RedPajama-INCITE-7B-Base
. Browse all available models on this list or by executing this command:
$ together models list
If your desired model isn't listed, feel free to request a model.
Run your First Completion
Once you've installed the Together CLI library, authenticated, and selected a model, let's run your first completion by execution the following command:
together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base
š Congrats! You've run a model using the CLI.
Commands for Inference
Below you can find all the available commands in the CLI for inference:
# list commands
together --help
# list available models
together models list
# start a model
together models start togethercomputer/RedPajama-INCITE-7B-Base
# create completion
together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base
# start a chat
together chat -m togethercomputer/RedPajama-INCITE-7B-Base
# generate an image from a prompt
together image "Space robots" --model SG161222/Realistic_Vision_V3.0_VAE
# check which models are running
together models instances
# stop a model
together models stop togethercomputer/RedPajama-INCITE-7B-Base
# list commands
together --help
# list available models
together models list
# start a fine-tuned model for inference
together models start <finetuned-model-name>
# create completion
together complete "Space robots are" -m togethercomputer/RedPajama-INCITE-7B-Base
# start a chat
together chat -m togethercomputer/RedPajama-INCITE-7B-Base
# generate an image from a prompt
together image "Space robots" --model SG161222/Realistic_Vision_V3.0_VAE
# check which models are running
together models instances
# stop a model
together models stop togethercomputer/RedPajama-INCITE-7B-Base
Updated 24 days ago