Examples
Use these examples to learn best practices for using the inference API.
Sentiment Analysis
You can use togethercomputer/RedPajama-INCITE-7B-Base
for classification tasks with longer context. In this guide, you will use the model to classify the sentiment of any sentence as either positive, negative, mixed, or neutral.
- Begin the prompt with an instruction describing the task.
Label the sentences as either "positive", "negative", "mixed", or "neutral":
- After the instruction, provide a few examples with known labels.
Sentence: I can say that there isn't anything I would change.
Label: positive
Sentence: I'm not sure about this.
Label: neutral
Sentence: I liked some parts but I didn't like other parts.
Label: mixed
Sentence: I think the background image could have been better.
Label: negative
- Finally, append the tweet that the model should classify and prompt the model to complete the label.
Sentence: I really like it.
Label:
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
endpoint = 'https://api.together.xyz/inference'
TOGETHER_API_KEY = os.getenv('TOGETHER_API_KEY')
res = requests.post(endpoint, json={
"model": 'togethercomputer/RedPajama-INCITE-7B-Base',
"prompt": """\
Label the sentences as either "positive", "negative", "mixed", or "neutral":
Sentence: I can say that there isn't anything I would change.
Label: positive
Sentence: I'm not sure about this.
Label: neutral
Sentence: I liked some parts but I didn't like other parts.
Label: mixed
Sentence: I think the background image could have been better.
Label: negative
Sentence: I really like it.
Label:""",
"top_p": 1,
"top_k": 40,
"temperature": 0.8,
"max_tokens": 1,
"repetition_penalty": 1,
}, headers={
"Authorization": f"Bearer {TOGETHER_API_KEY}",
"User-Agent": "<YOUR_APP_NAME>"
})
print(res.json()['output']['choices'][0]['text']) # ' positive'
Topic classification
RedPajama-INCITE-Instruct-7B can be used to classify the topic of text passages. In this guide, you will use the model to classify the topic of a news article as either World, Sports, Business, or Sci/Tech.
- Begin the prompt with an instruction describing the task.
Given a news article, classify its topic.
Possible labels: 1. World 2. Sports 3. Business 4. Sci/Tech
- After the instruction, provide a few examples with known labels.
Article: A nearby star thought to harbor comets and asteroids now appears to be home to planets, too.
Label: Sci/Tech
Article: Soaring crude prices plus worries about the economy and the outlook for earnings are expected to hang over the stock market next week during the depth of the summer doldrums.
Label: Business
- Finally, append the tweet that the model should classify and prompt the model to complete the label.
Article: Murtagh a stickler for success Northeastern field hockey coach Cheryl Murtagh doesn't want the glare of the spotlight that shines on her to detract from a team that has been the America East champion for the past three years and has been to the NCAA tournament 13 times.
Label:
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
endpoint = 'https://api.together.xyz/inference'
res = requests.post(endpoint, json={
"model": "togethercomputer/RedPajama-INCITE-Instruct-7B",
"prompt": """
Given a news article, classify its topic.
Possible labels: 1. World 2. Sports 3. Business 4. Sci/Tech
Article: A nearby star thought to harbor comets and asteroids now appears to be home to planets, too.
Label: Sci/Tech
Article: Soaring crude prices plus worries about the economy and the outlook for earnings are expected to hang over the stock market next week during the depth of the summer doldrums.
Label: Business
Article: Murtagh a stickler for success Northeastern field hockey coach Cheryl Murtagh doesn't want the glare of the spotlight that shines on her to detract from a team that has been the America East champion for the past three years and has been to the NCAA tournament 13 times.
Label: """,
"top_p": 1,
"top_k": 40,
"temperature": 0.8,
"max_tokens": 1,
"repetition_penalty": 1,
}, headers={
"Authorization": "Bearer <YOUR_API_KEY>",
"User-Agent": "<YOUR_APP_NAME>"
})
print(res.json())
Question answering
RedPajama-INCITE-Instruct-7B can be used to answer questions with reasoning on a user-provided context.
- Begin the prompt by providing the context.
Daniel travelled to the kitchen.
Mary grabbed the milk.
Daniel went back to the hallway.
John moved to the kitchen.
Daniel travelled to the office.
John grabbed the apple.
John journeyed to the garden.
John dropped the apple.
John moved to the hallway.
Mary dropped the milk.
Daniel went back to the bathroom.
Mary grabbed the milk there.
Mary went back to the garden.
Mary went to the hallway.
Mary left the milk.
John journeyed to the bedroom.
- Then, add a question followed by "Question:" and prompt the model to answer the question.
Question: Where was the milk before the hallway?
Answer:
- Sample response:
Answer: garden
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
endpoint = 'https://api.together.xyz/inference'
res = requests.post(endpoint, json={
"model": "togethercomputer/RedPajama-INCITE-7B-Instruct",
"prompt": """\
Daniel travelled to the kitchen.
Mary grabbed the milk.
Daniel went back to the hallway.
John moved to the kitchen.
Daniel travelled to the office.
John grabbed the apple.
John journeyed to the garden.
John dropped the apple.
John moved to the hallway.
Mary dropped the milk.
Daniel went back to the bathroom.
Mary grabbed the milk there.
Mary went back to the garden.
Mary went to the hallway.
Mary left the milk.
John journeyed to the bedroom.
Question: Where was the milk before the hallway?
Answer: """,
"top_p": 1,
"top_k": 40,
"temperature": 0.1,
"max_tokens": 1,
"repetition_penalty": 1,
}, headers={
"Authorization": "Bearer <YOUR_API_KEY>",
"User-Agent": "<YOUR_APP_NAME>"
})
print(res.json())
Troubleshooting code
You can use RedPajama-INCITE-Chat-7B for troubleshooting your code.
- For example, begin the prompt by providing an error message you have.
I got "ImportError: No module named matplotlib", how to solve it?
- Sample response
Make sure that matplotlib is installed on your computer. You can check by running the following command in your terminal:
```
pip install matplotlib
```
If the above command does not work, try the following:
```
sudo apt-get install python-pip
pip install matplotlib
```
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
url = "https://api.together.xyz/inference"
payload = {
"model": "togethercomputer/togethercomputer/RedPajama-INCITE-7B-Chat",
"prompt": "I got \"ImportError: No module named matplotlib\", how to solve it?",
"max_tokens": 128,
"stop": ".",
"temperature": 0.1,
"top_p": 0.7,
"top_k": 50,
"repetition_penalty": 1
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"Authorization": "Bearer <YOUR_API_KEY>",
"User-Agent": "<YOUR_APP_NAME>",
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)
Writing Python functions
You can use RedPajama-INCITE-Chat-7B for writing a Python function.
- For example, begin the prompt by providing an error message you have.
Generate a Python function to calculate the factorial of a number.
- Sample response
Here's a Python function to calculate the factorial of a number:
```
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n - 1)
```
This function takes a numeric argument n and returns the factorial of n, which is the product of all integers i such that 0 <= i < n. The base case is when n is 0, which has a factorial of 1. The recursive case calculates the product of all integers i such that 0 <= i < n, including n. n.
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
url = "https://api.together.xyz/inference"
payload = {
"model": "togethercomputer/togethercomputer/RedPajama-INCITE-7B-Chat",
"prompt": "Generate a Python function to calculate the factorial of a number.",
"max_tokens": 256,
"stop": ".",
"temperature": 0.1,
"top_p": 0.7,
"top_k": 50,
"repetition_penalty": 1
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"Authorization": "Bearer <YOUR_API_KEY>",
"User-Agent": "<YOUR_APP_NAME>",
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)
HTML Page
You can use RedPajama-INCITE-Chat-7B for generating a simple HTML page.
- For example, begin the prompt by providing a task you want to perform in HTML
Generate a simple HTML page showing "Hello world!"
- Sample response
Here's an example of a simple HTML page with the text "Hello World!":
```
<!DOCTYPE html>
<html>
<head>
<title>Hello World!</title>
</head>
<body>
<h1>Hello World!</h1>
</body>
</html>
```
- Send the prompt to the API with any appropriate parameters. The code below shows an example in Python using the
requests
package.
import requests
url = "https://api.together.xyz/inference"
payload = {
"model": "togethercomputer/togethercomputer/RedPajama-INCITE-7B-Chat",
"prompt": "Generate a simple HTML page showing \"Hello world!\"",
"max_tokens": 256,
"stop": ".",
"temperature": 0.1,
"top_p": 0.7,
"top_k": 50,
"repetition_penalty": 1
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"Authorization": "Bearer <YOUR_API_KEY>",
"User-Agent": "<YOUR_APP_NAME>",
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)
Further reading
Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. The Prompt Engineering Guide is a great introduction to the subject.
Updated 3 months ago