Chat & image generation — Superagent

Multi-modality is something we have seen more. AI Assistants that are able to generate text, video, images and audio can be super powerful. This tempplate aims to show how you can build an AI Assistant that can both generate text and images. We will be using the newly released Playground v2.

Prerequisites

A Superagent account and API key
A Replicate account and API key.

Demo

Setup

We will start by installing the Superagent SDK inside of your project.

$ poetry add superagent-py
> # or 
> pip install superagent-py

Next, we will instantiate a new client.

1 import os
2 
3 from superagent.client import Superagent
4 
5 client = Superagent(
6     token=os.environ["SUPERAGENT_API_KEY"], # replace with your Superagent API
7     base_url="https://api.beta.superagent.sh" # or your local environment
8 )

With our client setup we can now go ahead and create a Replicate tool and corresponding agent.

1 promt = """You are a helpful AI Assistant that can use Playground AI API to generate images.
2 
3 Follow these steps:
4 
5 1. Ask the user what image they want to create
6 2. Create a generative ai prompt scene based on the user input. Ask if the prompt is OK before proceeding.
7 3. Use the prompt to generate an image with Playground AI.
8 """
9 tool = client.tool.create(request={
10     "name": "Playground AI API", 
11     "description": "Useful for generating an image", 
12     "type": "REPLICATE",
13     "metadata": {
14         "model": "playgroundai/playground-v2-1024px-aesthetic:42fe626e41cc811eaf02c94b892774839268ce1994ea778eba97103fe1ef51b8",
15         "apiKey": os.environ["REPLICATE_API_KEY"],
16         "arguments": {"width": 1024, "height": 1024}
17     }
18 })
19 
20 agent = client.agent.create(request={
21     "name": "Playground AI Assistant",
22     "description": "An assistant that can generate images",
23     "isActive": True,
24     "avatar": "" # Optional avatar url jpg/png
25     "prompt": prompt,
26     "llmModel": "GPT_4_0613"
27 })
28 
29 # Connect the tool to the agent
30 client.agent.addTool(agent_id=agent.data.id, tool_id: tool.data.id)

If you log in to your Superagent agent account you should see an Agent created. Clicking on it should render an agent similar to the image below:

You now have the option to use the SDKs to invoke the agent or use the Superagent UI to start a chat.

1 output = client.agent.invoke(
2     agent_id=agent.data.id,
3     input="Generate an image of a cat",
4     enable_streaming=False,
5     session_id="my_session_id"
6 )
7 
8 print(output)

Full code

Below you will find the full code for this assistant. You can optionaly use the following NextJS template adapt it.

1 import os
2 
3 from superagent.client import Superagent
4 
5 client = Superagent(
6     token=os.environ["SUPERAGENT_API_KEY"], # replace with your Superagent API
7     base_url="https://api.beta.superagent.sh" # or your local environment
8 )
9 
10 promt = """You are a helpful AI Assistant that can use Playground AI API to generate images.
11 
12 Follow these steps:
13 
14 1. Ask the user what image they want to create
15 2. Create a generative ai prompt scene based on the user input. Ask if the prompt is OK before proceeding.
16 3. Use the prompt to generate an image with Playground AI.
17 """
18 tool = client.tool.create(request={
19     "name": "Playground AI API", 
20     "description": "Useful for generating an image", 
21     "type": "REPLICATE",
22     "metadata": {
23         "model": "playgroundai/playground-v2-1024px-aesthetic:42fe626e41cc811eaf02c94b892774839268ce1994ea778eba97103fe1ef51b8",
24         "apiKey": os.environ["REPLICATE_API_KEY"],
25         "arguments": {"width": 1024, "height": 1024}
26     }
27 })
28 
29 agent = client.agent.create(request={
30     "name": "Playground AI Assistant",
31     "description": "An assistant that can generate images",
32     "isActive": True,
33     "avatar": "" # Optional avatar url jpg/png
34     "prompt": prompt,
35     "llmModel": "GPT_4_0613"
36 })
37 
38 # Connect the tool to the agent
39 client.agent.addTool(agent_id=agent.data.id, tool_id: tool.data.id)
40 
41 output = client.agent.invoke(
42     agent_id=agent.data.id,
43     input="Generate an image of a cat",
44     enable_streaming=False,
45     session_id="my_session_id"
46 )
47 
48 print(output)

1	import os
2
3	from superagent.client import Superagent
4
5	client = Superagent(
6	token=os.environ["SUPERAGENT_API_KEY"], # replace with your Superagent API
7	base_url="https://api.beta.superagent.sh" # or your local environment
8	)

1	promt = """You are a helpful AI Assistant that can use Playground AI API to generate images.
2
3	Follow these steps:
4
5	1. Ask the user what image they want to create
6	2. Create a generative ai prompt scene based on the user input. Ask if the prompt is OK before proceeding.
7	3. Use the prompt to generate an image with Playground AI.
8	"""
9	tool = client.tool.create(request={
10	"name": "Playground AI API",
11	"description": "Useful for generating an image",
12	"type": "REPLICATE",
13	"metadata": {
14	"model": "playgroundai/playground-v2-1024px-aesthetic:42fe626e41cc811eaf02c94b892774839268ce1994ea778eba97103fe1ef51b8",
15	"apiKey": os.environ["REPLICATE_API_KEY"],
16	"arguments": {"width": 1024, "height": 1024}
17	}
18	})
19
20	agent = client.agent.create(request={
21	"name": "Playground AI Assistant",
22	"description": "An assistant that can generate images",
23	"isActive": True,
24	"avatar": "" # Optional avatar url jpg/png
25	"prompt": prompt,
26	"llmModel": "GPT_4_0613"
27	})
28
29	# Connect the tool to the agent
30	client.agent.addTool(agent_id=agent.data.id, tool_id: tool.data.id)

1	output = client.agent.invoke(
2	agent_id=agent.data.id,
3	input="Generate an image of a cat",
4	enable_streaming=False,
5	session_id="my_session_id"
6	)
7
8	print(output)