Agent

This example demonstrates how to integrate Composio’s Gmail tool with Weaviate to create an agent that will automatically respond to Gmail messages by searching a Weaviate collection for relevant information and using that information to draft replies. A common use case for this might be to automate email support using an agent that has access to your docs/FAQs. Composio allows an AI agent/your app to easily connect to your user's apps like Gmail, Slack, Trello etc. Paired together with Weaviate to store, manage and query data, more personalized and context-aware agents can be created.

Setup and Dependencies

Composio Setup

In the command line, run -

Follow the link from your command line to set up an integration between your gmail account and Composio.

Also run -

This command enables a trigger for when a new mail is received in gmail. When a new email arrives on the connected account, the trigger provides data about the mail like the sender, mail content etc.

Set OpenAI api key

This tutorial uses an embedding model and LLM from OpenAI, for which you will need an API key set as an evironment variable.

Next, we connect to a Weaviate instance. Here, we're using Weaviate Embedded. This embedded instance will stay alive for as long as the parent application is running. For a more persistent instance, you can look at the docs here.

Creating Weaviate Collection

Let's create a collection to store our data.

Now that we have a collection, we can import whatever data we want -

After importing the data, we can get our collection -

Then, we initialize Composio's tools for Gmail -

The GMAIL_REPLY_TO_THREAD action represents a function that the LLM can call. Composio provides these actions with optimized JSON schemas which makes it very easy to integrate external tools with LLMs/agentic workflows.

We are also going to define a tool that queries our Weaviate collection based on the question asked by the user -

Setting up the Agent

Next, we create a listener for the trigger we created above -

Listener started!

data received - appName='gmail' payload={'threadId': '1916dc9e35a3401p', 'messageId': '1916dc9e35a3401p', 'messageTimestamp': '2024-08-20T03:16:59Z', 'messageText': 'I wanted to know how generative ai works for art. Can i generate images\r\nfrom gen ai models?\r\n', ...}

> Entering new AgentExecutor chain...

Invoking: `search_document` with `{'query': 'how generative ai works for art and generating images'}`


Search Result: 1:
text:OpenAI Codex.
Producing high-quality visual art is a prominent
application of generative AI.[45] Generative AI
systems trained on sets of images with text
captions include Imagen, DALL-E, Midjourney, Adobe  Firefly, Stable Diffusion and others (see Artificial
intelligence art, Generative art, and Synthetic media). They are commonly used for text-to-image generationpage:2.0source:Generative_artificial_intelligence.pdf
Search Result: 2:
text:Théâtre D'opéra Spatial, an image
generated with Midjourney
Generative artificial intelligence
Generative artificial intelligence (generative AI, GenAI,[1]
or GAI) is artificial intelligence capable of generating text,
images, videos, or other data using generative models,[2]
often in respons e to prompts.[3][4] Generative AI models
learn the patterns and structure of their input training data andpage:0.0source:Generative_artificial_intelligence.pdf
Search Result: 3:
text:artistic works. By the early 1970s , Harold Cohen was creating and exhibiting generative AI works created
by AARON, the computer program Cohen created to generate paintings.[26]
Markov chains have long been used to model natural langua ges since their development by Russian
mathematician Andrey Markov in the early 20th century. Markov published his first paper on the topic inpage:1.0source:Generative_artificial_intelligence.pdf

Invoking: `gmail_reply_to_thread` with `{'thread_id': '1916dc9e35a3401p', 'message_body': 'Dear John,\n\nThank you for your inquiry about generative AI and its application in art. \n\nGenerative AI works by utilizing models that can generate text, images, videos, or other data based on training data. In the context of art, it involves systems trained on large datasets of images paired with text captions. Some popular generative AI models for image generation include DALL-E, Midjourney, and Stable Diffusion. These models learn the patterns and structures from the training data, allowing them to create new and unique images in response to prompts.\n\nSo yes, you can definitely generate images using generative AI models!\n\nBest regards,\n[Your Name]', ...}

[2024-08-20 08:48:08,645][INFO] Executing action: GMAIL_REPLY_TO_THREAD
{'execution_details': {'executed': True}, 'response_data': {'id': '1916dcac6c9424e9', 'threadId': '1916dc9e35a3401p', 'labelIds': ['SENT']}}I have replied to John's inquiry about how generative AI works for art and the possibility of generating images using generative AI models. If you need any further assistance, feel free to ask!

> Finished chain.

Connect with Composio and learn more

If you encounter any problems, please let us know at our Discord.

Check out Composio's documentation to learn more about how to use and integrate various tools for different usecases.