This is a submission for theNylas Challenge: AI Expedition & Galaxy Brain

What I Built and Why

MailTales is a web app designed to breathe life into your email inbox. With the help ofNylas & AI, MailTales offers unique and creative ways to interact with your emails, transforming them into bite-sized stories, engaging summaries, and even providing insights through sentiment analysis. You can try ithttps://mailtales.amitwani.dev

Demo

MailTales: You can go tohttps://mailtales.amitwani.dev

Features

Chat Over Your Inbox: With MailTales, you can chat with your email inbox!

AI-Powered Email Summaries: MailTales provides concise, AI-generated summaries of your emails

Listen to AI-Generated Stories: MailTales can convert the content of your emails into engaging audio stories.

Sentiment Analysis and Categorization: MailTales' sentiment analysis feature helps you understand the tone of your messages. The app also categorizes emails based on their content.

Code

Tech Stack

Full Stack: NextJS
Backend: NodeJS
Deployment: Vercel, Fly.io
Database: PostgreSQL
Queues: Upstash Kafka
Email: Nylas
AI: Gemini

Architecture

Nylas: Nylas is used for authentication of user and fetching their emails from inbox
Gemini: Gemini is used for generating AI responses as well as generating embeddings. Model used for generating responses isgemini-1.5-flash and for embeddings it istext-embedding-004
Postgres: Postgres is used to store user and emails data.@vercel/postgres is used for this.
Upstash Kafka: Upstash Kafka is used for the job of generating email embeddings. Generating email embeddings is done by a NodeJS Backend by consuming messages from Upstash Kafka topic.

Your Journey

Nylas

I have usedNylas to get email inbox and its contents. I have used theNylas NodeJS SDK for this. Logging in the user was very easy using the SDK. ThegrantId received is then saved to the DB for further requests. Overall it was really interesting experience in building on top ofNylas features. There are many more features which are still left to explore!

Get Emails

constqueryParams:ListMessagesQueryParams={in:["INBOX"],limit:limit,}if(pageToken&&pageToken.length>0){queryParams.pageToken=pageToken;}constmessages=awaitnylas.messages.list({identifier:grantId!,queryParams,});

Search Emails

constqueryParams:ListMessagesQueryParams={in:["INBOX"],limit:limit,searchQueryNative:"query"}if(pageToken&&pageToken.length>0){queryParams.pageToken=pageToken;}constmessages=awaitnylas.messages.list({identifier:grantId!,queryParams,});

Upstash Kafka

Used Upstash Kafka's REST API to produce message.

constresponse=awaitfetch(`${process.env.UPSTASH_KAFKA_REST_URL}/produce`,{method:'POST',headers:{Authorization:`Basic${process.env.UPSTASH_KAFKA_REST_AUTH}`},body:JSON.stringify({topic:"generate-email-embeddings",value:user.id})})

Usedkafkajs in the backend to consume the messages

constkafka=newKafka({brokers:[`${process.env.UPSTASH_KAFKA_BROKERS}`],ssl:true,sasl:{mechanism:'scram-sha-256',username:`${process.env.UPSTASH_KAFKA_USERNAME}`,password:`${process.env.UPSTASH_KAFKA_PASSWORD}`},logLevel:logLevel.ERROR,});constconsumer=kafka.consumer({groupId:'consumer_group_1'});construn=async()=>{awaitconsumer.connect();awaitconsumer.subscribe({topic:'generate-email-embeddings',fromBeginning:true});awaitconsumer.run({eachMessage:async({topic,partition,message})=>{console.log({partition,offset:message.offset,value:message.value?.toString(),});constuserId=Number(message.value?.toString());console.log("Generating email embeddings for user",userId);awaitgenerateEmailEmbeddings(userId);},});};

Gemini & Vercel AI SDK

Used Vercel AI SDK to interact with Gemini models.

Get Streaming Response

exportasyncfunctiongetGoogleResponse(body:any,system:string="",tools:any=null){constresult=awaitstreamText({model:google('gemini-1.5-flash'),system,messages:body,temperature:0.0001,tools,});returnresult;}

Generate Embeddings

exportasyncfunctiongenerateEmbeddings(emailStrings:string[]){const{embeddings}=awaitembedMany({model:google.embedding('text-embedding-004'),values:emailStrings,});returnembeddings;}

Also, this was the first time of using NextJS and building a proper RAG agent from the scratch using Vercel AI SDK, so overall a great learning experience.

Future Scope

• More deeper insights into mailbox
• Integration of AI in alot of parts of the app
• Crafting beautiful professional replies.

Conclusion

It was a great experience in building this app usingNylas &Gemini over NextJS. I have learnt alot in the last week while building this project. Thanks toDev.To as well for providing such platforms for building the projects.