I was involved last year and earlier this year in building tech stack for Wand AI. More specifically their main product offering Wandy. Building an AI product as capable as Wandy in the fast paced startup environment was a great learning for me. Wandy uses a very wide array of technologies, like Kubernetes, GCP Cloud Functions, NextJS hosted at Vercel, Firestore, Supabase and much more. Some of my learnings are,
1. Retrieval Augmented Generation(RAG) solves a lot of Enterprise use cases
RAG is golden standard when it comes to solving business use cases, such as finding some information like HR policy, summarization of large unstructred informaiton etc. The best thing about RAG is that its fairly easy to setup, all you need is a vector database, as ChromaDB(Open Source), PineCone(SaaS) or Qdrant DB(Open Source).
2. Summarization is a must
We used a lot of summarization techniques to manage long and short term memory of Wandy. One thing we realized fairly early on was that if we are using tools such as Python Code Execution, Internet Search etc, we were hitting the limit of 128k tokens very regularly. In order to keep the context of the conversation alive and keeping Wandy on topic, we employed summarizations, TODOs to get outstanding results from Wandy.
3. Multiagent Tool calls
We built many tools for Wandy, from simple tools such as Internet Search, Gmail and Calendar tools to very complex tools such as LaTeX based PDF report generator. We often found ourself writing a bunch of multiagent workflows, like agents to write each section of report and later putting everything in a fluid way. With the tools such as Autogen or CrewAI, writing multiagentic flows are simpler than ever today.
4. Event driven Backend Systems
We used tasks and queues extensively along with our cloud Functions, this allowed us to scale our infrastructure on demand. Also even driven backend actually helped us to keep everything responsive on the frontend.
Conclusion
I am immensely grateful to Wand Team and in particular Rotem, for letting me be part of Wand Development team. Checkout more about my work at Wand.