Skip to main content

About WrenAI and Our Mission

WrenAI is dedicated to reimagining how businesses can interact with and leverage their data through LLM, by bringing comprehension capabilities to small and large data teams.

Why Now?​

In the rapidly evolving data landscape, data analysts play a pivotal role as the vital bridge between the data and the diverse business contexts within an organization. Different business units, each with unique perspectives and requirements, often seek specific insights from data, making the role of data analysts both critical and challenging. Their ability to interpret, translate, and communicate data in a way that aligns with the distinct needs of various stakeholders is indispensable.

The advent of advanced technologies such as Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) is revolutionizing this space by augmenting the capabilities of business data analysts. RAG further enhances this process by integrating retrieved external information, enabling LLMs to generate more comprehensive and accurate information.

With their understanding of context and natural language processing abilities, LLMs with RAG empower analysts to navigate and interpret vast datasets efficiently and in nuance.

Challenges of Using RAG with LLMs to Query Database​

Using RAG coupled with LLMs to query databases is not a new concept. Many solutions have been proposed to tackle this problem, but they still face challenges in four crucial phases: context collection, retrieval, SQL generation, and collaboration. Despite the attempts to overcome these challenges, we are still grappling with them in all four phases.

challenges

Phase 1: Context Collection Challenges​

  • Interoperability Across Diverse Sources: To generalize and normalize searched and integrated information seamlessly across varied sources, metadata services, and APIs.
  • Complex Linking of Data and Metadata: This involves associating data with its metadata in a document store. It involves storing metadata, schema, and context, such as relationships, calculations, and aggregations.

Phase 2: Retrieval Challenges​

  • Optimization of Vector Stores: Developing and implementing optimization techniques for vector stores, such as indexing and chunking, are critical for enhancing search efficiency and precision.
  • Precision in Semantic Search: The challenge lies in the nuances of comprehension of queries in the context, which can significantly affect the accuracy of the results. This usually involves techniques such as query rewrite, re-ranker, etc.

Phase 3: SQL Generation Challenges​

  • Accuracy and Executability of SQL Queries: Generating SQL queries that are both accurate and executable poses a significant challenge. This requires the LLM to have an in-depth understanding of SQL syntax, database schemas, and the specific dialects of different database systems.
  • Adaptation to Query Engine Dialects: Databases often have unique dialects and nuances in SQL implementation. Designing LLMs that can adapt to these differences and generate compatible queries across various systems adds another layer of complexity to the challenge.

Phase 4: Collaboration Challenges​

  • Collective Knowledge Accumulation: The challenge lies in creating a mechanism that can effectively gather, integrate, and utilize the collective insights and feedback from a diverse user base to enhance the accuracy and relevance of the data retrieved by LLM.
  • Access Control: While we are finally retrieving the data, the next most important challenge is ensuring that the existing organizational data access policies and privacy regulations also apply to the new LLM and RAG architecture.

Introducing WrenAI​

We have some core design philosophies that were used when developing WrenAI.

design

  • Interactive Experience: WrenAI is designed to engage users in a dialogue, clarifying their queries and refining results in real time. This interactivity ensures that the generated SQL queries accurately reflect the user's intent, enhancing precision and overall experience and making the data querying process more intuitive and user-friendly.
  • Explainability: WrenAI ensures that every SQL query generated in natural language is accurate, concise, and reliable. Understanding the rationale behind each query and its outcome is crucial. This transparency empowers users, enabling them to trust and verify the insights derived from their data.
  • Interoperability: Interoperability is foundational to WrenAI's architecture. It enables users to query data from multiple sources without dealing with the complexities of different data formats and dialects, providing a standard interface across different sources.
  • Continuous Learning: Continuous learning is the cornerstone of WrenAI's advancement. WrenAI will proactively learn through ongoing query history, feedback, and interactions. Incorporating new patterns, information, and data structures into our LLM knowledge base.

What's Next?​

Moving forward, while we're working on implementations of WrenAI, we also have some ideas on areas we could explore and improve in the near future.

  • Proactive Collaboration:
    • AI agents not only report progress in real time but actively seek feedback.
    • Agents ask human advisors (e.g., senior data analysts and data engineers) for guidance to clarify complex definitions or knowledge gaps.
  • Collective Knowledge Sharing:
    • Facilitates an ecosystem where insights and learning are shared, not siloed, across the organization, enriching the AI's knowledge base.