Query Analytics for RAG-Based Systems
Elelem's Analytics dashboard is a comprehensive tool designed for organizations looking to gain actionable insights from their Retrieval-Augmented Generation (RAG) based AI applications and agents. By providing a comprehensive view of system performance, user behavior, and content effectiveness, it enables developers, knowledge base owners and business analysts to optimize their AI-driven solutions.
.png)
01
Query Analysis
Users can view and analyze incoming queries, helping to understand user intent and identify trends in usage patterns
02
Retrieval Performance Metrics
Enabling users to assess and improve the accuracy and relevance of retrieved information
03
Content Gap Analysis
Identify missing or inadequate content by analyzing user queries that yield unsatisfactory results, enabling continuous improvement of the knowledge base(s)
04
Clustering Capabilities
Queries are automatically clustered by topic and subtopic, providing a high-level view of common themes and topics in user interactions
“A conservative estimate for failed GenAI projects lies between $4.5 billion and $10.5 billion annually”
ELELEM Analytics Dashboard
The results of the analysis are displayed in a web browser. Key sections include:
Application Tracker
Overview of applications accessing the RAG pipeline and knowledge bases. All subsequent elements can be filtered to include only selected applications or agents
Gap Analysis
Gap analysis: missing or inadequate content by topic, with sample queries
Query Clusters
Query cluster summaries, showing the top topics (by frequency) that are being queried by users or agents
Content Suggestions
Content suggestions: content modifications with the highest ROI opportunity, based on the frequency of queries on these topics and the quality of existing coverage
Top Performing
Top content accessed from the knowledge base(s), with hyperlinks where applicable
Content Relevancy
Relevance over time: an at-a-glace overview of the performance
Implementation Overview
Telematics are sent from the RAG-based system to Elelem’s BigQuery Database on Google Cloud Platform (GCP), where they are analyzed automatically. As illustrated in the schematic diagram, the telematics are sent either by:
1. Integrating Elelem’s lightweight SDK into the GenAI application(s) ​​
​
2. Enabling the Elelem plugin within the RAG backend system (e.g. Databricks, Dataiku, Vertex AI, ElasticSearch, Cohere)
