rag – Shekhar Gulati

I enjoy reading books on Oreilly learning platform https://learning.oreilly.com/ . For the past month, a new feature on the Oreilly platform called “Answers” has been staring me down, and I haven’t been tempted to click it. Maybe it’s LLM fatigue, or something else I just didn’t give it a try. I do use LLM tools daily but most of these tools I have designed for myself around my workflows.

Today, I decided to give it a try. If you go to a book page like the one I am reading currently https://learning.oreilly.com/library/view/hands-on-large-language/9781098150952/ you will see Answers icon in the right side bar.

When you click on Answers it will show a standard Chat input box and suggestions. We all have seen them million times by now.

It looks like a standard Retrieval Augmented Generation (RAG) use case. When you ask a question it will search in its knowledge base(some sort of Vector/Hybrid search) and then generate the answer.

Creating an AI assistant that generate helpful answers from a knowledge base is a complex problem. A significant hurdle is the frequent mismatch between how users ask questions and how information is structured within the data. Most people struggle to ask good questions. This often results in irrelevant or incomplete answers, frustrating users.

As builders of these systems we should not expect users to write well-crafted queries. In our application, we have implemented query rewriting to rephrase user queries to better align with the underlying data. This has dramatically improved the accuracy and helpfulness of our AI assistant responses.

In this post I will share details on how we implemented query rewriting in our application. We will end the post by looking at how popular open source systems do query rewriting.

You can hear this blog in podcast format here – https://notebooklm.google.com/notebook/ed6e648e-c95c-4ad8-88a2-767be02c7c4d/audio

Tag: rag

Oreilly Answers: A case study of poorly designed LLM powered RAG system

Query Rewriting in RAG Applications