It feels good to get back to writing and sharing some thoughts from the data trenches. Lately, I've been spending time deep-diving into how we're building the next generation of enterprise AI agents, particularly focusing on grounding them in our own data using the RAG (Retrieval Augmented Generation) pattern. And honestly, the recent leaps forward, especially within Azure AI Search, are truly game-changing.
For the past couple of years, RAG has been our go-to for making LLMs actually useful by connecting them to our internal knowledge. We used the tools we had – a search engine to find relevant text chunks and feed them to the model. This got us some solid initial wins and solved real problems. But let's be honest, as business needs and user questions get more complex, the simple "single-shot search, feed to LLM" approach just doesn't cut it anymore. Complex, multi-part queries, references back to previous conversation threads – these break the simpler pattern.
This is where I'm super excited to see Microsoft announce Agentic Retrieval!
This is where things get seriously interesting. The evolution from traditional search to agentic retrieval within Azure AI Search is a massive step forward for handling complex scenarios. This isn't just about finding the top-K results in one go. Instead, an internal AI agent takes the query, plans how to tackle it, can decompose it into multiple sub-questions, execute searches (even in parallel!), and then rank & merge the results. Think of it as the search engine itself getting smarter about the nuances of real-world queries. This approach is already showing significant gains – up to 40% higher answer relevance and 30% better result rates for complex questions in evaluations. This is crucial for building truly intelligent and dependable AI agents.
But having a smarter retrieval engine is only half the battle. The data you feed it matters, a lot. Real-world business data isn't neatly packaged text; it's full of diagrams, images, and complex layouts that are often missed by simple text processing. This missing context can cripple an AI agent. With this new announcement from Microsoft, data ingestion capabilities are catching up to reality. New features are rolling out to handle this complexity during the ingestion process itself, including extracting detailed layout information and using techniques like image verbalisation. This ensures that the AI agent gets a richer, more complete understanding of the multi-modal content it's working with. Add to this the growing set of native connectors for bringing in data from diverse sources like Blob Storage, ADLS Gen2, Cosmos DB, SQL, and OneDrive, and the process of building these rich ingestion pipelines becomes much quicker.
And now for the non-negotiable part - security and access control. Building powerful AI agents that can access vast amounts of organisational knowledge is amazing, but they must respect existing data permissions. Manually trying to manage user permissions, group expansions, and filtering at the application level can be error-prone and dangerous. This is why native Entra-based document access control directly within Azure AI Search is such a big deal. It handles the complex work of group expansion and filtering automatically, ensuring that a user only sees results derived from documents they actually have permission to access. The ability to automatically propagate access permissions directly from sources like Azure Data Lake Storage Gen2 simplifies things even further. Plus, there's emerging support for respecting document sensitivity labels from Microsoft Purview, adding another layer of critical security by enforcing policies defined within the documents themselves.
In a nutshell, I'm super excited to see the whole Azure AI world improve. I've not played around a tonne and I'm kind of glad... It was all too easy to get stuck on the RAG single shot, only to be told that you can just 'fix all that in a CLI'. Most businesses do not have experts who can write command line style code. However, what Microsoft has shown through their demo (see here) is that with a few "wizard-style" setup screens, you can set up AI agents MUCH easier than ever before. It's a fantastic time to be working in data and AI!
