There's a big shift coming for Business Intelligence. One where the use of dashboards and reports become obsolete and is replaced by the use of LLMs via data agents. Moving forward, users interact with said data agents to ask questions and generate visuals that can help tell the story. As a result, BI engineers and developers must move further downstream to help the LLM and said data agent understand the organization's data and provide the guardrails that allow it to flourish.

The introduction to AI or LLMs embedded in almost every application used by an organization has highlighted a major gap in seeking the extract value from their use - the need for them to have the right context in order to accurately provide answers. This is the emergence of the Context Layer in the BI/analytics/data stack. The Context Layer is a new abstraction layer whose purpose is to tie together all of the different components that make up an organization's data in writing. Yes - that's the big difference now when it comes to using AI/LLMs - writing is back!

BI engineers and developers will become writers in the development of their deliverables to users. What used to result in a dashboard or report that met user requirements and was supported for adoption turns into a simple "hey the LLM is ready for your questions and use, let us know if you encounter any hallucinations." Before getting to that step, modeling the data remains the most important function of the BI engineer. Models continue to be developed and metrics, measures, and KPIs are built within the model. Once the model is built, the context of the model needs to be provided to ensure appropriate interaction between the user and the model. This becomes an exercise of getting into the users head and asking what questions they might ask. And places even more importance of capturing user questions during the requirements gathering process. Their questions should be answered not through visuals but in writing so that the LLM can understand what it needs to pull and analyze before spitting out an answer.

Providing context means writing down all of the idiosyncrasies of an organizations. Acronyms, explanations to key words and key phrases used by various branches of the business, Excel files, relationships between different data points (think about how much is inferred through our conversations but not explicit when viewing the data), and questions that need to be answered. In a way, it's a giant Q&A document stored in the backend. And so the BI engineer spends a substantial portion of their time developing the context and validating that their words are translating to accurate answers provided by the LLM.

This shift also opens up a new market and the growing importance of tools that seek to create a context layer. Because where should this context live? In text files stored on Sharepoint the LLM is pointed to? exclusively in Power BI semantic models? There will be a need to store the context and make it accessible. One could develop that using the LLM itself. An option that I've been exploring is Microsoft's Ontology feature which I believe is their first foray into enabling users to create a context layer and makes use realize that data leaders seeking to stay on top of AI development need to start thinking about the context of their data and how to connect it all together.

Another component to the mapping of data and LLMs are the multiple LLMs existing within an organization. We all know there's not just one but many that are being used by all different individuals. Whether those are embedded within tools found in common engineering stacks or standalone models where all users export their thinking or ask their questions, there is a need for governance (One LLM To Rule Them All - future post 😉 ). We've lived through the fragmented siloed data landscape where the preeminent problem was "Hey your data is siloed across various databases. Let's bring it all together in a data warehouse/lakehouse." Next comes the fragmented siloed LLM landscape where the problem will be "Hey your agents don't talk to each other. Let's bring them all together in a new AI unification layer." Data leaders need to think through how to stop the spread of agents and model use within their organizations.

At this point, I should most likely shift my terminology to "agents" and more specifically "data agents" whose task will be to retrieve data for users or create reporting for them. I will reserve a deeper dive into how these agents can be best served across the data stack for a future post. For now, the focus is on the shift that is coming to data teams and Business Intelligence teams.

Data Engineers will continue to focus on ingesting data into centralized repositories. BI Engineers will continue to model the data. And there will be a focus on writing the context around all of the data that is accrued across the enterprise. The rise of Context Engineers?

Additionally data leaders and data teams will shift their long-term focus (we should be doing this now to be fair as we are a couple of years in) on managing the various LLMs and agents living within the enterprise ecosystem. Frameworks need to be established that help users use them and prevent privacy and security issues. With the framework, the layer that ensures agents and LLMs talk to each other, keeping themselves informed of each other's activities, potentially checking themselves on accuracy and anything else that might need to be shared to represent a shared context across the enterprise. We do not want to be in a situation where LLMs provide different answers yet we are undoubtedly heading in that direction.

There's a big shift coming and I'll seek to guide us through it over my next posts - stay tuned!