The other challenge is an information management challenge. This has all of the telltale signs of information spread across different mediums resulting in a fragmented output becoming an obstacle to measurable AI ROI if not addressed. Much like the world of enterprise databases before they were all connected/replaced by lakehouses and warehouses, IT organizations need to start planning for enterprise AI offerings for their users and stakeholders. Easily said and will be an endeavor that reshapes technology teams as the AI vertical is developed through AI engineers and AI-prioritizing leaders.

An organization's biggest challenge will be to not turn away from AI. I know that seems silly given how much everything in our professional lives is now revolving around AI but there cannot be a half step approach. It's either yes in full or get left behind. Both feet into AI, accepting that resistance is futile. Let's remove the thin veil preventing discussions and talk in full confidence between individuals who have a desire to use it or have used it to solve problems. Which raises the question - is using AI wrong? I'd argue that's not the question. The question is: do you want to get left behind? I don't, hence we're here talking about it.

Organizations that don't want to get left behind should consider adopting AI/LLMs into their enterprise at the enterprise level. Not using AI functionality that a vendor is pushing you to use. Not telling users you'll reimburse a Claude Pro license. Instead, finding an agonistic AI tool that gives you general AI use + the potential to connect your applications to. Users want one location to be told "this is where you go to use AI". That solution has to be onboarded just like any other software application used by a company, with security and privacy being top of mind. Once it has been enabled inside and made available, it will require the right governance to avoid misuse and costs spreading like wildfire. Two big components of that governance are models and tokens.

Understanding models and tokens are critical to thinking about AI governance as they drive not only the cost but the results (or quality of the results) for the company and its users. Models will need to be identified and bucketed for specific uses. Activities centered around automating a process should use a more complex model. Agents released to answer simple questions should use the lightest/smallest model available. Engineers building a new app with AI should use the smartest or most robust model that can guide them to an appropriate solution. All different types of uses will need to be categorized and a corresponding model called out for them. If not, the risk of misusing tokens increases dramatically. And I say misuse because leveraging deep thinking models for simple agent-esque questions is how costs start to run out of control. The cheapest model for the highest quality (accurate) answer is what should be strived for.

Having tokens in mind, we can aim for the least amount of tokens used to the highest quality answer. Since we are essentially charged by the amount of text inputted into the LLM, the focus is on the smallest amount of prompts that retrieve the penultimate answer. This is more of an organizational focus that a company's users need to implement but one that greatly determines cost and is potentially mitigated by an enterprise level agreement. Less prompts cost less since LLMs usually reprocess the conversation history with every prompt thus the more you ask questions, the more tokens you are feeding and the more it's going to cost in usage. Another way to minimize cost is leveraging prompt caching. Prompt caching essentially addresses the reprocessing of previous prompts in a conversation by storing it in memory and referencing it when new prompts are entered. The trick here will be identifying the ideal TTL window so that tokens writing to cache are minimized and data is not stale. If you are building an information agent, you might consider caching the context document once per session (when the user initiates their first interaction with the agent).

Both model selections and prompt mindfulness are getting into the weeds of the use and governance of LLMs. Both highlight key elements that organizations need to prioritize as they take on AI. However, both take a back seat to what is the most important aspect of onboarding AI into an organization. And that's ensuring there is one enterprise AI solution that connects all of your data sources, platforms, BI tools, and user excels together. Effectively, in the hierarchy of enterprise applications, AI models sit at the very top because they are going to be the eventual sole point of interaction with users. The enterprise AI models application will be the interaction layer and the action layer. Users won't update excel files anymore, they'll ask the model to update it (using the right model of course). Users won't get into Salesforce or Power BI anymore, they'll tell the AI what they want done and the model will do it. This is the shift of the information management landscape.

From an information management perspective, all other applications are now on the same level - below the AI tools. This is important to understand as it solidifies the need to focus on one AI solution that is capable of handling all of an organizations tools and generated data. If that's not taken into consideration, we will be repeating the fragmented data landscape that all organizations have been working to escape for the last 10 plus years. Organizations cannot afford to fragment their AI tools and hope for consistent, accurate answers received by their users. Getting different results across two enterprise applications that claim to have the same data will be a huge inhibitor to adoption, success and measurable ROI.

Additionally, engineering teams cannot be expected (nor should they be asked) to define and build their own AI tools that facilitate the use of their responsible tech stack. That work is redundant within the scope of thinking enterprise-first and building context within the topmost layer. And that work should be led by a dedicated AI engineering team whose responsibilities will be to enhance and optimize models, build guardrails with model selection and prompting, and imbue context into interactions with models.

Successful AI adoption = prioritized( dedicated AI team + governance + dedicated enterprise AI solution).

The introduction to AI or LLMs embedded in almost every application used by an organization has highlighted a major gap in seeking the extract value from their use - the need for them to have the right context in order to accurately provide answers. This is the emergence of the Context Layer in the BI/analytics/data stack. The Context Layer is a new abstraction layer whose purpose is to tie together all of the different components that make up an organization's data in writing. Yes - that's the big difference now when it comes to using AI/LLMs - writing is back!

BI engineers and developers will become writers in the development of their deliverables to users. What used to result in a dashboard or report that met user requirements and was supported for adoption turns into a simple "hey the LLM is ready for your questions and use, let us know if you encounter any hallucinations." Before getting to that step, modeling the data remains the most important function of the BI engineer. Models continue to be developed and metrics, measures, and KPIs are built within the model. Once the model is built, the context of the model needs to be provided to ensure appropriate interaction between the user and the model. This becomes an exercise of getting into the users head and asking what questions they might ask. And places even more importance of capturing user questions during the requirements gathering process. Their questions should be answered not through visuals but in writing so that the LLM can understand what it needs to pull and analyze before spitting out an answer.

Providing context means writing down all of the idiosyncrasies of an organizations. Acronyms, explanations to key words and key phrases used by various branches of the business, Excel files, relationships between different data points (think about how much is inferred through our conversations but not explicit when viewing the data), and questions that need to be answered. In a way, it's a giant Q&A document stored in the backend. And so the BI engineer spends a substantial portion of their time developing the context and validating that their words are translating to accurate answers provided by the LLM.

This shift also opens up a new market and the growing importance of tools that seek to create a context layer. Because where should this context live? In text files stored on Sharepoint the LLM is pointed to? exclusively in Power BI semantic models? There will be a need to store the context and make it accessible. One could develop that using the LLM itself. An option that I've been exploring is Microsoft's Ontology feature which I believe is their first foray into enabling users to create a context layer and makes use realize that data leaders seeking to stay on top of AI development need to start thinking about the context of their data and how to connect it all together.

Another component to the mapping of data and LLMs are the multiple LLMs existing within an organization. We all know there's not just one but many that are being used by all different individuals. Whether those are embedded within tools found in common engineering stacks or standalone models where all users export their thinking or ask their questions, there is a need for governance (One LLM To Rule Them All - future post 😉 ). We've lived through the fragmented siloed data landscape where the preeminent problem was "Hey your data is siloed across various databases. Let's bring it all together in a data warehouse/lakehouse." Next comes the fragmented siloed LLM landscape where the problem will be "Hey your agents don't talk to each other. Let's bring them all together in a new AI unification layer." Data leaders need to think through how to stop the spread of agents and model use within their organizations.

At this point, I should most likely shift my terminology to "agents" and more specifically "data agents" whose task will be to retrieve data for users or create reporting for them. I will reserve a deeper dive into how these agents can be best served across the data stack for a future post. For now, the focus is on the shift that is coming to data teams and Business Intelligence teams.

Data Engineers will continue to focus on ingesting data into centralized repositories. BI Engineers will continue to model the data. And there will be a focus on writing the context around all of the data that is accrued across the enterprise. The rise of Context Engineers?

Additionally data leaders and data teams will shift their long-term focus (we should be doing this now to be fair as we are a couple of years in) on managing the various LLMs and agents living within the enterprise ecosystem. Frameworks need to be established that help users use them and prevent privacy and security issues. With the framework, the layer that ensures agents and LLMs talk to each other, keeping themselves informed of each other's activities, potentially checking themselves on accuracy and anything else that might need to be shared to represent a shared context across the enterprise. We do not want to be in a situation where LLMs provide different answers yet we are undoubtedly heading in that direction.

There's a big shift coming and I'll seek to guide us through it over my next posts - stay tuned!

When we talk about analytics at an organization, two operating models are usually at the forefront of preferences or strategic options. A centralized analytics model and a self-service analytics model. The nature of the approach is pretty self-explanatory in their naming convention. Centralized analytics is defined by having 1 team or groups of teams all reporting to a centralized function while dedicated to servicing the business. Self-service or decentralized analytics is where each business team runs their own analytics teams dedicated to supporting that business department exclusively.

Before we dive further, let's take a step back and understand that this operating model is tied to or limited by the business unit that it supports. Going beyond the different department or business teams of that business unit, it's important to understand that an analytics operating model can be replicated across different business units but its implementation and subsequent operation should not span different business units. For example, a large multinational organization operates various business units from selling cars, houses, and consumer goods. Under each of those business units, an analytics operating model should be established and supported by appropriate technological and people resources.

Each of these business units should the have tools, individuals, and technologies to fully support analytics for that business unit. One single analytics stack in support of multiple business units would quickly cripple the analytics function as those teams would either be too small to support demand and deliver insights on time. Or they would be too big and be weighed down by the sheer bureaucracy naturally adopted from multiple large teams.

An analytics stack I define as the tools, teams, and technologies that support analytics functions and are composed of a data platform (data warehouse), a team of data/analytics engineers, BI engineers and a BI platform, and data analysts.

In a centralized function, the entire analytics stack sits under one individual - usually a Vice-President of Data & Analytics. This individual works with business teams to understand their needs and works to deliver data products or data solutions that provide said business teams with the data or reporting they need. As an example under this model, a request will be made to the front-end data team (BI engineers or data analysts), the data and analytics engineers will develop a new table and data model, and the BI engineers and data analysts will further refine the model while working with business users to ensure their needs and requirements are met. Granted, this is a gross oversimplification but you understand the point. All of the above happens "in-house". The business's only ability is to use the data or reporting solution that will be provided to them. Even for ad-hoc data requests, those go to the analysts to write a query and provide the data (usually in excel).

Under the centralized model, the business will face a bottleneck in the access of data and information. Both will be slow as requests are prioritized and acted upon by the corresponding data team. Not necessarily a bad thing if you ask me, especially if the business is too reactionary versus proactive. In the centralized model, the data teams focus on building the right thing for the right insights. The right solution is always more effective than a dozen misguided attempts at the solution that ultimately get tossed away. Faced with finite resources, the business is forced to make decisions on what it truly values and needs to grow.

On the other hand, in a decentralized function, the flow of information is correlated to the amount of analysts on the business teams. In the decentralized or democratic model, the top (or front) end of the analytics stack sits with the business and reports directly to the business. Analysts are empowered to act as data experts, BI engineers, report developers, and anything in between that involves the retrieval and use of data. Business analysts work with their heads of business to identify data and reporting needs building data models, and reports that fit the requirements. In this scenario, speed-to-value is prioritized as requests can be turned around quickly as there is no reliance on an engineering team to build something for them. And if there is a need for more data, what's wrong with sticking a few excel files into a BI tool and maintaining that on a daily and weekly basis?

The decentralized or democratic analytics operating model gets data into the hands of the business faster but at the cost of maintaining manually-operated reports and data solutions. Business teams are not set up to maintain technical engineering solutions. Lack of knowledge into best practices and standards leads to things that "work" in the sense that the output is trusted but don't work as soon as the individual who's built it is gone. Business created and owned solutions last as long as the person who put it together. There are countless examples where the business maintains something but has no idea what it actually is and therefore can't make changes to it.

I'm quite obviously showing my bias at this point in time. But I would make the argument that is it worth discussing the benefits of an operating model that only has... one benefit? Organizations that establish a decentralized function in their analytics operating model will get fast results that fizzle quickly. One way to address this is to establish a hybrid model. Decentralized centralized analytics.

Self Service = Initial Burst of Value

Centralized = Value over Time

Decentralized centralized analytics has the greatest chance of finding the balance to deliver value over time at fastest speed. In a decentralized centralized analytics operating model, analytics functions still sit under a central authority but front-end members of the data team (BI engineers and/or data analysts) sit with the business and work exclusively with that business team or function. This is arguably more commonly known as the hub-and-spoke approach or the business partners approach where members of the analytics function serve as direct partners in support of business needs.

Under this operating model, the business knows the individual they need to work with for reporting and data needs. Sitting within the team, the data analyst or business engineer becomes knowledgable in data that business team generates (and it's preferences for reporting). Being part of the centralized analytics function, the data analyst or business engineer is empowered to build data and reporting solutions that are efficient, scalable, and follow standards and best practices. The centralized function teaches them those standards and best practices as they report to their analytics leaders and that knowledge is leveraged in their output for the business team they are attached to. This way, the business receives a fast response to its reporting and data needs while governance and scalability are addressed.

In the prior paragraph, I say "sitting with the team" which I mean quite literally. In the decentralized centralized analytics operating model, the BI engineers and data analysts desk's should be located with the business team in the office - in fact, there should be nothing that should prevent them from not being considered a member of the team. They should act as a team. Talk to each other in person to solve data and reporting issues. Understand what each is working on. The only thing that would identify the engineers and analysts as being different are their weekly meetings with their managers and governance-focused ceremonies (this is important and a future blog post!).

Let's quickly summarize. Self-service/decentralized/democratized analytics operating models prioritize speed-to-value which fizzles over time. Centralized analytics operating models prioritize value which takes a while to actualize. Decentralized centralized analytics aims to prioritize both through analytics partners assigned to specific business functions.

AI has its place in the world of BI. I believe there is value in embedding AI capabilities into your BI stack - whether it's an aide to engineers as they develop or an agent that the business can interact with. That is the spectrum as I see it today - the generation of code based on solutions led by engineers and engineering teams and the front-end agents or data agents that serve as the go-between the business and its data. I'm sure a good portion of you reading this have used a few of the models for generating code across various languages. Who can deny that boost in productivity and creativeness (if you ask for creative solutions). That's a gain that, to me, has already been realized across organizations leveraging AI whether they know it or not. As such, it's vital that AI use is understood and spoken about. Building said understanding allows the building of appropriate AI governance which organizations will need to drive innovation and value from AI.

When it comes to the front-end "AI agents", I have been pleasantly surprised as what they can do. In testing, we created a table and PowerBI semantic model that contains a massive amount of KPIs and metrics. Releasing an agent on top of that model yielded poor results until a few hundred lines explaining the relationship of the data, anticipating user questions, and translating regular written English to column names provided enough guidance for the agent to start answering questions with increased accuracy. To the point where we were impressed with how well it was responding.

Armed with that wealth of information, the agent was not only able to answer questions that a user might usually look at a report or dashboard for, it was also able to generate appropriate graphs and visuals that further helped the user understand the data. And this point right here is where I believe AI has its place in Business Intelligence. Where we get to "GenBI".

I imagine a future maturity state of Business Intelligence across an organization that does not involve the use of common BI tools such as Power BI or Looker. In this maturity state, the top level of the stack is simply an agent embedded within the warehouse or lakehouse, provided with guidance, and business users interact solely with the agent in the answering of questions, the generation of reports and dashboards. BI teams are no longer developing reports and dashboards for the user, they are building data models and injecting the model with relational data context in order to give the users the ability to generate their answers and visuals for storytelling. This is the foundational shift in Business Intelligence via AI. I no longer need to grow a BI team to model data and build reports for users. This, as we know, is time-intensive approach. Queue all of the stories we've heard about gathering requirements from users and doing UAT... With a focus on AI, I can have my team ensure the agent has the appropriate guardrails and then letting users run with it. Speed to delivery can be quasi-eliminated as no one needs to wait for the BI team to build a report. Any information can be available as soon as it has been modeled and the agent been given the context.

Now I realize I might be getting ahead of myself. No matter how excited I am about piloting an agent-first BI stack - there's plenty of things to watch out for as we move towards GenBI. Governance will be key (again). Organizations will need to spend time logging and documenting their data and the relationship of data across the whole enterprise. Users will need to be trained in interacting with agents. AI teams will most likely need to be stood up in support of managing various agents. I could go on and on but I'm not writing about AI Governance. Not yet anyways. However AI Governance will supersede the need for BI Governance as the former takes over the latter in its importance of giving organizations the insights they need into their operations.

Even if we are at the beginning stages of AI in BI, there already is plenty of value to extract from what's been made available. Whether that's through something like Microsoft Fabric data agents or a fully open-sourced solution, the landscape is changing and BI leaders must be ready to incorporate AI into their stack.

Business Intelligence comes in many shapes and sizes. What might work for one organization might not work for another. Maturity differs and implies a different approach that suits the organization's needs. The solution is not always to purchase a BI tool like Tableau. Sometimes all that's needed is a bit of governance. Other times it's a vision and strategy to that vision. Maybe there's not even a need to purchase a BI tool (the first step might be to mature analytics - we'll get to that). The possibilities are endless.

Whatever the path chosen, the content is deep. There are layers and layers to uncover and think through. It might seem like a simple decision but if not thought out carefully, you'll end up at the bottom of Kilimanjaro instead of at the bottom of the rolling hills of Windows XP (you know which background I'm talking about).

My goal is to get you to understand the puzzle that we're looking at so you focus on putting the right pieces together instead of questioning if there even is a puzzle.

We always hear that BI is about turning data into actionable insights - and that's true but what does that mean? To me, it's about giving the business the ability to look itself into the mirror and know why it's making the decision it makes. Let's look at Business Intelligence through the mirror and figure out what to do.

Pierre