How do you define good AI products?
Product focus first gives you an edge
AI Product Thinking is a series covering insights, patterns, and trends within AI product. Check it out here!
Recently, I've had more discussions with data teams and PMs about AI products. We've there’s been increasing discussion around what good AI products look like and what not so good ones look like. Even how do we define an AI product.
It’s been an interesting process chatting with a lot of people. And wow, have I gotten so much valuable feedback to think about.
For some of us, the AI product revolves around the model. And for others of us, it revolves around the data. Each of these are correct. And we found out that use cases can really affect how you use AI to enhance a current product.
By now, some data teams have started figured out how not to fall into the build trap. In other words, building models that don't enable the product or enhance it. It prevents us from using AI to build product features that nobody wants.
In all of these conversations, I’ve started to find a common thread:
We need to define an AI product.
We need to define AI product quality
We need to know how to iterate a product
I’m going to break these down a bit.
What is an AI Product?
We may think of the AI product as the model. It’s pretty low hanging fruit, since that’s what we often work with. But products are a bit different.
An AI product is a tool or system enhanced by AI or ML models. At its core, the output of an AI model forms the foundation.
Data is the foundation of an AI product. It defines the model and the outputs. Data that goes into the model and comes out defines the product - the output and the input.
Businesses use these outputs to improve existing products or to elevate customer experiences. Integrating outputs from ML and AI models can significantly enhances value, utility, and efficiency.
The outputs augment and enhance user experience and product features. They add features or improve existing ones. And they must have a market and meet an end user’s needs. This is the first shift data science teams must take if we want to take a product focus. It's not enough to build prototypes.
Data teams find this tricky because we're used to focusing on the model. We tend to focus on the features, monitoring, and end to end process. We don't pay as much attention to how we use the outputs.
You can tell the potential of an AI product by looking at how the team approaches the data.
What are the levels of AI product?
Teams often approach a model two ways: a model centric or data centric approach. The model centric prioritizes code over data. The data centric puts data first. This has massive effects on AI products.
Below Average AI product. These are model and tech focused. AI integrated without determining if it's even needed. Existing (any) data is used to justify the model. Data isn’t explored, understood, and or curated. AI products built with model focus first rarely find good market fit.
Average AI Product. These are data focused. The data sources are available. Models used in them have proven their ROI. But the data is low quality or inconsistent. The outputs vary, which result in very inconsistent experiences for users. Data is good enough, so AI products here may succeed if the functionality is enough for the user.
Great AI Product. This is data and functionality focused. Data is available, explored, and the context is understood. Its curated and aligns well with the business model. AI models augment and enhance existing product need. AI products in this category provide consistent, reliable, and highly valuable outputs to the users. They increase monetization of products and improve product market fit.
The level of AI product is dependent on the data. Great AI products are dependent on effectively manage and organized data assets.
Data holds significant intrinsic value for teams and business can be a driver of growth and innovation. It provides a competitive advantage in AI products.
But that requires good processes. I believe we also need good iteration.
Product Iteration Raises Quality Level
Product focused, not protype iteration needs to be at the core. We need to shift towards product functionality.
Technical iteration is great for building technical solutions or supporting AI projects - they’re important. But they’re less useful when our goal is to use AI to augment or enhance a product.
From my conversations with data science teams, it's clear. We often start with the model first. Then, we build for product functionality. It becomes model focused over outcome focused. We end up optimizing for models first, before optimizing the model and curating data for product functionality.
For product managers, this can be frustrating. Teams can fail to meet delivery times or product feature releases because they’re still building. Common feedback I’ve heard from product managers is we spend a lot of time building models, but not enough on functionality. We can end up prioritizing what we are building without understanding why.
But is there a better way?
AI product iteration is about user need and functionality. AI model iteration is a matter of data and tech. When we focus too much on the tech for products? We end up flat on our face, with deadlines inching ever closer.
If we're trying to help users get from point A to point B. we need to start with a basic functionality. Then, move to an advanced one. This doesn't mean that you have to start off even with an AI.
You need to start off with the product or products that already exist. Those needs are already defined, customers are (hopefully) understood, and product market fit exists.
Starting off with AI first is like trying to build a car but starting with the wheels. While tires are important, they don't define the car's purpose or functionality. Your first step should be to define what the car is supposed to do, who will use it, and in what context. Only then do you decide on the tires that best fit this purpose.
Creating AI models or LLM models is very different from making an AI product. The key here to understand is it's not linear. The goal should be to allow AI to target a particular use case and user. Each iteration in an AI product development is jump in enhancing functionality.
If AI models cannot address these? Then it's not necessary to use them. Businesses and teams that figure this out will save a lot on development and time. I’ve seen teams that put the product first over an AI first focus. I’ve also seen ones that put AI first over product. The difference is like night and day.
Our focus needs to be functionality - the end goal that the product is supposed to enable. Functionality is iterative. User experience is also iterative.
What do we need to do?
Building a great AI product is a lot of work. It means you have to go outside your comfort zone.
Data teams should start by asking 'why' they're using AI in their projects. Understanding this reason is important before beginning any work. After that, it's about working with product managers to make sure this idea turns into real features in the product.
Product managers should clearly communicate the product's vision and intent to data teams. This ensures that everyone understands the purpose behind the project before starting. Then, data scientists on those teams can work effectively with product managers to turn this vision into actual product features.
Bridging these two sides needs to be a priority. We need to set the relationship early between these two groups. PMs need to explain the final functionality and intent, while redirecting focus. Data teams need to ask why and build to the bare bones functionality first. Both need to really listen to the other.
This doesn’t mean we throw out all technical skill to make a product. We need to use technical skill we have to shape effective AI products. We need product vision to know where to mass our time and efforts. Technical limitations affect product quality and functionality. AI product vision and features affect a data team’s tactical focus.
As the current market goes competitive, the focus must shift. The days of large budgets and teams spinning out models without a larger product vision are over. Product needs to be top of mind if data teams building want to show value.
Data teams will need to take an AI product focus, as the field advances. The old world where we could build models without a product is no more. In a competitive environment we need to focus on product.
Thanks for reading! I write frequently about AI strategy, AI product, and data science.