How to build an AI business case
I recently surveyed danish CIO’s(Chief information officers) about their relationship with AI and I had some interesting results. One of the results was that one of the biggest barriers to get started on AI projects is that building the business case is difficult. I completely understand the issue and I agree with the CIO’s. Building an AI business case is difficult and if you try to build it as a traditionnel IT business case it’s down right impossible.
Building a business case is all about understanding the cost and revenue drivers well enough to work them into a model that yields a profit with high certainty within an agreed timeline. When building AI solutions or even buying them off-the-shelf that whole process turns out to be way more challenging than what you will experience with traditional IT-projects. In my experience this is for many a lesson hard-learned by many in the IT business that naturally grabs their well-known tools and methods but quickly fails. This often results in AI being disregarded as being a too immature technology. With the right approach, that I’m going to show you here, you can actually build a business case that makes sense. The technology is ready and at a stage where most businesses can successfully utilize it. New technology just requires new approaches.
Before moving on to how you build an AI business case, let’s understand why this is such a difficult task. The reason is simply, that everything in AI is experimental in its natural form and as a result nothing is predictable. How much data you need, what algorithmic approach will work and how good the result will be is very difficult to know beforehand. You can look at a similar project but small differences in the problem, the data or the environment will often to much surprise make a big difference. So knowing the exact costs, results and the road there is just not possible.
The cost side
What will it cost to an AI? You just can’t know. In traditional IT we try to break down the project into smaller and smaller pieces until each piece is such a size that we can easily estimate the time and costs that go into it. In AI the process is experimental and we can’t even know the pieces in advance.
To combat this problem there is a set of strategies that will make it a lot easier to control the cost side. On purpose I’m writing, controlling and not predicting the costs. In the AI paradigm, predicting costs should not be the goal. The goal should instead be to control it. I’ll get back to why that makes sense a little later.
The cost control strategies are the following:
Iterations
For years now we have been talking about the agile approaches in IT. Some have used it with success others have steered clear and stayed with traditional methods and some unfortunately have used it as an excuse not to have a plan at all. The agile approach suggests iterating through projects several times to account for new learnings during the project and changes in demand. Similarly AI projects should use an iterative approach to get a set of important learnings. Just by doing one very quick simply iteration you should get these learnings:
You understand the data better. You understand how much effort it requires to attain, how to attain it and get a sense of how much you will need.
You can get a sense of how the users react to a certain quality and how difficult it will be to deploy
You get a good idea of potentially attainably quality.
Last point here should be seen as a stop-test. If you don’t see a close to acceptable quality in the very first iteration it’s very unlikely that you will see much better results in the near future with either attaining much much more data or putting a significant larger investment into the algorithm work. So many AI projects should be abandoned if the first iteration is not close to a useful solution. In some cases though this can be just a wrong algorithmic approach. This is where you have to rely on the tech people's judgement.
For the first iteration you can in my opinion start very small by using AutoML solutions. AutoML is AI without coding that can be trained and deployed within hours just needing only data. There are pros and cons here to be aware of. I wrote another blogpost about this here.
Milestone funding
I preach a lot about milestone funding in AI. This is a very effective strategy to control costs. In AI the milestones are natural and project funding should only be released for each milestone once a set of agreed criteria is successfully met. The milestone naturally would look like this:
Collect data
First step is to collect a certain amount of data at a certain quality at a certain cost. Collecting, cleaning and preparing data is almost always the most underestimated cost in relation to AI projects so making these first steps success criteria very specific is not a bad idea at all.
An important aspect of data collection is the frequency you need in updating the data. Some projects require only initial or rare data collection and for others it’s required to build an entire data operation that in itself should be a good business case. Many projects die from costly data operations so take it into account early. The trick here is to measure a lot in the process.
Building models
Next step is building models. This is in the business case not that complicated. This is where tech people have to estimate but they will do so with great uncertainty and that is just how it is. As mentioned before the first iteration should be as quick as possible and if you don’t see potential for good results after this, you should be willing to stop the project or change technical strategies.
Deploying
You should also try to deploy the AI models into a test or staging environment already in the early iterations. It might seem like overcomplicating the problem, but AI models are just more clumsy to work with in my experience than other code bases. The amounts of data also makes the dev ops challenge a bit more interesting.
Studies have also shown that up to 99% of the code in AI projects are all the “glue code” around the actual AI that makes it work in the given environment. So getting a sense of this early is also a good idea.
For deploying the model there is also very often a human aspect that should be a part of the criteria for success. People respond differently to AI solutions than other solutions since AI is harder to understand as a layman.
Bundle your AI projects
My last advice for controlling costs in AI projects is to bundle more projects in one business case. As you can see, the risk of an AI project early on turns out to be too costly or not good enough, is there. There’s a tendency in IT to keep working on projects that have already shown signs of being a failure since we as people overestimate our abilities to improve the situation and we just want to deliver something. If we deliver nothing we feel as complete failures.
To avoid this, put more projects in one business case so you can let the bad one die and the good once flourish. You might argue that people should just be better at calling it quit when failure is inevitable but to me changing the structures and letting people be people is a much superior strategy.
AI business culture
Before moving on to the revenue side I wanted to add some notes on AI and company culture. As I mentioned the possible is that some AI projects should be shut down early. It can look like a failure but with the right culutre this can be seen as a successful null-result. Collecting a ceratin amount of null-results is very valuable to a business, especially if done at a low cost. By knowing for sure what does not work a business can much easier navigate and plan ahead. The only problem is that null-results are not always cultural acceptable. Untill it is a lot of the AI business case strategies to control cost won’t be very easy to implement. So management has a very important responsibility to make sure that the company culture supports these approaches.
The same goes for the cost control. If there’s not a culture for controlling cost instead of predicting AI will hardly be good experience. AI ironacly doesn’t offer predicability. So a culture that instead supports budget or time boxing is much more effecient for AI.
The revenue side
When someone asks me if a certain problem can be solved with AI I answer “probably yes” since people are usually on the right track. The natural followup question it “how good will the AI be then?”. The right answer here is “I don’t know”. This is to many a hard to shallow pill. The people that demand answer here will rarely succeed with AI. Those that can work around that lack of information will be much more likely to successeed so naturally that should also go for the business case.
If the revenue, value or profit is based on the quality then you can’t calculate the expected profit since you can’t know the results. Even if the AI is sold at a predetermined prices it’s hard to predict since adoption among users is often based on quality.
Who are you trying to beat?
Besides utilizing the very quick interations to get an idea about the quality to expect you should also be clear on what to expect from your AI. Don’t try to make a business case for a perfect AI. Make one for a good enough AI that solves the problem at hand. Very often new technology is held to golden standards and the expected results will be out of this world. Be very specific here in your communication to avoid this. I also wrote about it here.
Presenting your business case
Now that you know that you can’t build a business case on AI projects as you would classical IT projects you all set right? Not quite. The last challenge is when you have to present the business case. Your peers that might need to review or accept the business case usually expect classical IT paradigm business cases. So my last piece of advice here is simple - Start by presenting the core principles of AI and how that makes the business case different. If you get a buying on your new approach everything will be much more smooth.
