AI and Decision Science - A forced marriage that is largely ignored

You might not be aware or do this unconsciously but, if you work with AI you also work in the decision science space. 

Imagine this: You have made an AI model that can take in support tickets and classify them into different subjects and sentiments. With that you can prioritize support tickets by how critical they are and have them directed to the appropriate support team. Sounds great right? But is it really that simple? No. With the AI model in place we are really only halfway to the finish line. If you decided to make an AI like the one I just described you must have had the goal of optimizing the support ticket workflow. Either for happier customers or to lower costs or maybe some other business objective. Either way, the way we choose to act on the data we get as a result of the AI is equally important to the actual AI, if not more. When we take a stand on how to act on the data we get we actually make a decision model. The science that goes into these models are not as simple as it might sound. Look at this example:

The support ticket AI suggests that with 60% likelihood a new ticket is about termination, 30% about a new feature and scores medium critical on the sentiment analysis. Now it doesn’t seem so easy anymore does it? How do we handle this information? Who should get this ticket? And isn’t a termination critical no matter the sentiment score?  

I’m in no way a decision scientist and cannot teach anyone much here. But what I can tell you for certain is that the decision models on top of AI are way too often left to be a secondary priority with no conscience or strategic approach. And even worse - The decision model is only discussed after we are finished with the AI models. I would argue that it is in the making of the decision model that we actually get to understand what data we really need, so making the AI first rarely makes sense since we don’t know what we actually need. 

There’s also a lot of traps to be aware of in decision making such as survival bias(Thinking you made the right decision because you got the right result) and many of us think we are better decision makers than we really are.

If you want to learn more about decision science my best advice is to follow the Chief Decision Scientist at Google Cassie Kozyrkov. She really succeeds at taking decision science to an understandable level. 

So to sum up. If we want to have better results with our AI solutions we should pay more attention to decision making and in many cases start with that before we go modelling.