AutoML - is it useful?
I’m very excited about writing this blog post. AutoML has been on my mind for a long time. I wonder if it’s the future of AI or just a sidestory when we look back in 10 years? I think I settled on an opinion but before I reveal that, I’d like to discuss AutoML a bit more in depth.
I would like to answer a few questions in this post. Remember - My posts are always meant to be seen from a product manager, commercial or at least not a data scientist or engineering perspective. So I won’t go into very technical details. The important objective for me is that non-technical people can get a better understanding of applied AI.
Now for the questions:
What is AutoML really and how does it work?
What are the pros and cons?
When is AutoML appropriate to use and when is it not?
Excited? Let’s get at it.
What is autoML?
Before defining AutoML I want to narrow down what I mean by AutoML. When talking about AutoML here I’ll mainly be referring to hosted solutions or at least solutions trained in a hosted environment by an AutoML vendor. I won’t be going into open source solutions or take their possibilities or workings into account. I might be wrong but I’m afraid it would be too broad a discussion with too many if’s to few useful conclusions. Feel free to contact me if you feel like I should add this.
The definition of AutoML
AutoML is the automated process of selecting architecture, training and in most cases deploying machine learning models. The abstraction is at a level so that the AutoML can be given labeled data, find the right architecture, train and tune the model with intervention. When the model is trained it’s ready to classify when queried with new data.
So what does this mean? It means that you don’t have to do any machine learning or data science work to get an actual AI model up and running. Depending on the vendor you use and the solution you might have to do some code to interface with the model and the training environment. So no-code AI. Does it sound too good to be true? It might be. I’ll go into the pros and cons in a minute, but first I’ll go a little deeper into how it works.
How AutoML works
So no AutoML vendor will of course lay out exactly how their AutoML works but they do give some very good pointers.
First of all it seems like a lot of the AutoML out there is based on Transfer Learning. Transfer learning in nutshell is when you pre train a neural network with a lot of data to get a good stable model. You then take the last few layers off the neural network and retrain new layers with data specific to your new problem. In that way you can utilize the existing neural networks general knowledge to your new problem. That ensures higher quality with less data on your new problem.
This is the reason AutoML solutions can take a small amount of data and still get good results. Sometimes even less than a 100 records in a dataset. The pre trained models already had a lot of background knowledge that your model gets put on top of.
The reason this is interesting is that it gives you an idea of when AutoML is useful. It’s useful when your problem is a bit too specific to be in a general knowledge space but looks a lot like a general problem. This could be categorization of chairs. General machine learning models like the Google Vision API can easily recognize chairs from tables and label that for you and it can even tell a chair from an armchair. But what if you are a furniture designer that wants an AI-model to tell apart your different chair models? Then you can train the AutoML to do exactly that.
A last point about how AutoML works is that it looks to me like at least Google uses reinforced learning to figure out the optimal architecture. That’s pretty interesting. Using reinforcement learning to find the right architecture and tuning sounds like a computation heavy solution but cool nevertheless. There’s a paper here if you are not afraid of the techie explanation.
When should I use AutoML?
So now for the main course of this blog post. When is AutoML the better option? When writing this I’m comparing AutoML to do-it-yourself machine learning. Or the conventional solution as of now.
Just for a minute let’s turn the question around - Why would you ever make your own machine learning models when you can get training, tuning and deployment out the box? Besides having fun with machine learning of course.
Right now it seems that doing machine learning by hand is default and I believe a bit too many problems are solved from scratch not utilizing the already made work.
But to answer the question - When to use AutoML, I think there’s currently an obvious sweet spot. When the problem you are trying to solve is kind of general but too specific to be solved by the standard solutions. The problem has to also be somewhat simple still. If it’s a part of a bigger architecture it might be too inflexible. I’ll go into pros and cons in a minute.
First I’ll mention that AutoML can be used for very different kinds of problems. Not just image recognition problems. It can also be used in:
Language processing (such as sentiment analysis)
Translation
Video object detection
Document classification
Churn analysis
And a lot more problems.
There’s also generally two different ways to deploy the models once they are trained. You can either have them hosted with the vendor or you can download them ready to be deployed on edge products.
If you leave them hosted you can access them by querying through an API. In that way you can build the model easily into other parts of your product.
The other option, installing on edge products is meant for IoT solutions. If you are making IoT solutions it might not be optimal to send images of data to an API. It could be inconvenient due to the latency when connecting to the API, it could be impossible to have an internet connection all the time or it could be due to GDPR. Sometimes GDPR is easier to comply with if data is processed locally and not sent to a third party provider.
The pros and cons
Now for the pros and cons.
Con: You get less insights on your data. One of the big advantages in doing machine learning by hand is that you get a lot of learnings into why your models do not work as expected. This gives you valuable insight in both the existing business domain but also a good idea of what data you might need to go out and collect in order to get the necessary results.
Pro: You get quicker results. With AutoML you can skip a lot of the machine learning and data science work and as a result save some time. If you are building a prototype to test a product market fit or get early experience on how users will react and use the AI when applied.
Con: AutoML is inflexible. You might get results quickly but as most IT goes it won’t be long before the requirements are changing and with a AutoML solution you are running the risk of these requirements suddenly being out of scope of what AutoML can handle. If that happens you have to start all over with a custom model and that can be a cold start.
Pro: You’re less likely to be outdated. A very common problem in AI development is that your models are getting old almost as you’re making them. The AI technology is moving so fast that what today might be state of art and requires a lot of hard and intelligent efforts to achieve is tomorrow performing worse than out of the box solutions. With AutoML that pain is placed with the big tech AutoML vendors that have the economics of scale so they can invest in staying ahead.
Con: Running cost might be too high when you scale. A pay-per-use model is great for small to medium scale solutions. But if you’re expecting very high volumes in use then you might end up spending too much money with your vendor. In this case it might be a better solution to make and host the models yourself.
Pro: With a hosted AutoML solution you will save a lot of time on not having to build the surrounding infrastructure. In this paper by Google the conclusion is that on average 95% of code in machine learning solutions is “glue code” that builds the infrastructure around the models. The actual machine learning code is just 5%.
Con: Quality probably won’t be the highest. If the most important feature in your solution is the highest quality compared to competitors AutoML probably will fall short. AutoML is generalized models and can in most cases not compete with truly specialized models that have been carefully built and tuned to the specific problem from the ground up.
Pro: You will need less experts. A common problem in AI is that it requires a lot of expertise and usually involves several different kinds of experts. That’s extremely expensive and experts are hard to find. You can skip a lot of that with AI.
Google AutoML
I also wanted to go through a specific vendor to make it even more hands on. I choose Google simply because I know it the best. I’ll list some alternatives a little further down.
Google has spent a lot of effort in AutoML. If it’s reasoned by the expectation that AutoML is the next big thing or if it’s a convenient way to make people upload tons of very specific labeled data that would be too expensive for them to do I can’t tell. It could also just be a way for them to keep advancing their machine learning knowledge.
You can find Google AutoML here: https://cloud.google.com/automl. With Google their AutoML is a part of their cloud solution. This is convenient since this is probably also here you’ll store a lot of the training data and maybe even deploy the rest of your code.
A really cool feature with Google AutoML is that you can set a threshold for recall and precision. If you’re not sure what that is you can read about it in my other post here. It’s really useful for making the models something you can Apply to real world scenarios.
Google AutoML Vision
With Vision you can do a couple of different things. The primary solutions are following:
Image labeling: The AutoML can categorize images from the training you give it. If multiple answers are correct it will suggest what it thinks is the most dominant category. This is really easy to train since you just have to divide images into folders.
Object detection: This is a bit more complicated. With object detection you can detect multiple objects in the same image. Really useful but it also requires you to label that data and that makes it a bit more difficult.
Video intelligence: You can both classify objects and categorize video segments.
Language and translation
Entity extraction: With entity extraction you can train the engine to spot your domain specific entities in texts. As an example you can teach the engine to find specific data in your customer cases that are unique to your business or domain.
Sentiment Analysis: There’s a lot of good sentiment analysis out of the box, but sometimes the clues are very domain specific. As an example, if a customer states the time from ordering to receiving the product it’s very domain specific. In a restaurant it’s a disaster but in a translation service it’s amazing.
Translation: Translation can also require some specific training. If I translate my blog to danish I don’t want “Google cloud” to become “Google sky” since it’s a name. Cases like that are usually difficult for general translation.
Tables
I think Tables was actually the first AutoML solution Google announced and it’s also the simplest one from a user perspective. This is simply taking tables of data, uploading it and making predictions on it. I’m not even sure that there’s any Transfer Learning or other pre trained solution in the background here. It’s probably just a question of Google finding the best architecture for the problem.
Other AutoML providers
I just wanted to mention there are other providers as well. I haven’t had time to look into them but they do look interesting.
Update: I made a list of AutoML solutions now with comparison of 20 different solutions.
H2O.ai
This one looks a bit more developer friendly than Google but also like it has a lot more features.
Datarobot
This looks like it’s very enterprise focused with a bit more AI as a Service model.
So is AutoML the future?
I actually think it will be a very dominant paradigm in the future. There will always be plenty to do on all custom made AI solutions but for most AI projects I think AutoML is sufficient. Most problems are very closely related after all. Most people think that their problems or their businesses are unique but really it’s a matter of nuances and nuances to general problems are what AutoML is good at.
For some it will be a prototyping tool and for others a production tool. But as I see it the advances in the technology is quickly making this tool more and more production friendly.
This kind of AI solution also lowers the entry barrier to work with AI. In other words it is not necessarily a question of AutoML “stealing” the AI market. It could also just make the market bigger and grab the newly opened land here.
Conclusion
To conclude I think AutoML offers an interesting approach to AI and will lower the AI entry barrier in many cases. As a result both prototyping and small scale AI solutions can get up and running with virtually no cost. There’s of course down sides. Especially the inflexibility and the knowledge gap you get by not doing the machine learning in-house should be considered carefully.
I do believe AutoML will have a more dominant position in the AI world in the years to come.
