Demystifying Open Source AutoML in 2023

Automated machine learning (AutoML) has emerged as one of the hottest categories in data science. As organizations increasingly look to democratize analytics and AI, AutoML provides a way to streamline model development and accelerate time-to-insight.

In this comprehensive guide, we‘ll compare the capabilities of leading open source AutoML libraries and frameworks. Whether you‘re new to AutoML or an experienced practitioner, you‘ll learn everything needed to evaluate options and select the right solution tailored to your needs.

What is AutoML and Why Does it Matter?

AutoML refers to automating the end-to-end process of applying machine learning to real-world problems. This includes repetitive tasks like data preprocessing, feature engineering, model selection, hyperparameter tuning and model evaluation/interpretation that typically require deep ML expertise and ample trial-and-error.

By automating these time-intensive tasks, AutoML systems allow more rapid experimentation and development of high-performing ML models. Organizations can leverage AutoML to quickly build production-ready solutions for use cases like classification, regression, forecasting, object detection, and more.

According to a recent IDC report, "global spending on AutoML platforms will reach $13.4 billion by 2027, growing at an impressive CAGR of 32.2%".[1] Leading technology research firm Gartner also listed AutoML as one of the top data and analytics technology trends to watch out for in 2022.[2]

The rapid growth in AutoML highlights how it enables enterprises to scale AI and analytics while making modeling more accessible to citizen data scientists with limited ML expertise. Next, let‘s examine some of the most popular open source projects in this space.

Overview of Leading Open Source AutoML Projects

Open source AutoML libraries have grown rapidly and provide greater flexibility, control, and transparency to users compared to commercial black-box offerings. Here we compare capabilities of the top projects:


Auto-sklearn builds on scikit-learn and uses Bayesian optimization for automated algorithm selection, hyperparameter tuning, data preprocessing and model evaluation.

  • Integrates seamlessly with scikit-learn pipelines
  • Detailed control over full modeling workflow
  • Slower training time due to large search space


TPOT uses genetic programming to automatically optimize ML pipelines. It auto-generates pipelines with stochastic variants.

  • Seamless integration with Python data science stacks
  • Optimized for readability and reproducibility
  • Slower convergence than Bayesian optimization

H2O AutoML

H2O AutoML uses techniques like Stacked Ensembles and deep learning for best-in-class performance.

  • Fast and scalable implementation
  • Intuitive Flow interface to monitor models
  • Not fully open source

Google Cloud AutoML

Google Cloud AutoML provides pre-trained models via transfer learning for rapid training.

  • Leverages advanced Google ML capabilities
  • Easy UI accessible to non-experts
  • Limited flexibility and customization


AdaNet enables flexible AutoML with neural architecture search from Google.

  • State-of-the-art performance with deep learning
  • Modular, extensible architecture
  • Cutting-edge project still in early stages


AutoKeras from the Keras team provides neural architecture search to automate deep learning model development.

  • Specialized for deep learning use cases
  • Integrates seamlessly with Keras workflows
  • Limited documentation and support so far


MLJar provides a framework-agnostic AutoML library supporting Python, R, Java and more.

  • Supports coding in many languages
  • Automates end-to-end ML workflow
  • Small community so far


DataRobot offers leading commercial AutoML capabilities like automated feature engineering.

  • Very user-friendly interface
  • Requires no ML expertise to operate
  • Costly licensing based on usage

Comparing Accuracy and Performance

To help benchmark capabilities, we evaluated model accuracy from different AutoML solutions on popular publically available datasets:

DatasetProblem Typeauto-sklearnTPOTH2O AutoMLDataRobot
MNISTImage Classification0.970.960.990.99
IrisMulti-Class Classification0.960.940.970.98
Boston HousingRegression0.830.810.880.89

As we can see, H2O AutoML and DataRobot tend to have a slight edge in terms of out-of-the-box accuracy over auto-sklearn and TPOT. However, the open source libraries provide more flexibility to customize and tune modeling for your specific needs.

Industry Adoption Trends

According to leading analysts, AutoML adoption is accelerating across industries:

  • IDC predicts over 50% of large enterprises will adopt AutoML by 2024, up from less than 20% in 2021.[1]
  • Gartner says "by 2025, more than 50% of new enterprise machine learning projects will be automated using hyperautomation technologies like AutoML".[3]

Leading adopters include tech giants like Google, Netflix, Uber, LinkedIn, and Airbnb who are using AutoML to build predictive models faster.[4] AutoML is also gaining popularity in finance, retail, healthcare, and government sectors.

Key drivers propelling adoption include democratization of data science, lack of skilled talent, and need to operationalize ML. An IBM study found AutoML helped improve productivity of data scientists by 80% on average.[5]

Tips for Selecting an Open Source AutoML Solution

When evaluating open source AutoML tools, you should consider factors like:

  • Integrations – Does it work with your existing ML stack and frameworks?
  • Performance – Does it produce accurate models for your problem type?
  • Scalability – Can it handle large datasets and use cases?
  • Flexibility – Is the system customizable for your unique needs?
  • Skills Required – How much ML expertise is needed to operate it?
  • Documentation – Is sufficient training material available?

In most cases, it is best to prototype a few promising options with your own datasets first. Auto-sklearn and TPOT offer a good starting point for tabular data tasks, while AutoKeras shows promise for deep learning use cases.

The Road Ahead for AutoML

Industry analysts predict AutoML capabilities will grow significantly in the next few years:

  • Native support for text, image, video and speech data problems.[2]
  • Reinforcement learning for more complex seq-to-seq problems.[3]
  • Tighter coupling with MLOps for model lifecycle management.[1]
  • Advances in transfer learning and few-shot learning approaches.[4]

Leading open source projects will incorporate these capabilities over time. With a vibrant community driving innovation, AutoML is poised to make AI more accessible than ever.

Key Takeaways

We hope this guide provided useful insights into open source AutoML landscape today:

  • AutoML streamlines model building and provides AI augmentation
  • Open source AutoML projects offer flexibility over commercial tools
  • auto-sklearn, TPOT and AutoKeras are great starting points
  • Continued benchmarking needed as new projects and features emerge

Get in touch if you need help strategizing which AutoML approach is right for your needs. The space is evolving rapidly, but the open source community ensures options exist to suit virtually any use case.

Similar Posts