Automated Machine Learning: State-of-The-Art and Open Challenges

Nowadays, machine learning techniques and algorithms are employed in almost every application domain (e.g., financial applications, advertising, recommendation systems, user behavior analytics). In practice, they are playing a crucial role in harnessing the power of massive amounts of data which we are currently producing every day in our digital world. In general, the process of building a high-quality machine learning model is an iterative, complex and time-consuming process that involves trying different algorithms and techniques in addition to having a good experience with effectively tuning their hyper-parameters. In particular, conducting this process efficiently requires solid knowledge and experience with the various techniques that can be employed. With the continuous and vast increase of the amount of data in our digital world, it has been acknowledged that the number of knowledgeable data scientists can not scale to address these challenges. Thus, there was a crucial need for automating the process of building good machine learning models. In the last few years, several techniques and frameworks have been introduced to tackle the challenge of automating the process of Combined Algorithm Selection and Hyper-parameter tuning (CASH) in the machine learning domain [1]. The main aim of these techniques is to reduce the role of human in the loop and fill the gap for non-expert machine learning users by playing the role of the domain expert. In this tutorial, we aim to present a comprehensive survey for the state-of-the-art efforts in tackling the CASH problem. In addition, we highlight the research work of automating the other steps of the full complex machine learning pipeline (AutoML) from data understanding till model deployment. Furthermore, we provide a comprehensive coverage for the various tools and frameworks that have been introduced in this domain. Finally, we discuss some of the research directions and open challenges that need to be addressed in order to achieve the vision and goals of the AutoML process. This tutorial is intended to benefit researchers and system designers in the broad area of machine learning. The tutorial would benefit both designers as well as users of automated and interactive machine learning systems since a survey of the current systems and an in-depth understanding will be essential for choosing the appropriate system as well as designing an effective system. This tutorial does not require any knowledge on automated machine learning techniques but basic understanding of machine learning pipeline is required. After attending this tutorial, the audience will have: - An overview of the Machine learning pipeline (10 min.). - A good understanding of the challenges of implementing efficient and high quality machine learning pipeline (10 min.). - A comprehensive review of the state-of-the-art in the domain of automated combined algorithm selection and hyperparameter tuning (25 min.). - A comprehensive review of the state-of-the-art of the centralized, distributed and interactive AutoML frameworks (25 min.). - Highlights for potential research directions to improve the state-of-the-art and support the efforts towards achieving the broad vision of AutoML (10 min.). - A demo of our prototype iSmartML1, an interactive and user-guided framework for automated machine learning (10 min.). The tutorial is timely and quite relevant for the data management and machine learning research communities due to the rapid growth in the applications of machine learning in almost every application domain. The increasing momentum for developing AutoML frameworks would enrich the discussion for potential directions to improve the usability and wide acceptance of these tools among data scientists and domain experts.