End-to-End Automation of ML Model Lifecycle Management using Machine Learning Operations Platforms

In machine learning research, automation presents numerous challenges, particularly in areas such as environment setup, model deployment, and maintenance, which can lead to significant time consumption and tedious tasks. To address these challenges, Machine Learning Operations (MLOps) has emerged as a solution. In this study, we propose to use the open-source Kubeflow to tackle the challenges. Kubeflow provides tools like Pipeline, Katib, and Kserve, which support tasks such as training, hyperparameter tuning, model comparison, deployment, and maintenance.The modular design of Pipeline is highly beneficial for debugging and ensuring environment consistency. The tight integration between Katib and Kubeflow enables highly automated hyperparameter tuning. Kserve addresses the issue of manual deployment, reducing the potential for human error. Our experimental result demonstrates that leveraging Kubeflow and its associated tools allows for a more streamlined and automated approach to machine learning operations, mitigating many of the challenges and labors associated with manual processes.