Understanding the implementation issues when using deep learning frameworks

Context: Deep Learning (DL) frameworks like TensorFlow can help developers implement DL applications (e.g., computer vision) faster and easier. When using DL frameworks, developers encountered a large number of questions and posted them on Stack Overflow (SO). Objective: The goal of this paper is to conduct a comprehensive empirical study on the SO questions, summarize the implementation issues, and suggest future opportunities.Methods: This paper focuses on three DL frameworks (i.e., TensorFlow, PyTorch, and Theano), groups 2,401 relevant SO questions into various implementation issues, and constructs a taxonomy. We also analyze the popularity and difficulty of these issues under the taxonomy.Results: For the identified various implementation issues, we constructed a taxonomy consisting of seven major categories with 63 subcategories. Our analysis reveals that 91.7% of questions are related to the implementation categories of data processing, model setting, model training, and model prediction. Developers frequently address the remaining three categories (i.e., Model evaluation, runtime environment, and visualization), where runtime environment is the most difficult category. Based on empirical findings, we provide some suggestions for future research. Conclusion: In this paper, we summarized the issues of DL implementation and proposed corresponding opportunities for future study. We expect this paper to help developers and researchers understand these issues and design better tools to improve the productivity of DL implementation.