How to leverage the data distributed among/between multiple parties to efficiently and securely enforce high-performance machine learning training and inference with privacy preservation has become a hot spot of two research topics, i. e., secure multi-party computation and machine learning. This paper proposes the concept of secure multi-party learning based on the investigation of the latest developments in the hot spot. Secure multi-party learning, a research topic in (secure) software engineering rather than cryptography, hereby refers to the methods, frameworks, and platforms that enforce privacy-preserving machine learning based on secure multi-party computation. It enables multiple parties to perform secure training and secure inference of machine learning models without directly leveraging their plaintext data and any private information beyond the final result. Therefore, secure multi-party learning can be applied to several practical fields involving private data, such as risk control in the financial field and medical diagnosis. Researchers have proposed a dozen of secure multi-party learning frameworks recently. Considering the rapid development of secure multi-party learning, a comprehensive and systematic survey, which covers the underlying technologies and classification of secure multi-party learning frameworks, is still absent so far. Therefore, this paper is motivated to conduct a literature review of the categories, characteristics, and frameworks of secure multi-party learning to help researchers choose suitable secure multi-party learning frameworks for various scenarios» further identify research gaps» and improve the weaknesses of secure multi-party learning frameworks. This paper analyzes the security models, system deployment methods, and functional scenarios in secure multi-party learning and starts with the underlying secure multi-party computation primitives and the privacy-preserving technologies to summarize secure multi-party learning frameworks systematically and comprehensively. The underlying technologies used in secure multi-party learning include holomorphic encryption, oblivious transfer, garbled circuit» and secret sharing. According to these underlying technologies, secure multi-party learning frameworks are classified into four categories: homomorphic encryption-based secure multi-party learning frameworks» garbled circuit-based secure multiparty learning frameworks, secret sharing-based secure multi-party learning frameworks» and mixed-protocol-based secure multi-party learning frameworks. Besides, this paper summarizes the characteristics of these four categories of secure multi-party learning frameworks from six aspects: computational complexity, communication rounds, communication size, linear operation efficiency, nonlinear operation efficiency, and functional scenarios supported. Further, this paper investigates 38 typical secure multi-party learning frameworks and compares them regarding the number of parties supported, security models, functional scenarios supported, machine learning models supported, activation functions supported, pooling implemented, and accuracy. Then, this paper analyzes the differences between secure multi-party learning and other privacy-preserving machine learning techniques» including federated learning and confidential computing based on trusted execution environment. Finally, this paper presents suggestions for future development of secure multiparty learning as follows: (1) to improve security, including support for a security model with stronger security guarantees, access control to the final model, and protection of the final model; (2) to prove the security of secure multi-party learning processes by the universally composable scheme; (3) to improve performance and efficiency by reducing the online communication overhead, accelerating the local computation with GPU and designing the machine learning models that adapt to the underlying technologies of secure multi-party learning; (4) to realize interoperability between secure multi-party learning frameworks. © 2023 Science Press. All rights reserved.
