Data reduction in big data: a survey of methods, challenges and future directions

Data reduction plays a pivotal role in managing and analyzing big data, which is characterized by its volume, velocity, variety, veracity, value, variability, and visibility. However, several surveys have been conducted to summarize these techniques in the field of big data, and there are several concerns that require attention, such as limited discussions of reduction techniques. Also, most of these studies focused on applications and only described their techniques. In contrast, this survey provides a comprehensive overview of data reduction methods, challenges, and future directions in the context of big data analytics in general concepts. This survey begins discussing the significance of data reduction in addressing the scalability and complexity issues inherent in big data processing. Subsequently, a classification data reduction method in big data is provided. For each category, the underlying principles, popular algorithms, and applications in big data analytics are highlighted. Moreover, the key challenges associated with data reduction in the era of big data, such as scalability, computational complexity, quality preservation, and interpretability, are found and discussed, while the importance of addressing these challenges to ensure the effectiveness and reliability of data reduction techniques in large-scale data analytics are reviewed. This survey can serve as a comprehensive reference for researchers, practitioners, and stakeholders interested in understanding and using data reduction techniques to address the challenges and opportunities posed by big data. Finally, tangible results of this study can be listed as introducing techniques for improving storage efficiency and faster computational processing by minimizing dataset size, while these techniques can enhance data analysis by removing redundancy and noise, leading to more accurate and actionable insights.