Efficient Cloud-Based Framework for Big Data Classification

Big Data is a term that describes the large volume of data both structured and unstructured that is difficult to process using traditional database and software techniques. Cloud computing is a technology that offers a solution to this problem. We have designed a cloud-based framework for unstructured data analysis that is motivated by the goal of efficient image data analysis. The framework consists of two general stages of feature extraction and machine learning that can be used in training mode and classification mode. The framework uses sampling and feedback mechanisms whereby the system learns which image features are most important, and also learns which algorithm(s) is best (under user criteria of accuracy and speed). This information allows the system to be more efficient by automatically reducing the number of features captured, and number of algorithms being used to evaluate the data. While there is an overhead to the auto adjusting mechanisms, they do lead to a more efficient solution for big data sets. The solution is evaluated using a leaf images classification problem, and the results shows the improvements in efficiency.