Humans have the intrinsic ability to determine if fruit is fresh. However, there has not been much interest in deep learning research that aims to create a fruit grading system based on digital pictures. Fruit waste or fruit being thrown away can both be avoided by using the method proposed in this article. In this study, we provide a comprehensive analysis of the freshness rating system using computer vision and deep learning. The visual analysis of digital pictures serves as the foundation of our grading system. ResNet, VGG, and GoogLeNet are only a few of the deep learning methods used in this research. The area of interest (ROI) in digital photographs is found using YOLO, with AlexNet as the base network. This study proposes a model based on various convolutional neural networks (CNNs) types to quickly and accurately assess fruit quality. The proposed model effectively captured particular, complex, and beneficial visual characteristics for detection and categorization. The suggested model performed better than earlier techniques at learning high-order features of two adjacent layers that were strongly coupled but not in the same channel. The suggested model was tested and validated, and method’s correct classification rate (CCR) for apples, bananas, and oranges was 93.5%, 90.5%, and 92.5%, respectively, while the ANN’s CCR was 85.5%, 89.5%, and 88.3% on Indian fruits image dataset. © 2023, The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd.