The agriculture sector contributes significantly to the economic growth of a country. However, plant diseases are one of the leading causes of crop destruction that decreases the quality and quantity of agricultural produce, which can further cause huge economic losses. Therefore, it is imperative to detect diseases in plants well in advance in order to avoid crop destruction. In past studies, authors have employed either machine learning or deep learning techniques for predicting plant diseases, however machine learning techniques lacks the facility to automatically extract features, whereas deep learning is computationally expensive and needs a large volume of training data to achieve effective classification performance. Therefore, the current study proposed a novel framework that integrates the advantages of both machine learning and deep learning. The proposed framework includes 40 different Hybrid Deep Learning models that contain the combination of eight different variants of pre-trained deep learning architecture, viz., EfficientNet (B0–B7) as feature extractors and five machine learning techniques, i.e., k-Nearest Neighbors (kNN), AdaBoost, Random Forest (RF), Logistic Regression (LR), and Stochastic Gradient Boosting as classifiers. Optuna framework has been used in the current study to optimize the hyperparameters of these classifiers. To conduct this study, a real-time image dataset of tomato early blight disease has been collected from Indian Agricultural Research Institute. The proposed HDL models performed exceptionally well on the IARI-TomEBD dataset and have achieved a high level of accuracy in the range of 87.55–100%. Further, the validation of the proposed approach has been done using two openly available plant disease datasets, i.e., PlantVillage-TomEBD and PlantVillage-BBLS. Finally, the Friedman statistical test has also been performed to calculate the mean rank of HDL models. Results show that EfNet-B3-ADB and EfNet-B3-SGB achieved the highest rank for all the three plant disease datasets. A farmer can use this model to reduce their workload, which in turn will allow them to treat disease early and cure it. The proposed approach will, therefore, prevent plants from getting deteriorated at an early stage.