In India, agricultural production is a crucial factor for economic growth. However, plants can be attacked by diseases that range from mild to severe, leading to their destruction. Therefore, it is essential to detect plant diseases at an early stage to prevent such damage. To address this issue, we propose a Hybrid Xception transfer learning with crossover optimized kernel extreme learning machine (HXTL-COKELM) method for identifying healthy and diseased plant leaves. The proposed approach preprocesses and extracts image features using transfer learning with the Xception model to improve computation accuracy before classification. Additionally, we developed a crossover-based tasmanian devil-optimized kernel extreme learning machine model to optimize the KELM’s parameters using the TDO algorithm. We collected a dataset with 43,466 images of plant leaves from healthy and diseased plants to train, test, and validate the CNN model. Finally, we evaluated the system's efficacy using performance metrics such as accuracy, recall, specificity, F1-score, and kappa static value. Our experimental findings show that the proposed HXTL-COKELM method achieves a 98.9% accuracy rate, outperforming existing methods. The HXTL-COKELM model also exhibits superior overall performance. Thus, the proposed HXTL-COKELM model can effectively detect plant leaf diseases.