In this work, an embedded system for fault detection and diagnosis of photovoltaic (PV) modules based on infrared thermographic images and deep conventional neural networks (DCNNs) is introduced. First, a binary classifier is developed for PV modules fault detection. Then, a multiclass classifier is developed to diagnose the type of defects occurred on PV modules. In this study, four common defects are examined: partial shading effect, dust deposit on PV modules surface, short-circuited PV module and bypass diode failure. The developed DCNN-based classifiers have been first optimized and then embedded into a low-cost microprocessor (Raspberry Pi 4). The models have also been compared with three main TF-Lite optimization techniques (Simple conversion, Dynamic range quantization and Float 16 quantization). Experimental results demonstrate the feasibility of the developed embedded system to operate in real-time, and can detect and diagnose anomalies with acceptable accuracy. The trade-off between accuracy and models size has been also discussed. Furthermore, the operator could be notified about the state of the PV array through SMS (phone message) using a GSM module (SIM808), as well as by email. This embedded solution can help to make real-time analyses for decision making (i.e. removing fault, cleaning PV modules, changing PV modules, replacing diodes, etc.).