Evaluating lightning forecasts of a convective scale ensemble prediction system over India

In this comprehensive study, we have delved into the intricate realm of lightning forecasting, a captivating yet challenging pursuit due to the elusive nature of storm electrification. Our focus rested on evaluating the skill of lightning forecasts of NCMRWF Regional Ensemble Prediction System (NEPS-R), based on the Met Office Global and Regional Ensemble Prediction System (MOGREPS). Based on the lightning hotspots in the observed seasonal mean lightning flash count, study regions were identified over East-Northeast India (ENEI) and Peninsular India (PI) during pre-monsoon season and Central and East-Northeast India (CENEI) for monsoon season during the year 2022. Probabilistic and deterministic skill of NEPS-R in lightning prediction using rank histogram, Relative Operating Characteristic (ROC), Brier Skill Score (BSS), Continuous Ranked Probability Skill Score (CRPSS), reliability diagrams, Probability of Detection (POD), False Alarm ratio (FAR), Equitable Threat Score (ETS), and Fractions Skill Score (FSS) were investigated in the present study. Rank histograms brought out the negative bias, which increases with increasing lead time over all the study regions. While ROC and BSS indicate that forecast is skillful on day-1 over ENEI for all the thresholds (> 1, > 5 and > 10) and over PI on day-2 and day-3, CRPSS has a contrasting trend with most skillful forecast over CENEI region. Reliability diagrams indicate under-forecasting for lower probabilities and over-forecasting for higher probabilities for all the thresholds for all the study areas. The overconfidence in the forecast could be at least partially attributed to the sampling error caused by a small ensemble size of NEPS-R, which also displayed similar trend of skill as BSS. Higher POD values in the CENEI region have surpassed those in ENEI and PI. Lower FAR and higher ETS are found over PI region compared to ENEI and CENEI. Furthermore, FSS (threshold > 1) indicates that CENEI region attains better skill (> 0.5) at a neighborhood size of 36 km in day-1 forecast compared to ENEI (68 km) and PI (84 km). The skill scores do emphasize the model's ability in predicting the dominant large scale features of monsoon. This is reflected in the better skill of CENEI region during monsoon as compared to ENEI and PI regions during the pre-monsoon season, where the model is falling short in predicting the prominent local scale features. Also, the model's inadequacy in proper representation of microphysics and kinematics does affect the lightning prediction capability of the model which could be addressed with improved model initial conditions and parameterization schemes and finer horizontal resolution.