In a 16-years long-term fertilizer experiment, an in-depth study was carried out to evaluate the changes in soil physical, chemical, and biological properties under long-term fertilizer application and establish cause and effect relationship between soil properties and rice productivity using interpretable machine learning. There were 12 treatments involving control (without fertilizer application), 100% N (recommended dose of nitrogen), 100% NP (recommended dose of nitrogen and phosphorus), 100% PK (recommended dose of phosphorus and potassium), 100% NPK (recommended dose of nitrogen, phosphorus, and potassium), 150% NPK (50% higher nitrogen, phosphorus, and potassium than recommended), 100% NPK + Zn (recommended nitrogen, phosphorus, and potassium along with Zinc), 100% NPK + FYM (recommended nitrogen, phosphorus, and potassium along with farmyard manure (FYM)), 100% NPK + FYM + LIME (recommended nitrogen, phosphorus, and potassium along with FYM and lime), 100% NPK + Zn + S (recommended nitrogen, phosphorus, and potassium along with zinc and sulphur), 100% NPK + Zn + B (recommended nitrogen, phosphorus, and potassium along with Zinc and Boron) and 100% NPK + Lime (recommended nitrogen, phosphorus, and potassium along with lime). At first, a conditional random forest model was built, based on which important variables were selected using the permutation-based variable importance approach. Further, the accumulated local effect plot was used to establish a cause and effect relationship between important soil properties and rice yield. Although most of the soil properties varied across the treatments, total potassium, protease, urease, and permanganate oxidisable carbon are the most important soil properties, individually accounting for up to 400 kg ha-1 variation in the rice productivity. The study demonstrated how interpretable machine learning techniques could be used in long-term fertilizer experiments to unravel the most meaningful information, and these techniques can be used in other similar long-term experiments.