Aim The purpose of this study was to clarify whether fatty pancreas might lead to impaired pancreatic endocrine or exocrine function. Material and methods The study involved 109 participants who had undergone the glucagon stimulation test and N-benzoyl-L-tyros-p-amino benzoic acid (BT-PABA) test to assess pancreatic function as well as unenhanced abdominal computed tomography (CT). Pancreatic endocrine impairment was defined as AC peptide immunoreactivity less than 2 [mmol/L] in the glucagon stimulation test, and pancreatic exocrine impairment was defined as a urinary PABA excretion rate less than 70% on the BT-PABA test. We defined as the mean CT value of pancreas / CT value of spleen (P/S ratio) as a marker to assess fatty pancreas. We analyzed the association between fatty pancreas and pancreatic impairment using the logistic regression model. The odds ratio (OR) is shown per 0.1 unit. Results Pancreatic endocrine function was impaired in 33.0% of the participants, and 56.9% of those were regarded as having pancreatic exocrine impairment. The P/S ratio was significantly correlated with pancreatic endocrine impairment in univariate analysis (OR = 0.61, 95% confidence interval (CI) = 0.43-0.83, P = 0.0013) and multivariate analysis (OR = 0.38, 95% CI = 0.22-0.61, P < .0001) for all participants. Similar significant relationships were observed in both univariate (OR = 0.70, 95% CI = 0.49-0.99, P = 0.04) and multivariate (OR = 0.39, 95% CI = 0.21-0.66, P= 0.0002) analyses for the participants without diabetes (n = 93). The amount of pancreatic fat was not associated with exocrine impairment in univariate analysis (OR = 0.80, 95% CI = 0.59-1.06, P = 0.12). Conclusion Fatty pancreas was associated with pancreatic endocrine impairment but did not have a clear relationship with pancreatic exocrine impairment.