There is a growing interest in applying AI technology in the field of mental health, particularly as an alternative to complement the limitations of human analysis, judgment, and accessibility in mental health assessments and treatments. The current mental health treatment service faces a gap in which individuals who need help are not receiving it due to negative perceptions of mental health treatment, lack of professional manpower, and physical accessibility limitations. To overcome these difficulties, there is a growing need for a new approach, and AI technology is being explored as a potential solution. Explainable artificial intelligence (X-AI) with both accuracy and interpretability technology can help improve the accuracy of expert decision-making, increase the accessibility of mental health services, and solve the psychological problems of high-risk groups of depression. In this review, we examine the current use of X-AI technology in mental health assessments for depression. As a result of reviewing 6 studies that used X-AI to discriminate high-risk groups of depression, various algorithms such as SHAP (SHapley Additive exPlanations) and Local Interpretable Model-Agnostic Explanation (LIME) were used for predicting depression. In the field of psychiatry, such as predicting depression, it is crucial to ensure AI prediction justifications are clear and transparent. Therefore, ensuring interpretability of AI models will be important in future research.