Certain inherent psychological risks in pilots pose challenges that are difficult to tackle using low-cost flight training. Evaluating the psychological competence of pilots or aspiring flight students is crucial, and this study introduces a pilot's psychological competency measurement tool and its application. The measurement framework and method for evaluating airline transport pilot's psychological competency were developed based on the professional characteristics of airline transport pilots and the requirements of implementing pilots' Professionalism Lifecycle Management (PLM) system proposed by the Civil Aviation Administration of China (CAAC). The evaluation tool, grounded in occupational adaptability psychology and mental health considerations, focused on six dimensions: general cognitive ability, operational and professional ability, social-interpersonal ability, personality traits and attitude, mental quality, and mental state. To measure general cognitive ability and operational and professional ability, a combination of flight control sticks, a computer, and a tablet was employed. Social-interpersonal ability, personality traits and attitude, mental quality, and mental state were assessed using a tablet. Collaborating with various airlines and flight academies, the data collection phase was executed in March, July, and October of 2023. Robust and valid psychological competency data were collected from a diverse pool, including 22 airline pilots (average age 27.23 +/- 3.32 years), 169 ab initio pilots (average age 24.57 +/- 1.52 years), and 130 flight students (average age 22.38 +/- 0.791 years). The findings from this comprehensive analysis illuminated some distinctions among ab initio pilots, airline pilots, and flight students. Notably, flight students show-cased exceptional performance in the dimension of general cognitive abilities, demonstrating their cognitive prowess. On the other hand, airline pilots exhibited superior skills in flight operational capabilities, highlighting their expertise in psychomotor skills. However, a noteworthy concern emerged regarding flight students, revealing a higher prevalence of personality disorders and dangerous attitudes. Additionally, flight students exhibited a more intense negative mental health state, raising critical considerations about their overall psychological well-being. In contrast, airline pilots and ab initio pilots displayed more positive psychological health overall. In conclusion, this research initiative goes beyond the mere introduction of a valuable psychological competency measurement tool; it delves into a comprehensive analysis of its practical application. The collaborative endeavors with airlines and flight academies, paired with the meticulous collection and analysis of data, not only vividly showcase the user-friendliness and practical applicability of the tool but also significantly contribute to the ongoing discourse on the optimization of pilot training programs.