Background and objective Traditional masculinity is characterised by traits of independence, toughness, assertiveness, competitiveness and physical competence. Multiple factors modulate the expression of masculinity, including age, social class, ethnicity and occupation. While there is a perception that physiological testosterone concentration impacts self-perceived masculinity in men, there are limited supporting data. This study aimed to examine the relationship between testosterone concentration and self-perceived masculinity as measured by the Masculinity in Chronic Disease Inventory (MCD-I), controlling for health-related and biopsychosocial factors in community-dwelling, middle-aged to elderly men. Materials and methods Participants were drawn from a longitudinally followed cohort (N=1195) of men participating in the Florey Adelaide Male Aging Study based in Adelaide, Australia. A final sample of 460 (mean age 65.15, standard deviation 9.72) men consisted of those with serum testosterone concentrations measured at wave one (2002-2005) and wave two (2007-2010), and who, in 2017, completed the Masculinity in Chronic Disease Inventory questionnaire and provided information related to demographics, medical conditions, health and lifestyle behaviours. Adjusted multivariable regression analyses were undertaken to determine the relationship between serum testosterone concentration at wave 2 and MCD-1 total masculinity score and sub-scores. Given the temporal difference between the testosterone concentration assessment and MCD-1 completion, a further analysis was performed including participants with a stable serum testosterone concentration over a 5-year period. Results There was no association between serum testosterone concentration and MCD-1 total masculinity score (p = 0.54) or sub-scores (p = 0.12-0.85). There was also no association between testosterone concentration and total masculinity score in men with stable serum testosterone over time (p = 0.35). Testosterone concentration was associated with serum sex-hormone-binding globulin (SHBG) concentration (p < 0.001), age (p < 0.001), waist circumference (p < 0.001) and a history of diabetes (p = 0.021). Total masculinity score was lower in men without a partner (widowed p < 0.013, separated/divorced p < 0.019), a history of anxiety (p = 0.036) and moderate (p = 0.05) to severe erectile dysfunction (p < 0.001). Conclusion This study provides evidence against the perception that physiological testosterone concentration impacts self-perceived masculinity. Rather, in middle-aged to older men, self-perceived masculinity is abrogated by psychosocial factors and chronic physical disorders.