Background We analyzed the racial differences in physical well-being, social/family well-being, functional well-being, emotional well-being, and prostate cancer specific concerns among men with clinically localized prostate cancer. Methods In this prospective cohort study, we recruited 121 Caucasian and 77 African American men with newly diagnosed prostate cancer. Participants completed the Functional Assessment of Cancer Therapy-Prostate (FACT-P) instrument at baseline and at 3, 6, 12 and 24 months follow-up. To determine the between and within group differences on all functional measures (physical well-being, social/family well-being, functional well-being, emotional well-being, and prostate cancer specific concerns), we used repeated measures ANOVA. Results Except for education, income and hospital type, the two racial groups had comparable socio-demographic and clinical attributes. At 3 months post-treatment, both groups experienced a decline in physical well-being and increase in prostate cancer specific concerns. Baseline values of these measures were not recovered by 24-month follow-up. In particular, lack of energy was the most prominent contributor to declining physical well-being. The ability to have and maintain an erection was the most severe prostate cancer specific concern. Lack of energy and concern regarding erection ability varied between the two groups over the follow-up period. For African American patients, there was substantial temporal variation in this item, whereas Caucasian patients reported improved energy over time. Conclusions Our findings provide an insight into not only the overall racial variation in physical well-being and prostate cancer specific concern, but also the temporal differences in these measures that can occur over a 24-month follow-up. This has important implications for effective management of localized prostate cancer patients from different racial groups and merits further research. Implications for cancer survivors Addressing the issue of lack of energy and concern regarding ability to have and maintain an erection can help improve physical well-being and alleviate prostate-specific concerns in survivors of localized prostate cancer.