We aimed to evaluate the associations between blood cadmium (Cd), mercury (Hg), lead (Pb), manganese (Mn), and selenium (Se) concentrations and skin cancer. This cross-sectional study was based on National Health and Nutritional Examination Survey (NHANES) data. A binomial logistic regression model was used to analyze the associations between exposure to the metal elements and the risk of skin cancer, and further stratified analyses were conducted by gender, age, body mass index, ethnicity, education, smoking, alcohol drinking, and hypertension. A total of 16,034 participants were included. After fully adjusting for multivariate, the odd ratio (OR)[95% confidence interval (95% CI)] values for skin cancer in those with blood Mn concentrations in the second, third, and fourth quartiles were 0.52 (0.33–0.82), 0.57 (0.36–0.9), and 0.56 (0.35–0.89), respectively, compared with those in the lowest quartile. The ORs (95% CI) for each 1-SD increment in log-transformed values for blood Mn concentrations were 0.79 (0.66–0.94), 0.8 (0.66–0.97), and 0.79 (0.66–0.96), respectively. A significant association between blood Hg and skin cancer was also observed in participants who drank alcohol, with a corresponding OR (95% CI) of 2.61 (1.37–5.00) (p interaction = 0.006). Our study indicated that a higher blood Mn concentration was negatively associated with skin cancer, and blood Hg was positively associated with skin cancer in participants who drank alcohol.