Infectious agents have been suspected as contributing factors to dementia, especially in Alzheimer disease. We intended to test whether viral or bacterial seropositivity is associated with cognitive impairment among home-dwelling elderly. Viral burden (seropositivity for herpes simplex type 1 (HSV1), type 2 (HSV2), or cytomegalovirus (CMV), and bacterial burden (Chlamydia pneumoniae and Mycoplasma pneumoniae) were tested among 383 home-dwelling individuals with vascular disease (mainly coronary heart disease) in the ongoing DEBATE study (mean age 80 years). Mini-mental state examination (MMSE) and its changes were used to define cognitive impairment. At baseline, 0-1, 2, and 3 positive titers toward viruses were found in 48 (12.5 %), 229 (59.8 %), and 106 (27.7 %) individuals, respectively. MMSE points decreased with increasing viral burden (p = 0.03). At baseline, 58 individuals (15.1 %) had cognitive impairment (MMSE < 24 points) which after adjustments was significantly associated with seropositivity for 3 viruses (risk ratio 2.5, 95 % confidence interval 1.3 to 4.7). MMSE score decreased in 150 cases (43 %) during 12-month follow-up. After adjustment for MMSE score at baseline and with 0-1 seropositivities as reference (1.0), the risk ratios were 1.8 (95 % confidence interval 0.9 to 3.6) and 2.3 (95 % confidence interval. 1.1 to 5.0) for 2 and 3 seropositivities, respectively. No significant associations were observed between bacterial burden and cognition. Viral burden of herpesvirus and cytomegalovirus was associated with cognitive impairment in home-dwelling elderly. The association may offer a preventable cause of cognitive decline.