The structural performance of reinforced concrete (RC) members affected by alkali-silica reaction (ASR) is difficult to predict because of the multi-scale phenomena. Recent structural tests reveal that the performance of RC members also depends on ASR-induced crack patterns, including localized cracks and dispersed microcracks. Additionally, microscopic factors, such as crack-filling by gel and presence of microcracks, are relevant. To explore this in detail, a computational system for finite element analysis of ASR-damaged RC members was developed. This study numerically investigated the structural behavior of ASR-affected RC members based on localized/dispersed crack patterns and microscopic factors. The applicability of the developed computational system was verified by comparing the analysis results with experimental data. The analysis results showed that ASR-damaged RC members with dispersed microcracks exhibited highly ductile behavior, while those with localized cracks failed in shear. This is because the dispersed crack pattern prevents the shear crack propagation and enhances the mechanical contribution of gel filling cracks, while the localized ASR cracks facilitate critical crack propagation, leading to failure, and minimize the gel-filling effect. Through the analytical investigations, it was found that the localized ASR cracks can result in significant loss of structural performance; thus, this study recommends the assessment of structural capacity of RC members in the case where the localized cracks were observed.