As an important part of the subgrade, the filling material quality of subgrade bed has a significant influence on the dynamic response of the subgrade, track, and train. In practical engineering, although filling material of group A, B can be directly used according to the design code in China, improved soils and even unimproved soils also may be used if the economic factors and mechanical properties are considered to meet the requirements. In order to study the influence of different filling materials, such as red mudstone, filling material of group A, B, and improved red mudstone, on dynamic response of subgrade, field vibration tests are carried out. According to the comparative analysis of the tested data, the following conclusions about dynamic stress, acceleration, and settlement have been obtained. When the red mudstone is used, the dynamic stress and settlement can meet the design requirements during the loading period. Compared with the responses when using red mudstone, the distributions of dynamic stress and acceleration along the cross-section of the subgrade surface are more uniform when filling material is group A, B and lime-modified red mudstone, so the overall bearing capacities of these subgrades are better at this time. Besides, the attenuations of the dynamic stress and acceleration along the depth of the subgrade are more obvious than that in the red mudstone; the settlement of the subgrade is smaller, especially when the improved red mudstone is used.