Because of the great mechanical properties and advantages of environmental protection, basalt fiber has gradually become a common fiber modifier in asphalt mixture. Due to the difference in reinforcing mechanism, the reinforcing effect of basalt fiber and lignin fiber on asphalt mixture is different. In this study, the performance of the SMA-13L mixture with basalt fiber, lignin fiber, and composite fiber was explored through experimental methods. The rutting test, the immersion Marshall test, the freeze-thaw indirect tensile test, the bending test, and the sand patch test were carried out. Compared with the mixture with more commonly used lignin fiber, the mixture with basalt fiber has better crack resistance and skid-resistant property but worse high-temperature stability and moisture susceptibility. The composite fiber composed of lignin fiber and basalt fiber can integrate the advantages of the two, reflecting a more comprehensive performance.