The detection of anomalies in the secondary lining of tunnel using ground penetrating radar (GPR) is affected by many factors, such as the difference of linear polarization mode caused by varying angle between the antenna connection and the detection direction. Analyzing the modes of linear polarization would offer guidance for construction and signal interpreting in detecting the defects of tunnel’s secondary lining using GPR. In this paper, the detection effects corresponding to horizontal polarization and vertical polarization are examined by establishing numerical models via GprMax3D. Numerical simulation and engineering example analysis reveal that with the angle varying from 0° to 90°, 1) the effectiveness of detecting steel bars in different depths in horizontal polarization mode attenuates gradually, while the effectiveness with vertical polarization remains unchanged; 2) the effect of detecting cavities of the secondary lining in horizontal polarization mode also diminishes gradually, whereas the effect with vertical polarization stays stable; 3) on the contrary, the detection of thickness-insufficient zone in horizontal polarization mode strengthens, but in vertical polarization mode the detection effect weakens; 4) the detection of crushed zone is barely affected by polarization mode but by the value of dielectric constant of surrounding rock. The above research findings can be used to guide the quality inspection for the secondary lining of tunnels.