The compressive performance of concrete was tested under uniaxial lateral stress and vertically equal strain cyclic loading and unloading. The envelopes, cyclic loading and unloading curves, and common-point (point of the intersection between unloading curve and reloading curve) tracks under different lateral stresses were analyzed. Moreover, the influence of lateral stress on stiffness degradation and cumulative residual plastic strain was studied. The energy release characteristics in the whole loading process were explored, and the damage characteristics of concrete were analyzed based on the damage variable determined by the number of AE event. Results revealed that 1) throughout the course of cyclic loading and unloading tests, the number of AE event was concentrated before the peak stress, and became less after the peak stress, and the AE energy also increased with the increase of lateral stress after peak stress; 2) with the increase of lateral compressive stress, concrete’s stiffness degradation rate and accumulated residual plastic strain decreased; 3) the damage development path was different under different lateral stresses. The higher the lateral stress, the shorter the damage path. The size ratio between horizontal and vertical loads determines the dominant factors and damage mechanism of concrete.