To reflect the mechanical properties of composite geomembrane in 3-D finite element calculation is a technical problem to be solved. On the basis of summarizing current numerical methods of geomembrane simulation, we propose to simulate the geomembrane using membrane element in 3-D finite element calculation. By building a finite element model of high earth-rock cofferdam of 65 m height with geomembranes contacted with dam in the upstream and downstream, we simulated the flexible and rigid constraint conditions of geomembrane boundary respectively in filling construction and impounding stages. Results suggest that the membrane element could well reflect the flexible tensile properties of geomembrane. The proposed method could also adapt to the stepped filling in 3-D finite element simulation. Moreover, it could conveniently and effectively display the displacements and stresses of geomembranes in post-processing. The displacement distribution of geomembrane under flexible boundary restraint was the same with that under rigid boundary constraint; while the principal tensile stress and its distributed region under flexible boundary constraint were slightly smaller than those under rigid constraint. The calculation results are consistent with the general rules of earth-rockfill dam.