This article presents a micromechanics model to investigate matrix damage evolutions of the particle-reinforced composites (PRCs), as well as matrix failures effect on the stiffness degradation. Compared with the finite-element results and experiment data, it is indicated that the developed micromechanics model can be employed to effectively predict the mechanical behaviors of PRCs. The microscopic three-dimensional stress field distribution is investigated on the basis of the stress concentration region to capture the initial damage behavior of the representative volume element. Moreover, the failure criteria of Hill, Tsai-Wu and maximum stress are incorporated into the proposed micromechanics model to investigate the stiffness reduction properties of PRCs subjected to a uniaxial tensile loading. The microstructure near the interface is further refined to slow down the rapid deterioration around stress concentration regions. The results revealed that the stiffness degradation of matrix significantly affects macroscopic mechanical properties of the PRCs.