We investigate the spin dynamics driven by terahertz magnetic fields in epitaxial thin films of cobalt in its three crystalline phases. The terahertz magnetic field generates a torque on the magnetization which causes it to precess for about 1 ps, with a subpicosecond temporal lag from the driving force. Then, the magnetization undergoes natural damped THz oscillations at a frequency characteristic of the crystalline phase. We describe the experimental observations solving the inertial Landau-Lifshitz-Gilbert equation. Using the results from the relativistic theory of magnetic inertia, we find that the angular momentum relaxation time η is the only material parameter needed to describe all the experimental evidence. Our experiments suggest a proportionality between η and the strength of the magnetocrystalline anisotropy.