We report all-electron variational and diffusion quantum Monte Carlo (VMC and DMC) calculations for the noble gas atoms He, Ne, Ar, Kr, and Xe. The calculations were performed using Slater-Jastrow wave functions with Hartree-Fock single-particle orbitals. The quality of both the optimized Jastrow factors and the nodal surfaces of the wave functions declines with increasing atomic number Z, but the DMC calculations are tractable and well behaved in all cases. We discuss the scaling of the computational cost of DMC calculations with Z.