To assess the accuracy of exchange-correlation approximations within density functional theory (DFT), diffusion quantum Monte Carlo (DMC) and DFT methods are used to calculate the energies of isomers of three covalently bonded carbon and boron clusters (C20, B18, and B20), and three metallic aluminum and copper clusters (Al13, Al55, and Cu13). We find that local and semilocal DFT methods predict the same energy ordering as DMC for the metallic clusters but not for the covalent clusters, implying that the DFT functionals are inadequate in such systems. In addition, we find that DFT fails to describe energy reductions arising from Jahn-Teller distortions.