We present Keck-Adaptive Optics and Hubble Space Telescope high resolution near-infrared (IR) imaging for 500 μm bright candidate lensing systems identified by the Herschel Multi-tiered Extragalactic Survey and Herschel Astrophysical Terahertz Large Area Survey. Out of 87 candidates with near-IR imaging, 15 (~17%) display clear near-IR lensing morphologies. We present near-IR lens models to reconstruct and recover basic rest-frame optical morphological properties of the background galaxies from 12 new systems. Sources with the largest near-IR magnification factors also tend to be the most compact, consistent with the size bias predicted from simulations and previous lensing models for submillimeter galaxies (SMGs). For four new sources that also have high-resolution submillimeter maps, we test for differential lensing between the stellar and dust components and find that the 880 μm magnification factor (μ880) is ~1.5 times higher than the near-IR magnification factor (μNIR), on average. We also find that the stellar emission is ~2 times more extended in size than dust. The rest-frame optical properties of our sample of Herschel-selected lensed SMGs are consistent with those of unlensed SMGs, which suggests that the two populations are similar.