We present the rest-frame optical spectral energy distribution (SED) and stellar masses of six Herschel-selected gravitationally lensed dusty, star-forming galaxies (DSFGs) at 1 <z <3. These galaxies were first identified with Herschel/SPIRE imaging data from the Herschel Astrophysical Terahertz Large Area Survey (H-ATLAS) and the Herschel Multi-tiered Extragalactic Survey (HerMES). The targets were observed with Spitzer/IRAC at 3.6 and 4.5 μm. Due to the spatial resolution of the IRAC observations at the level of 2″, the lensing features of a background DSFG in the near-infrared are blended with the flux from the foreground lensing galaxy in the IRAC imaging data. We make use of higher resolution Hubble/WFC3 or Keck/NIRC2 Adaptive Optics imaging data to fit light profiles of the foreground lensing galaxy (or galaxies) as a way to model the foreground components, in order to successfully disentangle the foreground lens and background source flux densities in the IRAC images. The flux density measurements at 3.6 and 4.5 μm, once combined with Hubble/WFC3 and Keck/NIRC2 data, provide important constraints on the rest-frame optical SED of the Herschel-selected lensed DSFGs. We model the combined UV- to millimeter-wavelength SEDs to establish the stellar mass, dust mass, star formation rate, visual extinction, and other parameters for each of these Herschel-selected DSFGs. These systems have inferred stellar masses in the range 8 × 1010-4 × 1011 M⊙ and star formation rates of around 100 M⊙ yr-1. This puts these lensed submillimeter systems well above the SFR-M* relation observed for normal star-forming galaxies at similar redshifts. The high values of SFR inferred for these systems are consistent with a major merger-driven scenario for star formation.