We analyze Hubble Space Telescope WFC3/H 160-band observations of a sample of 48 Atacama Large Millimeter/submillimeter Array detected submillimeter galaxies (SMGs) in the Extended Chandra Deep Field South field, to study their stellar morphologies and sizes. We detect 79% ± 17% of the SMGs in the H 160-band imaging with a median sensitivity of 27.8 mag, and most (80%) of the nondetections are SMGs with 870 μm fluxes of S 870 <3 mJy. With a surface brightness limit of μ H ~ 26 mag arcsec-2, we find that 82% ± 9% of the H 160-band-detected SMGs at z = 1-3 appear to have disturbed morphologies, meaning they are visually classified as either irregulars or interacting systems, or both. By determining a Sérsic fit to the H 160 surface brightness profiles, we derive a median Sérsic index of n = 1.2 ± 0.3 and a median half-light radius of re = 4.4+1.1-0.5 kpc for our SMGs at z = 1-3. We also find significant displacements between the positions of the H 160 component and 870 μm emission in these systems, suggesting that the dusty starburst regions and less-obscured stellar distribution are not colocated. We find significant differences in the sizes and the Sérsic index between our z = 2-3 SMGs and z ~ 2 quiescent galaxies, suggesting that a major transformation of the stellar light profile is needed in the quenching processes if SMGs are progenitors of the red-and-dead z ~ 2 galaxies. Given the short-lived nature of SMGs, we postulate that the majority of the z = 2-3 SMGs with S 870 gsim 2 mJy are early/mid-stage major mergers.