We quantify the fraction of the cosmic infrared background (CIB) that originates from galaxies identified in the UV/optical/near-infrared by stacking 81,250 (~35.7 arcmin-2) K-selected sources (K AB <24.0) split according to their rest-frame U - V versus V - J colors into 72,216 star-forming and 9034 quiescent galaxies, on maps from Spitzer/MIPS (24 μm), Herschel/PACS (100, 160 μm), Herschel/SPIRE (250, 350, 500 μm), and AzTEC (1100 μm). The fraction of the CIB resolved by our catalog is (69% ± 15%) at 24 μm, (78% ± 17%) at 70 μm, (58% ± 13%) at 100 μm, (78% ± 18%) at 160 μm, (80% ± 17%) at 250 μm, (69% ± 14%) at 350 μm, (65% ± 12%) at 500 μm, and (45% ± 8%) at 1100 μm. Of that total, about 95% originates from star-forming galaxies, while the remaining 5% is from apparently quiescent galaxies. The CIB at λ ~ 200 μm the bulk originates from 1 350 μm. The contribution from galaxies in the log(M/M ⊙) = 9.0-9.5 (lowest) and log(M/M ⊙) = 11.0-12.0 (highest) stellar-mass bins contribute the least—both of order 5%—although the highest stellar-mass bin is a significant contributor to the luminosity density at z >~ 2. The luminosities of the galaxies responsible for the CIB shifts from combinations of "normal" and luminous infrared galaxies (LIRGs) at λ ~ 500 μm. Stacking analyses were performed using SIMSTACK, a novel algorithm designed to account for possible biases in the stacked flux density due to clustering. It is made available to the public at www.astro.caltech.edu/~viero/viero_homepage/toolbox.html. Herschel is an ESA space observatory with science instruments provided by European-led Principal Investigator consortia and with important participation from NASA.