Sentinel-3 is a newly launched satellite implemented by the European Space Agency (ESA) for global observation. The Ocean and Land Colour Imager (OLCI) sensor onboard Sentinel-3 provides 21 band images with a fine spectral resolution and is of great value for ocean, land and atmospheric monitoring. The two platforms (Sentinel-3A and -3B) can provide OLCI images at an almost daily temporal resolution. The coarse spatial resolution of the 21 band OLCI images (i.e., 300 m), however, limits greatly their utility for local, precise monitoring. Sentinel-2, another satellite provided by ESA, carries the Multispectral Imager (MSI) sensor which can supply much finer spatial resolution (e.g., 10 m and 20 m) images. This paper introduces a new fusion framework integrating spatio-temporal-spectral information for downscaling Sentinel-3 OLCI images, which has two parts. Based on bands with similar wavelengths (i.e., bands 2, 3, 4 and 8a for Sentinel-2 and bands Oa4, Oa6, Oa8 and Oa17 for Sentinel-3), the four Sentinel-3 bands are first downscaled to the spatial resolution of Sentinel-2 images by applying spatio-temporal fusion to Sentinel-2 MSI and Sentinel-3 OLCI images. Then, to take full advantage of all 21 available OLCI bands of the Sentinel-3 images, the extended image pair-based spatio-spectral fusion (EIPSSF) method is proposed in this paper to downscale the other 17 bands. EIPSSF is performed based on the new concept of the extended image pair (EIP) and by exploiting existing spatio-temporal fusion approaches. The framework consisting of spatio-temporal and spatio-spectral fusion is entirely general, which provides a practical solution for comprehensive downscaling of Sentinel-3 OLCI images for fine spatial, temporal and spectral resolution monitoring.