Downscaling has an important role to play in remote sensing. It allows prediction at a finer spatial resolution than that of the input imagery, based on either (i) assumptions or prior knowledge about the character of the target spatial variation coupled with spatial optimisation, (ii) spatial prediction through interpolation or (iii) direct information on the relation between spatial resolutions in the form of a regression model. Two classes of goal can be distinguished based on whether continua are predicted (through downscaling or area-to-point prediction) or categories are predicted (super-resolution mapping), in both cases from continuous input data. This paper reviews a range of techniques for both goals, focusing on area-to-point kriging and downscaling cokriging in the former case and spatial optimisation techniques and multiple point geostatistics in the latter case. Several issues are discussed including the information content of training data, including training images, the need for model-based uncertainty information to accompany downscaling predictions, and the fundamental limits on the representativeness of downscaling predictions. The paper ends with a look towards the grand challenge of downscaling in the context of time-series image stacks. The challenge here is to use all the available information to produce a downscaled series of images that is coherent between images and, thus, which helps to distinguish real changes (signal) from noise.