A new method for performing a kernel principal component analysis is proposed. By kernelizing the generalized Hebbian algorithm, one can iteratively estimate the principal components in a reproducing kernel Hilbert space with only linear order memory complexity. The derivation of the method and preliminary applications
in image hyperresolution are presented. In addition, we discuss the extension of the method to the online learning of kernel principal components.