In order to tackle high-dimensional, complex problems, learning models have to go deeper. In this paper, a novel multi-layer ensemble learning model with firrst-order evolving fuzzy systems as its building blocks is introduced. The proposed approach can effectively learn from streaming data on a sample-by-sample basis and self-organizes its multi-layered system structure and meta-parameters in a feed-forward, non-iterative manner. Benefiting from its multi-layered distributed representation learning ability, the ensemble system not only demonstrates the state-of-the-art performance on various problems, but also offers high level of system transparency and explainability. Theoretical justifications and experimental investigation show the validity and effectiveness of the proposed concept and general principles.