In this chapter, a new type of deep rule-based (DRB) classifier with a multi-layer architecture is presented for image classification, which combines the computer vision techniques with a massively parallel set of zero-order fuzzy rules as its learning engine. With its prototype-based nature, the DRB classifiers are able to identify a transparent and human-understandable fuzzy rule-based (FRB) system structure from the data through an autonomous, non-iterative, non-parametric and highly parallel online learning process, and offer extremely high classification accuracy. The DRB classifier can start “from scratch”, and conduct classification from the very first image of each class in the same way as humans do. The DRB classifier can also learn in a semi-supervised mode initialized with only a small proportion of the labelled data and continue in a fully unsupervised mode after that. The ability of semi-supervised learning further allows the DRB classifier to learn new classes actively without human experts’ involvement. Thanks to the prototype-based nature of the DRB classifier, it is free from prior assumptions about the type of the data distribution, their random or deterministic nature, and there are no requirements to make ad hoc decisions. Its supervised and semi-supervised learning processes are fully transparent and human-interpretable. The semi-supervised DRB classifiers can perform classification on out-of-sample images and also support recursive online training on a sample-by-sample basis or a batch-by-batch basis. © 2019, Springer Nature Switzerland AG.