—This article considers perception-driven control of a mobile robot for path tracking where perception is performed by a machine learning system. The robot is subject to passive attacks and evasion attacks on image transmission. To defeat the passive attacks, we adopt chaotic encryption technique to disguise pixels of plain images in real time, and construct a bank of fuzzy unknown input observers to decrypt the cipher pixels in parallel. We characterize the security level of the proposed chaotic cryptographic scheme. As for the path tracking, we derive a set of linear matrix inequality (LMI) conditions of the existence of a robust controller, which renders the output zeroing manifold attractive and invariant by using internal model technique, and also attenuates the effects of the evasion attacks and learning errors of the machine learning system by reducing L2 gain. Simulations are conducted in the CARLA simulator to demonstrate robust path tracking and secure image transmission.