RFID-based localization and tracking has some promising potentials. By combining localization with its identification capability, existing applications can be enhanced and new applications can be developed. In this paper, we investigate a tag-free indoor localizing and tracking problem (e.g., people tracking) without requiring subjects to carry any tags or devices in a pure passive environment. We formulate localization as a classification task. In particular, we model the received signal strength indicator (RSSI) of passive tags using multivariate Gaussian Mixture Model (GMM), and use the Expectation Maximization (EM) to learn the maximum likelihood estimates of the model parameters. Several other learning-based probabilistic approaches are also explored in the localization problem. To track a moving subject, we propose GMM based Hidden Markov Model (HMM) and k Nearest Neighbor (kNN) based HMM approaches. We conduct extensive experiments in a testbed formed by passive RFID tags, and the experimental results demonstrate the effectiveness and accuracy of our approach.