The complex nature of the foreign exchange (FOREX) market along with the increased interest in the currency exchange market has prompted extensive research from various academic disciples in aiding traders in their in-depth analysis and decision making processes. An approach incorporating the use of historical data along with computational intelligence for analysis and forecasting is proposed in this paper. Firstly, the Gaussian Mixture Model method is applied for the purpose of data partitioning on historical observations. Then, the antecedent part of the neuro-fuzzy system of AnYa type is initialized by the partitioning result and the consequent part is trained using the fuzzily weighted RLS algorithm based on the same data. Numerical examples based on the real currency exchange data demonstrated that the proposed approach trained with historical data is able to produce optimizing results on forecasting the future foreign exchange rates for a very long period, and also show the potential of the proposed approach in real applications.