In this investigation, a new algorithm is developed for the updating of a neural network. It is concentrated in a fuzzy transition between the recursive least square and extended Kalman filter algorithms with the purpose to get a bounded gain such that a satisfactory modeling could be maintained. The advised algorithm has the advantage compared with the mentioned methods that it eludes the excessive increasing or decreasing of its gain. The gain of the recommended algorithm is uniformly stable and its convergence is found. The new algorithm is employed for the modeling of two synthetic examples.