Recursive density estimation is a very powerful metric, based on a kernel function, used to detect outliers in a n-dimensional data set. Since it is calculated in a recursive manner, it becomes a very interesting solution for on-line and real-time applications. However, in its original formulation, the equation defined for density calculation is considerably conservative, which may not be suitable for applications that require fast response to dynamic changes in the process. For on-line applications, the value of k, which represents the index of the data sample, may increase indefinitely and, once that the mean update equation directly depends on the number of samples read so far, the influence of a new data sample may be nearly insignificant if the value of k is high. This characteristic creates, in practice, a stationary scenario that may not be adequate for fault detect applications, for example. In order to overcome this problem, we propose in this paper a new approach to RDE, holding its recursive characteristics. This new approach, called RDE with forgetting, introduces the concept of moving mean and forgetting factor, detailed in the next sections. The proposal is tested and validated on a very well known real data fault detection benchmark, however can be generalized to other problems.