Predicting losses of wind turbine energy yield due to blade leading edge erosion is a major challenge, hindering blade predictive maintenance, and preventing further cost of energy reductions. Using jointly laser scans of operational offshore turbines, photographs of eroded leading edge samples from swirling arm rain erosion tests and validated simulation methods, this study estimates the growth of energy yield losses as erosion progresses from small-scale distributed roughness to severe damage of the leading edge. A multi-fidelity analysis is employed, combining high-fidelity computational fluid dynamics and blade element momentum theory. Erosion-induced aerodynamic performance losses are analyzed with a scale separation approach, modeling the effects of low-amplitude roughness with generalized rough-wall functions, and capturing the effects of larger geometry alterations by geometrically resolving them. The sensitivity of energy losses to the level of equivalent sand grain roughness, an uncertain parameter of the problem, is analyzed. For a typical North Sea installation site, the loss of energy grows from 0.6%, for moderate modeled roughness, to 2%, for resolved severe erosion. The largest loss at a typical Southern European onshore site is 2.5%. Severe erosion-induced energy losses are found to vary significantly with the damage topography, emphasizing the necessity or resolving larger erosion scales.