Currently, several competing computational frameworks, including free, open‐source, and commercial packages exist, that enable users to perform purely electrochemical simulations based on the Doyle‐Fuller‐Newman (DFN) model or simulations additionally coupled with thermal/mechanical physics. In this paper, the performance of several numerical software packages is reviewed and evaluated based on the ease of model setup, spatial dimension capability, and model accuracy. In the absence of standardized benchmark tests, a series of discharge simulations under various operating conditions, such as static and dynamic electric vehicle driving cycle loads, as well as galvanostatic intermittent titration techniques (GITTs), provide rigorous test methods to evaluate and benchmark battery modeling software packages. Two different lithium‐ion battery (LIB) parameter sets enable complete assessment of the software packages in terms of accuracy, validity, and solver sensitivity. The careful selection of actual operating condition simulations and independent evaluation tests serves as a benchmark for LIB electrochemical simulation packages and can help users of these packages develop their analyses with confidence in the validity of their results. It is hoped that this paper will serve as a reference for new and established researchers and simulation engineers in LIB simulation to gain knowledge about the capabilities of existing lithium‐ion simulation packages.