This research explores a new collaborative improvement framework called improvement matrix, where the instructions, functions and flexibility of tools are improved within three layers of engagement. This paper describes how the framework was tested in practice through a series of workshops, where engagement practitioners redesigned tools to improve their engagement practices as part of a larger action research project. This research provided a dual outcome that enabled participants to gain a tangible benefit from improved versions of tools that came out from the process as well as enabled us to develop a deep understanding of the improvement process as the research output. The findings from three case studies suggest how the framework plays out in practice, providing guidelines on how to improve tools using the improvement matrix. We found that the matrix can be used for different purposes, such as improving flexibility of tools or designing facilitation approaches.