Whilst participatory practice is increasingly adopted in end user studies, there has been far less use of a participatory approach when designing lower down the software stack. As a result, end users are often presented with security controls over which they have no control but for which they retain the responsibility. Conversely, hardware and software engineers struggle to innovate new security control designs that are resilient to new and emerging threats. In a study utilising ethnographic research and stakeholder interviews, we show that there is a siloing of communities of practice between hardware security engineers, software engineers and coders, manufacturers in the technology supply chain and end users. Our findings indicate that this siloing and a lack of participatory practice impedes the development of a more cohesive digital security design that integrates security through the stack from the hardware layer upwards to the OS and application layers. These barriers make difficult the negotiation between what is possible lower down the stack with what is needed and wanted higher up the stack. Our findings suggest that a more holistic and comprehensive participatory design approach is required to negotiate a digital security by design paradigm that more evenly distributes power over and responsibility for security controls throughout the stack. Working with the HCI literature on co-production in design, this paper will suggest that a pathway for breaking through this impasse is to utilise objects in the design process of the hardware secure instruction set architecture as a feedback mechanism to incorporate other sets of designers and users in the design process to create a more workable stack.