TY - JOUR

T1 - Using bench protocol platforms to improve compliance with systematic review guidance documents and reporting checklists

T2 - Proof of concept

AU - Whaley, Paul

AU - Wattam, Stephen

AU - Scott, Anna Mae

AU - Vidler, John

PY - 2023/12/31

Y1 - 2023/12/31

N2 - ContextA significant number of guidance documents and reporting checklists have been published to support researchers in planning, doing, and writing up scientifically rigorous systematic reviews (SRs). However, compliance of researchers with SR guidance and reporting checklists remains a significant challenge, with the majority of published SRs lacking in one or more aspects of the rigour of methods and transparency of reporting.ObjectiveTo explore how bench protocol development platforms might be repurposed for improving compliance of SRs with conduct guidance and reporting checklists.System designWe developed a proof-of-concept technology stack based around a general-purpose, guidance- and checklist-compliant SR protocol that was built in protocols.io. We used the protocols.io platform to create an integrated research planning and data collection process for planning guidance-compliant SRs. We used our own custom code and the mustache templating language to automatically create checklist-compliant first-draft SR protocol documents in Microsoft WordDiscussionCreating the operational process for SR protocol planning and the technology stack for automated documentation allowed us to develop our theoretical understanding of how such a system may improve compliance with research conduct and reporting standards. This includes the potential value of algorithmic rather than heuristic approaches to conducting and reporting research studies, positioning of labelled data rather than a study manuscript as the primary product of the research process, and viewing the process of developing research standards as being analogous to the development of open software. Our study also allowed us to identify a number of technological issues that will need to be addressed to enable further development and testing of our proposed approach. These include limitations in templating language, especially when working in Microsoft Word, and the need for more data labelling and export formats from protocols.io.

AB - ContextA significant number of guidance documents and reporting checklists have been published to support researchers in planning, doing, and writing up scientifically rigorous systematic reviews (SRs). However, compliance of researchers with SR guidance and reporting checklists remains a significant challenge, with the majority of published SRs lacking in one or more aspects of the rigour of methods and transparency of reporting.ObjectiveTo explore how bench protocol development platforms might be repurposed for improving compliance of SRs with conduct guidance and reporting checklists.System designWe developed a proof-of-concept technology stack based around a general-purpose, guidance- and checklist-compliant SR protocol that was built in protocols.io. We used the protocols.io platform to create an integrated research planning and data collection process for planning guidance-compliant SRs. We used our own custom code and the mustache templating language to automatically create checklist-compliant first-draft SR protocol documents in Microsoft WordDiscussionCreating the operational process for SR protocol planning and the technology stack for automated documentation allowed us to develop our theoretical understanding of how such a system may improve compliance with research conduct and reporting standards. This includes the potential value of algorithmic rather than heuristic approaches to conducting and reporting research studies, positioning of labelled data rather than a study manuscript as the primary product of the research process, and viewing the process of developing research standards as being analogous to the development of open software. Our study also allowed us to identify a number of technological issues that will need to be addressed to enable further development and testing of our proposed approach. These include limitations in templating language, especially when working in Microsoft Word, and the need for more data labelling and export formats from protocols.io.

KW - Automation

KW - systematic review

KW - reporting checklists

KW - research standards

U2 - 10.1080/2833373x.2023.2259938

DO - 10.1080/2833373x.2023.2259938

M3 - Journal article

VL - 1

JO - Evidence-Based Toxicology

JF - Evidence-Based Toxicology

SN - 2833-373X

IS - 1

M1 - 2259938

ER -