Rights statement: This article is (c) Emerald Group Publishing and permission has been granted for this version to appear here. Emerald does not grant permission for this article to be further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing Limited.
Accepted author manuscript, 390 KB, PDF document
Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License
Final published version
Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Assessing the reliability and validity of an outcomes star
AU - Sweet, Daryl
AU - Winter, Karen
AU - Neeson, Laura
AU - Connolly, Paul
N1 - This article is (c) Emerald Group Publishing and permission has been granted for this version to appear here. Emerald does not grant permission for this article to be further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing Limited.
PY - 2020/8/10
Y1 - 2020/8/10
N2 - Purpose: This paper aims to assess the reliability, validity and use of the Family Star Plus, one of several Outcomes Stars increasingly used as part of outcomes-based accountability approaches in the delivery of family support services. The Family Star Plus measures progress towards effective parenting but a lack of evidence exists on its psychometric properties and suitability for use as an outcomes tool. Design/methodology/approach: Based on data from 1,255 families receiving a pilot support service, Cronbach’s alpha was used to assess the internal reliability of the 10-item scale, while principal component analysis (PCA) examined the number of constructs in the tool. Using matched data from evaluation of 80 families, correlations between the Family Star Plus and psychometrically validated tools were used to assess concurrent validity. Findings from a process evaluation explore practical issues around use of the tool. Findings: Cronbach’s alpha indicated sufficient internal reliability of the Family Star Plus; however, the PCA raised questions concerning the internal validity the Star. Correlations between the Star and validated tools were not strong enough to support concurrent validity of the Star. Process evaluation findings highlight inconsistencies in Family Star Plus data capture, which may explain these differences. Practical implications: Further work is required before the Family Star Plus can be considered for use as an outcome measure. Originality/value: To the best of authors’ knowledge, this is the first peer-reviewed analysis of the psychometric qualities of the Family Star Plus.
AB - Purpose: This paper aims to assess the reliability, validity and use of the Family Star Plus, one of several Outcomes Stars increasingly used as part of outcomes-based accountability approaches in the delivery of family support services. The Family Star Plus measures progress towards effective parenting but a lack of evidence exists on its psychometric properties and suitability for use as an outcomes tool. Design/methodology/approach: Based on data from 1,255 families receiving a pilot support service, Cronbach’s alpha was used to assess the internal reliability of the 10-item scale, while principal component analysis (PCA) examined the number of constructs in the tool. Using matched data from evaluation of 80 families, correlations between the Family Star Plus and psychometrically validated tools were used to assess concurrent validity. Findings from a process evaluation explore practical issues around use of the tool. Findings: Cronbach’s alpha indicated sufficient internal reliability of the Family Star Plus; however, the PCA raised questions concerning the internal validity the Star. Correlations between the Star and validated tools were not strong enough to support concurrent validity of the Star. Process evaluation findings highlight inconsistencies in Family Star Plus data capture, which may explain these differences. Practical implications: Further work is required before the Family Star Plus can be considered for use as an outcome measure. Originality/value: To the best of authors’ knowledge, this is the first peer-reviewed analysis of the psychometric qualities of the Family Star Plus.
KW - Evaluation
KW - Family support
KW - Outcomes stars
KW - Psychometrics
KW - Reliability
KW - Validity
KW - article
KW - child parent relation
KW - concurrent validity
KW - Cronbach alpha coefficient
KW - human
KW - internal validity
KW - outcome assessment
KW - principal component analysis
KW - psychometry
U2 - 10.1108/JCS-03-2020-0009
DO - 10.1108/JCS-03-2020-0009
M3 - Journal article
JO - Journal of Children's Services
JF - Journal of Children's Services
SN - 1746-6660
ER -