Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study

Gautschi, Joel

Publikationstyp:	Konferenz: Sonstiges
Art der Begutachtung:	Peer review (Abstract)
Titel:	Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study
Autor/-in:	Gautschi, Joel
et. al:	No
Angaben zur Konferenz:	ISPCAN Québec City Congress, online, 28-30 March 2022
Erscheinungsdatum:	30-Mär-2022
Sprache:	Englisch
Schlagwörter:	Decision-making; Child protection; Variability; Noise
Fachgebiet (DDC):	362.7: Jugendhilfe
Zusammenfassung:	Background: When assessing child protection cases, information and expert knowledge are limited and sometimes ambiguous or contradictory. Moreover, decisions must be made in the context of conflicting goals of the actors involved, competing rationalities, and institutional logics. Therefore, child protection professionals are assumed to have some discretion in their assessment and decision-making. However, studies reporting statistical measures of agreement between professionals considering multiple case scenarios are still scarce. This paper presents findings on the agreement between professionals in child protection assessments in Switzerland, exploratory findings on what might explain (dis-)agreement between them and discusses approaches to how practitioners might deal with it considering the findings presented. Methods: The study used a multifactorial, experimental vignette design (factorial survey approach, cf. Rossi & Anderson, 1982). The vignettes described situations of suspected neglect of a toddler of a single mother and consisted of seven experimentally varied case factors. Out of all possible vignettes, 54 vignettes were selected using a fractional factorial sampling procedure (Kuhfeld, 2010), and split into 18 decks of 3 vignettes each. Vignette decks were randomly assigned to participants and presented in randomized order. For each vignette, respondents determined a) risk assessment (DV1), b) the likelihood of recommending an out-of-home-placement (DV2). Data were collected in an online survey of professionals, responsible for child protection assessments, in German-speaking Switzerland. 543 professionals (response rate: 63%) from 159 organizations participated, and rated 1625 vignettes. Each of the 54 vignettes were rated by 24 to 37 respondents. Two-way random effects models, accounting for case-level and respondent-level variance were used to calculate intraclass correlation coefficients ICC (2,1) as a measurement for absolute agreement among respondents. For exploratory analysis of possible determinants of disagreement, bivariate descriptive analyses were used. Results: The intraclass correlation coefficient for the respondents’ risk assessment (DV1) was ICC (2,1) = .32 (95% CI [.26, .40], F(53, 26394) = 476.30, p < .001). For the out-of-home placement recommendation the intraclass correlation coefficient was ICC (2,1) = .24 (95% CI [.19, .31], F(53, 26394) = 306.30, p < .001). However, agreement between respondents varied by vignettes. Exploratory analyses showed that the lowest standard deviations for DV1 and DV2 occurred in vignettes with very consistent case factor levels. Moreover, more severe case factor levels tended to be related to higher standard deviations for risk assessment (DV1) and to lower standard deviations for out-of-home placement recommendations (DV2). Currently, other possible determinants of (dis-)agreement are being investigated in exploratory analyses. Conclusions: Compared to standards for clinical testing the ICC indicated poor agreement between professionals (ICC < .40, Cicchetti, 1994; ICC < .50, Portney, 2020). Reasons for (dis-)agreement between professionals, and the effect of different types of assessment tools should be further investigated. Recommendations: 1) Perspective of multiple professionals should be considered, while avoiding biases of group thinking. 2) In cases with high disagreement between professionals, giving additional weight to the perspectives of the family should be considered. 3) Divergent opinions and plausible bounds of uncertainty should be made transparent to decision-makers (e. g., courts).
URI:	https://digitalcollection.zhaw.ch/handle/11475/27354
Volltext Version:	Publizierte Version
Lizenz (gemäss Verlagsvertrag):	Lizenz gemäss Verlagsvertrag
Departement:	Soziale Arbeit
Organisationseinheit:	Institut für Kindheit, Jugend und Familie (IKJF)
Enthalten in den Sammlungen:	Publikationen Soziale Arbeit

Dateien zu dieser Ressource:

Es gibt keine Dateien zu dieser Ressource.

Zur Langanzeige

Gautschi, J. (2022, March 30). Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study. ISPCAN Québec City Congress, Online, 28-30 March 2022.

Gautschi, J. (2022) ‘Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study’, in ISPCAN Québec City Congress, online, 28-30 March 2022.

J. Gautschi, “Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study,” in ISPCAN Québec City Congress, online, 28-30 March 2022, Mar. 2022.

GAUTSCHI, Joel, 2022. Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study. In: ISPCAN Québec City Congress, online, 28-30 March 2022. Conference presentation. 30 März 2022

Gautschi, Joel. 2022. “Disagreement among Professionals in Judgements About Risk and the Need for Out-of-Home Placement : Results on the Extent and Possible Determinants of Disagreement in a Swiss Vignette Study.” Conference presentation. In ISPCAN Québec City Congress, Online, 28-30 March 2022.

Gautschi, Joel. “Disagreement among Professionals in Judgements About Risk and the Need for Out-of-Home Placement : Results on the Extent and Possible Determinants of Disagreement in a Swiss Vignette Study.” ISPCAN Québec City Congress, Online, 28-30 March 2022, 2022.

Alle Ressourcen in diesem Repository sind urheberrechtlich geschützt, soweit nicht anderweitig angezeigt.