|Publication type:||Conference other|
|Type of review:||Peer review (abstract)|
|Title:||Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study|
|Conference details:||ISPCAN Québec City Congress, online, 28-30 March 2022|
|Subjects:||Decision-making; Child protection; Variability; Noise|
|Subject (DDC):||362.7: Youth services|
|Abstract:||Background: When assessing child protection cases, information and expert knowledge are limited and sometimes ambiguous or contradictory. Moreover, decisions must be made in the context of conflicting goals of the actors involved, competing rationalities, and institutional logics. Therefore, child protection professionals are assumed to have some discretion in their assessment and decision-making. However, studies reporting statistical measures of agreement between professionals considering multiple case scenarios are still scarce. This paper presents findings on the agreement between professionals in child protection assessments in Switzerland, exploratory findings on what might explain (dis-)agreement between them and discusses approaches to how practitioners might deal with it considering the findings presented. Methods: The study used a multifactorial, experimental vignette design (factorial survey approach, cf. Rossi & Anderson, 1982). The vignettes described situations of suspected neglect of a toddler of a single mother and consisted of seven experimentally varied case factors. Out of all possible vignettes, 54 vignettes were selected using a fractional factorial sampling procedure (Kuhfeld, 2010), and split into 18 decks of 3 vignettes each. Vignette decks were randomly assigned to participants and presented in randomized order. For each vignette, respondents determined a) risk assessment (DV1), b) the likelihood of recommending an out-of-home-placement (DV2). Data were collected in an online survey of professionals, responsible for child protection assessments, in German-speaking Switzerland. 543 professionals (response rate: 63%) from 159 organizations participated, and rated 1625 vignettes. Each of the 54 vignettes were rated by 24 to 37 respondents. Two-way random effects models, accounting for case-level and respondent-level variance were used to calculate intraclass correlation coefficients ICC (2,1) as a measurement for absolute agreement among respondents. For exploratory analysis of possible determinants of disagreement, bivariate descriptive analyses were used. Results: The intraclass correlation coefficient for the respondents’ risk assessment (DV1) was ICC (2,1) = .32 (95% CI [.26, .40], F(53, 26394) = 476.30, p < .001). For the out-of-home placement recommendation the intraclass correlation coefficient was ICC (2,1) = .24 (95% CI [.19, .31], F(53, 26394) = 306.30, p < .001). However, agreement between respondents varied by vignettes. Exploratory analyses showed that the lowest standard deviations for DV1 and DV2 occurred in vignettes with very consistent case factor levels. Moreover, more severe case factor levels tended to be related to higher standard deviations for risk assessment (DV1) and to lower standard deviations for out-of-home placement recommendations (DV2). Currently, other possible determinants of (dis-)agreement are being investigated in exploratory analyses. Conclusions: Compared to standards for clinical testing the ICC indicated poor agreement between professionals (ICC < .40, Cicchetti, 1994; ICC < .50, Portney, 2020). Reasons for (dis-)agreement between professionals, and the effect of different types of assessment tools should be further investigated. Recommendations: 1) Perspective of multiple professionals should be considered, while avoiding biases of group thinking. 2) In cases with high disagreement between professionals, giving additional weight to the perspectives of the family should be considered. 3) Divergent opinions and plausible bounds of uncertainty should be made transparent to decision-makers (e. g., courts).|
|Fulltext version:||Published version|
|License (according to publishing contract):||Licence according to publishing contract|
|Organisational Unit:||Institute of Childhood, Youth and Family (IKJF)|
|Appears in collections:||Publikationen Soziale Arbeit|
Files in This Item:
There are no files associated with this item.
Show full item record
Gautschi, J. (2022, March 30). Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study. ISPCAN Québec City Congress, Online, 28-30 March 2022.
Gautschi, J. (2022) ‘Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study’, in ISPCAN Québec City Congress, online, 28-30 March 2022.
J. Gautschi, “Disagreement among professionals in judgements about risk and the need for out-of-home placement : results on the extent and possible determinants of disagreement in a Swiss vignette study,” in ISPCAN Québec City Congress, online, 28-30 March 2022, Mar. 2022.
Gautschi, Joel. “Disagreement among Professionals in Judgements About Risk and the Need for Out-of-Home Placement : Results on the Extent and Possible Determinants of Disagreement in a Swiss Vignette Study.” ISPCAN Québec City Congress, Online, 28-30 March 2022, 2022.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.