Robustness of κ‐type coefficients for clinical agreement
Amalia Vanacore, Maria Sole Pellegrino
University of Naples Federico II
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
The degree of inter‐rater agreement is usually assessed through ‐type coefficients and the extent of agreement is then characterized by comparing the value of the adopted coefficient against a benchmark scale. Through two motivating examples, it is displayed the different behavior of some ‐type coefficients due to asymmetric distribution of marginal frequencies over categories. In order to investigate the robustness of four ‐type coefficients for nominal and ordinal classifications and of an inferential benchmarking procedure that, differently from straightforward benchmarking, does not neglect the influence of the experimental conditions, an extensive Monte Carlo simulation study has been conducted. The robustness has been investigated for several scenarios, differing for sample size, rating scale dimension, number of raters, frequency distribution of rater classifications, pattern of agreement across raters. Simulation results reveal an higher paradoxical behavior of Fleiss kappa and Conger kappa with ordinal rather than nominal classifications; the coefficients robustness improves with increasing sample size and number of raters for both nominal and ordinal classifications whereas robustness improves with rating scale dimension only for nominal classifications. By identifying the scenarios (ie, minimum sample size, number of raters, rating scale dimension) with acceptable robustness, this study provides guidelines about the design of robust agreement studies.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AIReliability and Agreement in Measurement
Meta-analysis and systematic reviews · Delphi Technique in Research
参考文献 67
此处列出前 3 条
引用本文 26
按被引量排序,此处列出前 3 条