Real-world Inter-rater Agreement of PI-QUAL Version 2 for Prostate Magnetic Resonance Imaging Quality Assessment and Its Association with Diagnostic Accuracy

Background and objective: Magnetic resonance imaging (MRI) has been shown to improve the detection of prostate cancer, the second most diagnosed cancer among men. This study evaluates the inter-rater agreement for MRI quality using Prostate Imaging Quality (PI-QUAL) v2. It studies the association of PI-QUAL with diagnostic accuracy and the proportion of indeterminate Prostate Imaging Reporting and Data System (PI-RADS) 3 lesions with PI-QUAL scores. Methods: This multicenter cohort study included biopsy-naïve patients from the PCAVISION trial (NCT06281769) who underwent MRI for a suspicion of prostate cancer. Four radiologists independently scored PI-QUAL v2 and PI-RADS. PI-QUAL v2 scores were dichotomized as PI-QUAL 1 versus PI-QUAL ≥2. PI-RADS was dichotomized as PI-RADS ≤2 versus PI-RADS ≥3 and evaluated using percentage agreement, kappa, and Gwet's agree coefficient (AC) 1. The PI-QUAL consensus score was defined by the majority score or by a referee if no majority was reached. In cases with biopsy pathology, the association between PI-QUAL and diagnostic accuracy was assessed. Key findings and limitations: = 0.011). Conclusions and clinical implications: Strict adherence to the technical PI-QUAL criteria led to poor inter-rater agreement, while visual-based assessment yielded moderate agreement. Importantly, higher PI-QUAL scores were linked to fewer indeterminate PI-RADS 3 lesions. Patient summary: We studied how doctors agree when judging the quality of prostate magnetic resonance imaging (MRI) scan using the Prostate Imaging Quality scoring system. When they focused too strictly on technical rules, agreement was poor; when scans were judged visually, agreement improved. Better-quality MRI scans led to fewer unclear ("indeterminate") results, helping doctors make clearer biopsy decisions.