Virginia Research Day 2025

Medical Student Research Clinical

38 The Vaginal Health Assessment Test for Gynecology Patients with Vaginitis: An Expert Reader vs. Trained Reader Study

Lisa Carroll 1 ; Nicolette Adderton 2 ; Brooke Burwell 2 ; Ashley Deer 2 ; Joe DeGroot 3 ; Ashley Harvey 1 ; Sara Karami 2 , Chris Kim 2 ; John Kearney 3 ; Mathushana Logeswaran 2 ; Sara Majeed 2 ; Sylvia Mast 2 ; Kent Murphy 3 ; Brooke Nelson 2 ; Nitika Pentakota 2 ; Swathi Sambatha 2 ; Akshata Sastry 2 ; Luis Ycaza 2 ; Catherine Pelligrino 1 ; Peggy Robinson 3 ; Yongjian Yu 3 ; Jim Mahaney 2 Corresponding author: jmahaney@vt.vcom.edu

1 Edward Via College of Osteopathic Medicine - Carolinas Campus 2 Edward Via College of Osteopathic Medicine - Virginia Campus 3 Caza Health, LLC, Earlysville, VA 4 Liberty University College of Osteopathic Medicine, Lynchburg, VA Vaginal infections are one of the most presented complaints by female patients. Current approaches to diagnosing vaginitis include in-office testing with wet-mount microscopy and laboratory testing with nucleic acid amplification testing (NAAT). NAAT provides greater accuracy than traditional wet mount but takes a greater amount of time to obtain results, which may delay appropriate treatment. In contrast, the use of artificial intelligence (AI) in this field has the potential to improve accuracy and subsequently, prompt treatment. Caza Health has developed the DayZTM Vaginal Health Assessment Assay (VHA), which uses artificial intelligence (AI) coupled with immunofluorescence antibody cocktails plus automated scanning microscopy to determine the targets of interest: bacterial vaginosis (BV), candida vaginitis (CV) and trichomonas vaginosis (TV) at the time of the office visit. As part of the DayZTM development, a multi-reader study was conducted on

a subset of samples, with the objective of assessing the ability of a trained reader to interpret results of the DayZ™ VHA test as compared to ground truth (expert reader). The effectiveness of the proposed training to be used was evaluated and reader-to-reader variations were evaluated by discordant analysis. The images used were obtained using vaginal fluid samples provided by research subjects with symptomatic vaginitis recruited at three Spartanburg Medical Center clinics in Spartanburg, SC, under approved IRB protocol 2020-017. Thirteen (13) individual readers were trained one-on-one using the DayZ ™ System analysis software and the same exemplary image file and instructions. Next, the individual readers independently read between 2 and 33 image files (39 total in the set) in random order as their time allowed. Readers were asked to evaluate target species images presented by the AI system, and to determine (yes/no) if the images

were clue cells (BV), yeast pseudohyphae (CV), or trichomonads (TV). Individual reader results were saved and sent to Caza Health for analysis. The AI team at Caza evaluated the reader results as compared to ground truth (the expert reader). The results show that overall, the 13 readers agreed with the expert reader in the identification of yeast, 96%, and trichomonads, 97%, but only 80% for clue cells. However, discordance analysis showed that individual readers varied widely in their individual assignments, with no correlation with total files read. As a result of the analysis, ways to enhance training effectiveness to better reader outcomes were identified including increased one-on-one training time, clearer definitions of clue cell characteristics, how to identify clue cells versus normal epithelial cells more accurately, and recognizing/rejecting non-specific staining, autofluorescence, and cellular debris.

161 2025 Research Recognition Day