Via Research Recognition Day 2024 VCOM-Carolinas

Educational Reports

THE ROLE OF ARTIFICIAL INTELLIGENCE TO DECREASE RETURN VISITS FOR VAGINITIS: A PILOT STUDY Kathryn Vess, OMS-IV, Ashley Harvey, OMS-IV, Sonia Patel, OMS-IV, Fiona Bradshaw, OMS-III, Alyssa Bevevino, OMS-III, Lisa Carroll, MD, David Redden, PhD Via College of Osteopathic Medicine – Carolinas Campus, Spartanburg, SC Abstract Results Conclusion

Table 1: Clinic Results of physician read and AI algorithm diagnoses of vaginitis and repeat visits within 6 months at Spartanburg Regional Family Medicine Clinic

Methods ,ĞĂůƚŚĐĂƌĞ ƐƉĞŶĚŝŶŐ ŝŶ ƚŚĞ h^ ƚŽƚĂůĞĚ Ψϰ͕ ϮϮϱ͘ ϭ ďŝůůŝŽŶ ƐƉĞŶƚ ŝŶ ϮϬϮϭ͘ ϭ ^ŝŐŶŝĨŝĐĂŶƚ ƌĞƐĞĂƌĐŚ ŝƐ ĐƵƌƌĞŶƚůLJ ĂŝŵĞĚ Ăƚ ƌĞĚƵĐŝŶŐ ŚĞĂůƚŚĐĂƌĞ ĐŽƐƚƐ͘ Ϯ ^ƚƵĚŝĞƐ ƐŚŽǁ ƚŚĂƚ ǁĂƐƚĞĨƵů ƐƉĞŶĚŝŶŐ ŝŶ ŚĞĂůƚŚĐĂƌĞ ŝƐ ůĂƌŐĞůLJ ƐƉĞŶƚ ŽŶ ůŽǁͲǀĂůƵĞ ŚĞĂůƚŚĐĂƌĞ ĂŶĚ ƌĞƉĞƚŝƚŝǀĞ ĚŝĂŐŶŽƐƚŝĐƐ͘ ϮͲϱ EĞǁ ƚĞĐŚŶŽůŽŐLJ ĐĂŶ ĞůŝŵŝŶĂƚĞ ůŽǁͲǀĂůƵĞ ĂŶĚ ǁĂƐƚĞĨƵů ŚĞĂůƚŚĐĂƌĞ͘ ϲ ƌƚŝĨŝĐŝĂů ŝŶƚĞůůŝŐĞŶĐĞ ; /Ϳ ĐĂŶ ďĞ ƵƐĞĚ ƚŽ ŝŵƉƌŽǀĞ ĐƵƌƌĞŶƚ ƉƌĂĐƚŝĐĞƐ ĨŽƌ ƚŚĞ ĚŝĂŐŶŽƐŝƐ ŽĨ ǀĂŐŝŶŝƚŝƐ͘ ϲͲϴ dŚĞ ƐƚĂŶĚĂƌĚ ŽĨ ĐĂƌĞ ĨŽƌ ǀĂŐŝŶŝƚŝƐ ĚŝĂŐŶŽƐŝƐ ƵƐĞƐ ƚŚĞ ŵƐĞů ĐƌŝƚĞƌŝĂ ǁŚĞƌĞŝŶ Ă ƉŚLJƐŝĐŝĂŶ ĞǀĂůƵĂƚĞƐ ƚŚĞ ǀĂŐŝŶĂů ƐĂŵƉůĞ ĨŽƌ Ɖ,͕ ƐĐĞŶƚ͕ ĚŝƐĐŚĂƌŐĞ ĐŚĂƌĂĐƚĞƌŝƐƚŝĐ͕ ĂŶĚ ƉƌĞƐĞŶĐĞ ŽĨ ƚƌŝĐŚŽŵŽŶĂĚƐ͕ LJĞĂƐƚ͕ Žƌ ĐůƵĞ ĐĞůůƐ ŽŶ ŵŝĐƌŽƐĐŽƉLJ͘ ϵ ,ƵŵĂŶ ĞƌƌŽƌ ĐĂŶ ůĞĂĚ ƚŽ ŵŝƐĚŝĂŐŶŽƐŝƐ ŽĨ ǀĂŐŝŶŝƚŝƐ ŝŶ ƚŚĞ ĐůŝŶŝĐ͘ /Ĩ ǀĂŐŝŶŝƚŝƐ ŐŽĞƐ ƵŶĚĞƌĚŝĂŐŶŽƐĞĚ ŝŶ ƚŚĞ ĐůŝŶŝĐ͕ ŝƚ ĐĂŶ ƉĞƌƐŝƐƚ ůĞĂĚŝŶŐ ƚŽ ƌĞƚƵƌŶ ŽĨĨŝĐĞ ǀŝƐŝƚƐ ǁŝƚŚ ƌĞƉĞĂƚƐ ƚĞƐƚŝŶŐ ĂŶĚ ĚŝĂŐŶŽƐƚŝĐƐ ǁŚŝĐŚ ŝŶĐƌĞĂƐĞƐ ǁĂƐƚĞĨƵů ŚĞĂůƚŚĐĂƌĞ ƐƉĞŶĚŝŶŐ͘ ϵ dŚĞ ŽďũĞĐƚŝǀĞ ŽĨ ƚŚŝƐ ƐƚƵĚLJ ǁĂƐ ƚŽ ĚĞƚĞƌŵŝŶĞ ƚŚĞ ĞĨĨŝĐĂĐLJ ŽĨ / ǀĞƌƐƵƐ ƉŚLJƐŝĐŝĂŶ ĚŝĂŐŶŽƐŝƐ ŽĨ ǀĂŐŝŶŝƚŝƐ ĂŶĚ ƚŚĞ ƌŽůĞ ŽĨ / ŝŶ ƌĞĚƵĐŝŶŐ ƌĞƉĞĂƚ ǀŝƐŝƚƐ ǁŝƚŚ ƐŝŵŝůĂƌ ƐLJŵƉƚŽŵƐ͘ tŽŵĞŶ ǁŚŽ ƉƌĞƐĞŶƚĞĚ ƚŽ ƚŚĞ ^ƉĂƌƚĂŶďƵƌŐ ZĞŐŝŽŶĂů &ĂŵŝůLJ DĞĚŝĐŝŶĞ ůŝŶŝĐ Žƌ ZĞŐŝŽŶĂů KďƐƚĞƚƌŝĐƐ ĂŶĚ 'LJŶĞĐŽůŽŐLJ ůŝŶŝĐ ŝŶ &ĞďƌƵĂƌLJ ϮϬϮϯ ǁĞƌĞ ŽĨĨĞƌĞĚ ĞŶƌŽůůŵĞŶƚ ŝŶ ƚŚĞ ƐƚƵĚLJ ĂŶĚ ŝŶĨŽƌŵĞĚ ĐŽŶƐĞŶƚ ǁĂƐ ŽďƚĂŝŶĞĚ ƉĞƌ s KD /Z ϭϲϮϲϳϵϭͲϮϰ͘ KŶĐĞ ĞŶƌŽůůĞĚ͕ ƉĂƚŝĞŶƚƐ ŚĂĚ ƉĞůǀŝĐ Žƌ ǀĂŐŝŶĂů ĞdžĂŵƐ ĂŶĚ ƚŚƌĞĞ ƐǁĂďƐ ǁĞƌĞ ĐŽůůĞĐƚĞĚ ŽĨ ǀĂŐŝŶĂů ƐĞĐƌĞƚŝŽŶƐ͘ dŚĞ ǀĂŐŝŶĂů ĚŝƐĐŚĂƌŐĞ ĐŽůůĞĐƚĞĚ ǁĂƐ ƚĞƐƚĞĚ ĨŽƌ Ɖ,͕ ƐĐĞŶƚ͕ ĚĞƐĐƌŝďĞĚ ǀŝƐƵĂůůLJ͕ ĂŶĚ ĚŝƌĞĐƚůLJ ŽďƐĞƌǀĞĚ ƵŶĚĞƌ ƚŚĞ ŵŝĐƌŽƐĐŽƉĞ͘ dŚĞ ƐǁĂďƐ ĂŶĚ ƐůŝĚĞƐ ǁĞƌĞ ĂĨĨŝdžĞĚ ĂŶĚ ƐĞŶƚ ĨŽƌ ĂŶĂůLJƐŝƐ ƚŽ ĂnjĂ ,ĞĂůƚŚ͘ dŚĞ ƐĂŵƉůĞƐ ǁĞƌĞ ĂŶĂůLJnjĞĚ ƵƐŝŶŐ ĂŶ / ĂůŐŽƌŝƚŚŵ ĂŶĚ ǀĞƌŝĨŝĞĚ ďLJ ƉŚLJƐŝĐŝĂŶƐ Žƌ ŵĞĚŝĐĂů ƐƚƵĚĞŶƚƐ͘ ĨƚĞƌ Ă ƉĞƌŝŽĚ ŽĨ ϲ ŵŽŶƚŚƐ͕ ĞĂĐŚ ƉĂƚŝĞŶƚ ŚĂĚ Ă ďƌŝĞĨ ĐŚĂƌƚ ƌĞǀŝĞǁ ƚŽ ĂƐƐĞƐƐ ĨŽƌ ƌĞƚƵƌŶ ǀŝƐŝƚƐ ǁŝƚŚ ǀĂŐŝŶŝƚŝƐ͘ ŐƌĞĞŵĞŶƚ ǁĂƐ ĂŶĂůLJnjĞĚ ďĞƚǁĞĞŶ / ĂůŐŽƌŝƚŚŵ ĂŶĚ ƉŚLJƐŝĐŝĂŶƐ ƌĞŐĂƌĚŝŶŐ ƚŚĞ ĐůĂƐƐŝĨŝĐĂƚŝŽŶ ŽĨ ϵ ŝŶĚĞƉĞŶĚĞŶƚ ƐůŝĚĞƐ ŝŶ the diagnosis of vaginitis. Cohen’s Kappa was used to measure ĂŐƌĞĞŵĞŶƚ͘

References The purpose of this study was to determine if the use of AI in place of the standard of care for diagnosis of vaginitis in the outpatient clinic could decrease misdiagnosis and subsequent incorrect treatment of vaginitis. The study measured misdiagnosis by evaluating return visits with the same complaint in a six-month period. Overall, physicians and AI had an agreement of 78%. Of the nine samples taken, two had discordance between the physician and AI diagnoses with return visits in the following six months. Return visits for the same problem with repeat diagnostics, treatment, and evaluation contribute to wasteful spending in healthcare as well as loss of time and income to patients. This study represents only a pilot study and further patients will need to be enrolled to determine the validity of the study. ϭ͘ ŵĞƌŝĐĂŶ DĞĚŝĐĂů ƐƐŽĐŝĂƚŝŽŶ͘ ;ϮϬϮϯ͕ DĂƌĐŚ ϮϬͿ͘ dƌĞŶĚƐ ŝŶ ,ĞĂůƚŚĐĂƌĞ ^ƉĞŶĚŝŶŐ͘ ŚƚƚƉƐ͗ͬͬ ǁǁǁ͘ ĂŵĂͲ ĂƐƐŶ͘ ŽƌŐͬ ĂďŽƵƚͬ ƌĞƐĞĂƌĐŚͬ ƚƌĞŶĚƐͲŚĞĂůƚŚͲĐĂƌĞͲƐƉĞŶĚŝŶŐ Ϯ͘ ĞŶƚůĞLJ͕ d͘ '͕͘ ĨĨƌŽƐ͕ Z͘ D͕͘ Θ WĂůĂƌ <͘ Ğƚ Ăů ;ϮϬϬϴͿ͘ tĂƐƚĞ ŝŶ ƚŚĞ h͘ ^͘ ,ĞĂůƚŚĐĂƌĞ ^LJƐƚĞŵ͗ ĐŽŶĐĞƉƚƵĂů ĨƌĂŵĞǁŽƌŬ͘ DŝůďĂŶŬ YƵĂƌƚĞƌůLJ͕ ϴϲ;ϰͿ͗ ϲϮϵͲϲϱϵ͘ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2690367/ ϯ͘ KƌŐĂŶŝnjĂƚŝŽŶ ĨŽƌ ĐŽŶŽŵŝĐ ŽͲŽƉĞƌĂƚŝŽŶ ĂŶĚ ĞǀĞůŽƉŵĞŶƚ͘ ;ϮϬϭϳͿ͘ dĂĐŬůŝŶŐ tĂƐƚĞĨƵů ^ƉĞŶĚŝŶŐ ŽŶ ,ĞĂůƚŚ͘ K WƵďůŝƐŚŝŶŐ͘ https://www.oecd.org/els/health-systems/Tackling-Wasteful-Spending-on-Health-Highlights revised.pdf ϰ͘ Herbert, L.F. (2016). Cutting the Cost of Healthcare: the Physician’s Role. dŚĞ dĞdžĂƐ ,ĞĂƌƚ /ŶƐƚŝƚƵƚĞ ,ŽƵƌŶĂů͕ ϰϯ;ϭͿ͗ ϰͲϲ͘ https://meridian.allenpress.com/thij/article/43/1/4/85234/Cutting-the-Cost-of-Health-Care-The-Physician-s ϱ͘ ŽůůĂ͕ ͘ ,͕͘ DŽƌĚĞŶ͕ E͘ ͕͘ ^ĞƋŝƐƚ͕ d͘ ͕͘ Ğƚ Ăů͘ ;ϮϬϭϱͿ͘ ŚŽŽƐŝŶŐ tŝƐĞůLJ͗ WƌĞǀĂůĞŶĐĞ ĂŶĚ ŽƌƌĞůĂƚĞƐ ŽĨ >ŽǁͲsĂůƵĞ ,ĞĂůƚŚĐĂƌĞ ^ĞƌǀŝĐĞƐ ŝŶ ƚŚĞ hŶŝƚĞĚ ^ƚĂƚĞƐ͘ :ŽƵƌŶĂů ŽĨ 'ĞŶĞƌĂů /ŶƚĞƌŶĂů DĞĚŝĐŝŶĞ͕ ϯϬ͗ ϮϮϭͲϮϮϴ͘ https://link.springer.com/article/10.1007/s11606-014-3070-z ϲ͘ ,ĂŽ͕ Z͕͘ >ŝŶ͕ >͕ Θ ŚĂŶŐ͕ :͘ Ğƚ Ăů͘ ;ϮϬϮϮͿ͘ ĂƚĂͲ ĨĨŝĐŝĞŶƚ &ƌĂŵĞǁŽƌŬ ĨŽƌ ƚŚĞ /ĚĞŶƚŝĨŝĐĂƚŝŽŶ ŽĨ sĂŐŝŶŝƚŝƐ ĂƐĞĚ ŽŶ ĞĞƉ >ĞĂƌŶŝŶŐ͘ :ŽƵƌŶĂů ŽĨ ,ĞĂůƚŚĐĂƌĞ ŶŐŝŶĞĞƌŝŶŐ͕ ϮϬϮϮ͘ https://www.hindawi.com/journals/jhe/2022/1929371/ 7. Du, X. Liu, L. & Wang, X. et al. (2017). Trichomonas Detection in Leucorrhea Based on VIBE Method. Computational and Mathematical Methods in Medicine, 2019. https://www.hindawi.com/journals/cmmm/2019/5856970/ 8. Song, Y., He, L., & Zhou, F. et al. (2017). Segmentation, Splitting, and Classification of Overlapping Bacteria in Microscope Images for Automatic Bacterial Vaginosis Diagnosis. IEEE Journal of Biomedical and Health Informatics, 21(4): 1095-1104. https://ieeexplore.ieee.org/abstract/document/7523288 ϵ͘ ŽůŽŶŶĂ͕ ͘ Θ ^ƚĞĞůŵĂŶ͕ D͘ ;ϮϬϮϯͿ͘ ŵƐĞů ƌŝƚĞƌŝĂ͘ EĂƚŝŽŶĂů ĞŶƚĞƌ ĨŽƌ ŝŽƚĞĐŚŶŽůŽŐLJ /ŶĨŽƌŵĂƚŝŽŶ͘ ŚƚƚƉƐ͗ͬͬ ǁǁǁ͘ ŶĐďŝ͘ Ŷůŵ͘ ŶŝŚ͘ ŐŽǀͬ ŬƐͬ E <ϱϰϮϯϭϵͬ

Table 2: Results of physician read and AI algorithm diagnoses of vaginitis and repeat visits within 6 months at Spartanburg Regional Obstetrics and Gynecology Clinic

dŚĞ / ĂůŐŽƌŝƚŚŵ ĐůĂƐƐŝĨŝĞĚ Ϯ ƐůŝĚĞƐ ĂƐ ƉŽƐŝƚŝǀĞ ĨŽƌ ďĂĐƚĞƌŝĂů ǀĂŐŝŶŝƚŝƐ͕ ϳ ƐůŝĚĞƐ ĂƐ ŶĞŐĂƚŝǀĞ͘ dŚĞ ƉŚLJƐŝĐŝĂŶ ĂůƐŽ ĐůĂƐƐŝĨŝĞĚ Ϯ ƐůŝĚĞƐ ĂƐ ƉŽƐŝƚŝǀĞ ĨŽƌ ĐĂŶĚŝĚŝĂƐŝƐ͕ ϳ ƐůŝĚĞƐ ĂƐ ŶĞŐĂƚŝǀĞ͘ dŚĞ ŽǀĞƌĂůů ƉĞƌĐĞŶƚ ĂŐƌĞĞŵĞŶƚ ǁĂƐ ŽŶůLJ ϳϴй ǁŝƚŚ ƚŚĞ ƉŚLJƐŝĐŝĂŶ ĂŶĚ ĂůŐŽƌŝƚŚŵ ĂŐƌĞĞŝŶŐ ŽŶ ϲ ŶĞŐĂƚŝǀĞ ƐůŝĚĞƐ ĂŶĚ ϭ ƉŽƐŝƚŝǀĞ ƐůŝĚĞ͘ ŝƐĂŐƌĞĞŵĞŶƚ ǁĂƐ ŽďƐĞƌǀĞĚ ŽŶ Ϯ ƐůŝĚĞƐ͕ ϭ ƐůŝĚĞ ǁŚĞƌĞ ƚŚĞ / ĂůŐŽƌŝƚŚŵ ĚĞĐůĂƌĞĚ ƚŚĞ ƐůŝĚĞ ƉŽƐŝƚŝǀĞ ĨŽƌ ďĂĐƚĞƌŝĂů ǀĂŐŝŶŝƚŝƐ ǁŚŝůĞ ƚŚĞ ƉŚLJƐŝĐŝĂŶ ŝŶĚŝĐĂƚĞĚ ŶĞŐĂƚŝǀĞ͕ ĂŶĚ ϭ ƐůŝĚĞ ǁŚĞƌĞ ƚŚĞ / ĂůŐŽƌŝƚŚŵ ĚĞĐůĂƌĞĚ ƚŚĞ ƐůŝĚĞ ŶĞŐĂƚŝǀĞ ǁŚŝůĞ ƚŚĞ ƉŚLJƐŝĐŝĂŶ ŝŶĚŝĐĂƚĞĚ ƉŽƐŝƚŝǀĞ ĨŽƌ ĐĂŶĚŝĚŝĂƐŝƐ͘ dŽ ŵĞĂƐƵƌĞ agreement, we used Cohen’s Kappa which was estimated at Ϭ͘ ϯϲ͕ Ă ŵĞĚŝƵŵ ůĞǀĞů ŽĨ ĂŐƌĞĞŵĞŶƚ͘ /Ŷ ƚǁŽ ŝŶƐƚĂŶĐĞƐ ǁŚĞƌĞ ƚŚĞƌĞ ǁĞƌĞ ƉŚLJƐŝĐŝĂŶ ĂŶĚ / ĚŝƐĂŐƌĞĞŵĞŶƚ͕ ƉĂƚŝĞŶƚƐ ƌĞƚƵƌŶĞĚ ƚŽ ƚŚĞ ĐůŝŶŝĐ ǁŝƚŚŝŶ ϲ ŵŽŶƚŚƐ ǁŝƚŚ ƐLJŵƉƚŽŵƐ ŽĨ ǀĂŐŝŶŝƚŝƐ ĐŽŶƐŝƐƚĞŶƚ ǁŝƚŚ ƚŚĞ ŝŶŝƚŝĂů / ĚŝĂŐŶŽƐŝƐ͘

tĞ ǁŽƵůĚ ůŝŬĞ ƚŽ ĂĐŬŶŽǁůĞĚŐĞ ƚŚĞ ^ƉĂƌƚĂŶďƵƌŐ ZĞŐŝŽŶĂů &ĂŵŝůLJ DĞĚŝĐŝŶĞ ůŝŶŝĐ ĂŶĚ ^ƉĂƌƚĂŶďƵƌŐ ZĞŐŝŽŶĂů KďƐƚĞƚƌŝĐƐ ĂŶĚ 'LJŶĞĐŽůŽŐLJ ůŝŶŝĐ ĨŽƌ ĂůůŽǁŝŶŐ ƵƐ ƚŽ ĐŽůůĞĐƚ ƐĂŵƉůĞƐ͘ tĞ ǁŽƵůĚ ĂůƐŽ ůŝŬĞ ƚŽ ĂĐŬŶŽǁůĞĚŐĞ ĂƐĂ ,ĞĂůƚŚ ĨŽƌ ƚŚĞŝƌ / ƚĞĐŚŶŽůŽŐLJ͘

124

2024 Research Recognition Day