AI-Powered Mobile App with AR Glasses for Remote Chronic Wound Management and Expert Consultation

Abstract

Purpose: Chronic wounds pose a significant healthcare challenge due to their resistance to healing. This study tackles this challenge by developing a mobile application powered by AI, aiming to identify and measure wounds using augmented reality (AR) glasses. The user-friendly application seamlessly integrates with Vuzix Shield Model 492 smart glasses running Android 11, embedding the AI model within its framework. This empowers nurses visiting patients with chronic wounds at home to conduct real-time video calls with wound care experts through the smart glasses. This enables remote expert assessment of the wounds and immediate guidance on treatment, potentially improving patient outcomes and reducing the healthcare burden.

Methods: This study presents the development of a novel application leveraging Vuzix Glasses and Android Studio to enhance remote wound care management. The application serves as a conduit between the Vuzix Glasses and Telko Live software, a web-based platform facilitating video calls. By enabling seamless connectivity, our solution enables real-time video transmission from the Vuzix Glasses worn by nurses at the patient's location to wound care experts at their desktops via Telko Live. Results and discussion: our approach leverages machine learning (ML) algorithms for precise chronic wound segmentation. We initially employ YOLOv4 for detection and then transition to TensorFlow for streamlined integration into the Android application. A dataset of 322 chronic wound images, augmented for enhanced performance, is utilized for training. While TensorFlow yields strong results in detecting a calibration panel (90.79%), chronic wound detection currently achieves a modest rate (38.46%) due to limited and diverse image data. However, segmentation achieves promising metrics (dice coefficient = 85%, precision = 94%, recall score = 86%, accuracy = 95%). This setup empowers remote experts to observe and provide guidance based on the nurse's perspective, thereby enhancing the quality and efficiency of wound care delivery. Our work exemplifies the fusion of wearable technology and telehealth solutions to address critical healthcare challenges, particularly in remote patient monitoring and specialized care delivery.