Developing a quality assessment framework for health technology in human movement care: A E-Delphi study

Abstract

Purpose: Worldwide healthcare faces major challenges. Digital health technology can potentially be part of a solution for the problem, but implementing and upscaling digital health technology remains challenging. One of the main barriers of healthcare professionals seem to be related to not having sufficient information or knowledge about the available technology and to make an educated decision to use a specific technology for a specific patient. This lack of knowledge can be translated into not knowing what technology is available and not knowing what the quality of technology is. Meanwhile several assessment frameworks are being developed, but these frameworks are specifically developed to assess one type of technology (i.e. mobile apps) or can only be executed with support from the developer¹. Therefore we aimed to develop a quality assessment framework for digital health technologies from the perspectives of experts in the field. Method: A national e-Delphi procedure was conducted². Thirty experts were included with expertise in physical therapy, research, health technology development, teaching (in physical therapy) and policy advisors. The Delphi consisted of four rounds and each round consisted of a questionnaire and an anonymized report. The first three rounds consisted of questions regarding (1) Information framework (2) Proposed criteria for the quality assessment framework and (3) Presentation. The fourth round a conceptual rubric was proposed. Consensus was reached when >70% of the experts agreed or a 7.0 or higher was scored on a criterium. Results/ discussion: After four rounds, this Delphi study resulted in an information framework, a quality assessment framework, a presentation label and a rubric. The information framework includes 15 items that describe functions, goals, costs and available scientific literature of the technology (2) a quality assessment framework consisting of 15 criteria that assess the technology in three categories: Healthy & Safe, Usability and Data safety. The experts found it important to present an overall score, a subscore per category and user-reviews. The experts also agreed on the proposed rubric which results in a five-star scoring system. This study showed what experts found important to assess before using digital technology in practice. Nevertheless, it remains important to consider whether the technology suits the digital literacy, health literacy of the patient and healthcare professional and whether technology supports in reaching the treatment goals. Additionally, technologies are rapidly evolving and therefore scores can change quickly, so keeping the scores up to date is challenging.

1.          Hoogendoorn P, Versluis A, van Kampen S, McCay C, Leahy M, Bijlsma M, et al. What Makes a Quality Health App-Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study. JMIR Form Res 2023

2.          Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med 2017 Sep 1;31(8):684–706.