Skip to main content

Table 5 Analysis and assessment of the frameworks

From: Evaluation frameworks for digital nursing technologies: analysis, assessment, and guidance. An overview of the literature

  Analysis
Technology Field Framework Authors/Year Perspective Stated Purpose Success Definition/Description
Information and Communication technologies
(Health Information Systems (HIS))
Infoway benefits evaluation Framework Francis Lau et al. 2007 [25] Investment programs for digital technologies (to guide evaluations) 1. Provide a high-level evidence based model to guide subsequent field evaluation Success measured by analysing the results of the evaluation (Factors based on the van der Meijden et al. model [36])
Information and Communication technologies
(Health Information Technologies (EHR))
Health Information Technology Evaluation Framework (HITREF) Sockolow et al. 2012 [26] Universal perspective (mainly influenced by health services research and informatics) 1. Conceptual tool for framing evaluations studies in assessing EHR-based implementations in organizational, systematic, and environmental contexts
2. Displaying evaluation criteria
No success definition (measuring the success by analysing the results of the evaluation)
Information and Communication technologies
(Hospital Information Systems)
Hospital Information System HIS Success Framework Sadoughi et al. 2013 [27] Universal perspective 1.Identification of Hospital Information System success and failure factors and the evaluation methods of these factors Success as a dynamic concept. Success is when the technology achieves its intended purpose. (+ time, budget, and user satisfaction)
Information and Communication technologies
(Integrated Health Information Systems (IHIS))
Development of an Evaluation Framework for Health Information Systems (DIPSA Framework) Stylianides et al. 2018 [28] Healthcare Organization 1.Evaluation framework for hospitals utilizing IHIS to help identify any existing deficiencies in the system No success definition (measuring the success by analysing the results of the evaluation)
Information and Communication technologies
(Health Information Systems)
Human, Organization, Process and Technology-fit (HOPT-FIT) Yusof 2019 [29] Healthcare Organization (focus on technology induced errors) 1. Evaluate HIS performance and efficiency
2. Systematically guide error evaluation
3. Describing the Human-Organization-Process-Technology fit
No success definition (measuring success with the included dimensions of HIS success)
Information and Communication technologies
(Clinical Information Systems (CIS))
Clinical Information Systems Success Model (CISSM) Garcia-Smith & Effken 2013 [21] Nurse’s perspective 1. Framework for evaluating CIS success from the nurse’s perspective Success = net benefit (“degree to which a nurse believes that using a particular system enhances job performance”)
Information and Communication technologies
(Information and Communication technologies for nurses)
Adapted nursing care performance framework Rouleau et al. 2017 [30] Nurse’s perspective 1. Illustrate how ICTs interventions influence nursing care and impact health outcomes No success definition (measuring the success by analysing the results of the evaluation)
Telemedicine/Telecare Model for Assessment of Telemedicine (MAST Manual) Kidhom et al. 2010 [31] Universal perspective (user-based decision making, research) 1. Describe effectiveness
2. Contribution to quality of care of telemedicine applications
3. Produce a basis for decision making
No success definition (measuring the success by analysing the results of the evaluation)
Telemedicine/Telecare Comprehensive evaluation framework for telemedicine implementation Chang 2015 [24] Universal perspective (decision making for individuals, organizations, and communities) 1. Summarising important themes for the evaluation of telemedicine systems
2. Support related stakeholders’ decision-making by promoting general understanding, and resolving arguments and controversies
Long-term implementation
Telemedicine/Telecare The layered telemedicine implementation model Broens et al. 2007 [23] Universal perspective (the focus on individual determinants/ perspectives changes throughout the development life cycle) 1. Detailed classification of the determinants of the success of future telemedicine implementations Successful implementation (“putting an idea or a concept into actual practice”)
Sensor Technologies
(Connected Sensor Technologies:
including wearables. biosensors)
Evaluation Framework for Fit-For-Purpose Connected Sensor Technologies Coravos et al. 2020 [19] Healthcare System Perspective (users and other stakeholders) 1. Working evaluation framework that reflects different types of risks
2. Framework is conducted to better manage these risks
3. Make information on sensor technologies more comparable and understandable
No success definition (success could be measured by analysing the results of the evaluation and comparing them with the standards for connected sensors)
Digital Health
(Digital Health Interventions (DHI))
Design and Evaluationof DHI Framework Kowatsch et al. 2019 [14] Universal perspective (researchers and practitioners) 1. Framework for the design and evaluation of DHI
2. Showing evaluation criteria and implementation barriers to be considered during the life cycle phases of DHI
3. Support researchers and practitioners from conception to large-scale implementations
A successful DHI needs to consider “the selection of suitable evaluation criteria and the overcoming of implementation barriers”
Digital Health
(Digital Healthcare Services: mHealth, AI, and robotics)
Health technology assessment framework for digital healthcare services (Digi HTA) Jari et al. 2019 [20] Healthcare System Perspective (decision making) 1. Inform decisionmakers in order to better support the introduction of new health technologies No success definition (success could be measured by analysing the results of the evaluation)
Digital Health
(Digital Health Technologies)
Digital Health Score Card Mathews et al. 2019 [32] Universal perspective (multi-stakeholder approach) 1. Multi-stakeholder approach to objectively evaluate digital health solutions Measuring the success by analysing the results of the evaluation (Success as the successful delivery of validated digital health solutions)
Health (and care) technologies Health Technology Adoption Framework Poulin et al. 2013 [33] Healthcare Organization 1. Framework with clear, user-validated criteria for evaluating new health technologies for adoption at the local level No success definition (success could be measured by analysing the results of the evaluation)
Health (and care) technologies Nonadoption, Abandonment, Scale-up, Spread, and Sustainability Framework
(NASSS Framework) [34]
Greenhalgh et al. 2017 Universal perspective 1. Framework to help predict and evaluate the success of a technology-supported health or social care program
2. Help to design, develop, implement, scale up, spread, and sustain technology-supported health or social care programs by identifying key challenges in different domains and the interactions between them
Adoption, scale-up, spread, and sustainability of a technology
E-health programs Khoja–Durrani–Scott Framework for e-Health Evaluation Khoja et al. 2013 [22] Universal perspective (included tools usable for managers, healthcare providers, and clients) 1. Comprehensive Framework to show relevant themes for e-health evaluation No success definition (measuring the success by analysing the results of the evaluation)
Clinical informatic interventions RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) (expanded to clinical informatics) Bakken & Ruland 2009 [35] Healthcare Organization (implementation in organizational practice) 1. Used to design, implementation, evaluation, and reporting of clinical informatics with a goal of translation of research into practice No success definition (measuring the success by analysing the results of the evaluation)