Skip to main content
  • Study protocol
  • Open access
  • Published:

Effectiveness and implementation of an online intervention (MINDxYOU) for reducing stress and promote mental health among healthcare workers in Spain: a study protocol for a stepped-wedge cluster randomized trial

Abstract

Background

The World Health Organization has formally recognized that healthcare professionals are at risk of developing mental health problems; finding ways to reduce their stress is mandatory to improve both their quality of life and, indirectly, their job performance. In recent years, particularly since the COVID-19 pandemic outbreak, there has been a proliferation of online interventions with promising results. The purpose of the present study is twofold: to test the effectiveness of an online, self-guided intervention, MINDxYOU, to reduce the stress levels of healthcare workers; and to conduct an implementation study of this intervention. Additionally, an economic evaluation of the intervention will be conducted.

Methods

The current study has a hybrid effectiveness-implementation type 2 design. A stepped wedge cluster randomized trial design will be used, with a cohort of 180 healthcare workers recruited in two Spanish provinces (Malaga and Zaragoza). The recruitment stage will commence in October 2022. Frontline health workers who provide direct care to people in a hospital, primary care center, or nursing home setting in both regions will participate. The effectiveness of the intervention will be studied, with perceived stress as the main outcome (Perceived Stress Scale), while other psychopathological symptoms and process variables (e.g., mindfulness, compassion, resilience, and psychological flexibility) will be also assessed as secondary outcomes. The implementation study will include analysis of feasibility, acceptability, adoption, appropriateness, fidelity, penetration, and sustainability. The incremental costs and benefits, in terms of quality-adjusted life years, will be examined by means of cost-utility and cost-effectiveness analyses.

Discussion

MINDxYOU is designed to reduce healthcare workers’ stress levels through the practice of mindfulness, acceptance, and compassion, with a special focus on how to apply these skills to healthy habits and considering the particular stressors that these professionals face on a daily basis. The present study will show how implementation studies are useful for establishing the framework in which to address barriers to and promote facilitators for acceptability, appropriateness, adoption, feasibility, fidelity, penetration, and sustainability of online interventions. The ultimate goal is to reduce the research-to-practice gap.

Trial registration

This study was registered in ClinicalTrials.gov on 29/06/2022; registration number: NCT05436717.

Peer Review reports

Background

Although stress levels among healthcare workers are normally high, they have been accentuated by the COVID-19 pandemic. Over the last two years, healthcare workers have had to deal with the same pandemic-related stressors as any other citizens, in addition to their specific job-related stressors: continued exposure to infection, longer shifts––which has led to inevitable physical exhaustion and neglect of important aspects related to a healthy lifestyle––and difficult decisions regarding how to prioritize available resources in order to treat their patients, among others [1, 2]. Subjection to this sustained pressure, which is added to the normally high care burden of healthcare workers, entails a risk factor for developing mental health problems, including anxiety disorders, depression, somatoform disorders, post-traumatic stress disorders, and even suicidal ideation [3, 4].

The World Health Organization (WHO) has formally recognized that healthcare professionals are at risk of developing psychopathological symptoms, particularly those working in public health, primary care, emergency departments, and intensive care units [5, 6], including personnel employed by nursing homes [7]. It is essential to take care of the mental health of healthcare workers, not only to improve their quality of life, but also because the success of their work largely depends on it. Hence, it is especially necessary in the current circumstances to offer these workers effective and highly adaptive strategies that will enable them to cope with the numerous stressors to which they are exposed in order to prevent the appearance of mental health problems, also in the long term.

Many psychotherapeutic programs have been developed to combat stress. Interventions based on cognitive behavioral therapy (CBT) have proven effective for reducing stress, anxiety, and depression in different clinical populations [8,9,10]. Similarly, so-called “third wave” psychotherapies, such as mindfulness-based interventions (MBIs), compassion-based programs, and acceptance and commitment therapy (ACT), have also proved effective for reducing different psychopathological symptoms [11,12,13,14,15], including stress in healthcare professionals [16,17,18,19,20,21]. The most common format for the application of these psychotherapies has been face-to-face, both in group and individual therapy. However, for a few years, and particularly because of the pandemic, online psychotherapeutic programs have proliferated, making it possible to overcome obstacles to face-to-face attendance. They are very well suited to the needs of healthcare workers with schedules that are often difficult to reconcile [22].

There are a good number of online interventions (e.g., web-based, mobile applications, videoconference-delivered interventions, among others) aimed at treating stress, anxiety, depression, and insomnia, among others. Different studies have been published on the efficacy of online interventions to improve the quality of life and reduce the stress of healthcare workers. Some have verified that online CBT-based interventions are effective for reducing stress, suicidal ideation, or emotional distress in this population [20, 23,24,25,26], and others have also studied the efficacy of “third wave” online psychotherapies with positive results [27,28,29,30]. However, most of these programs have only achieved small effect sizes in terms of their effectiveness, and a fact that cannot be overlooked is that studies have also been published regarding online interventions found to be ineffective for reducing stress of healthcare workers, which makes further study into the characteristics of these programs and their possible limitations necessary. Moreover, the interventions that are tested in effectiveness studies are mostly discontinued once the study ends, which poses a major problem when it comes to the translation of efficacy studies on evidence-based interventions to daily clinical practice, a period estimated to be 17–20 years [31].

Methods

Study aims

This study aims to evaluate the effectiveness of a self-guided online program, MINDxYOU, based on “third wave” psychotherapy principles, for reducing the stress levels of healthcare workers through the practice of different exercises that promote mindfulness, acceptance, and self-compassion, in addition to indications on how to apply these skills to a healthy lifestyle. Furthermore, this project intends to emphasize the implementation of the intervention; implementation science is a discipline that is developing methods and procedures to ensure that evidence-based interventions are incorporated more quickly into daily clinical practice [32]. The implementation study includes 3 types of evaluation: 1) evaluation of the process (characteristics of the use of the intervention; measures before, during, and after implementation); 2) formative evaluation (adaptations for improvement based on participants’ feedback); and 3) summative evaluation (a compilation of the impact of the implementation strategy, evaluating the impact on indicators such as increase in the use of the intervention in the target context and economic impact).

As secondary aims, the impact of the intervention on other health-related outcomes such as psychopathological symptoms (depression, anxiety, and others) and process variables (resilience, mindfulness, compassion, and psychological flexibility) will also be evaluated, and an economic evaluation will be conducted.

Study design

An effectiveness-implementation type 2 design will be used [33], since this study will assess both the effectiveness of the intervention and the implementation process in the same degree, as co-primary aims [34]. The empirical study will take a stepped wedge (SW) cluster randomized controlled trial design; this design implies that every participant starts the trial in the control phase (no exposure to the intervention), and then undertakes the intervention in a staggered and sequential way. Once it is completed (8 weeks), the maintenance phase takes place, during which the participant can continue to use the program at will, but without supervision. There will be three possible sequences (see Fig. 1), and each of the participating centers will be randomly assigned to one of them. The participants from each site will all follow the same sequence.

Fig. 1
figure 1

The three possible sequences of the study phases. Note: A = Control phase; B = Intervention phase; C = Maintenance phase. Each cell represents a period of 8 weeks

SW designs are particularly recommended when the intervention being tested has strong evidence of positive effects or is very unlikely to cause harm [35], which is the case of our study. The present work is considered a closed cohort study, since all the study participants will start at the same time and will be evaluated at different time-points to be defined beforehand; the inclusion of new participants will not be allowed once the trial has commenced.

Participants

The study sample will comprise healthcare workers currently working in clinical settings: a hospital, a primary care (PC) center, and a nursing home in the provinces of Zaragoza (Aragon) and Malaga (Andalusia), Spain. The following specific inclusion criteria will apply: 1) employment as a physician, nurse, physiotherapist, psychologist, nursing assistant, ambulance technician; a trainee student in any health profession; or employment in a nursing home assisting patients; 2) aged between 18 and 70; 3) ability to understand Spanish; 4) digital literacy and access to a smartphone, tablet, or personal computer with Internet connection; 5) prospects of continued employment at the same site for the following 6 months; and 6) provision of informed consent.

The following exclusion criteria will apply: 1) presenting with a disorder that affects the central nervous system; 2) diagnosis of a severe mental illness (including severe depressive disorders, suicidal tendencies, bipolar disorders, panic disorders, anxiety or stress-related disorders, obsessive–compulsive disorders, and substance-related disorders); 3) presenting with a medical, infectious, or degenerative illness that is not under control; and 4) expertise in “third wave” psychotherapies (i.e., meditation, mindfulness, etc.).

Sample size

The sample size was calculated following the recommendations for SW designs [36]. First, the sample size for individual randomization needed to be determined; in this case, with an expected effect size of d = 0.50 for the primary outcome (Perceived Stress Scale, PSS) [37], which is a continuous variable, a statistical power of 80%, and an alpha level of 0.050, the required sample size would be 126 participants (N1). Subsequently, the following formula was considered:

$$m=\frac{\mathrm{N}1 (1-p)}{\mathrm{K}-\mathrm{N}1\mathrm{p}}$$

where m is the necessary sample for each cluster and for each assessment moment (variable to be determined), N1 is the already calculated sample size (126), p is the intra-class correlation coefficient (assumed: 0.04), and K is the number of clusters (6). The result would be 24 participants per cluster, and assuming 20% attrition, 30 participants should be recruited from each site. This would imply a total sample of 180 participants.

Procedure

The strategy to recruit study participants will commence in October 2022. It will include indirect techniques such as informative posters in the selected hospitals/PC centers/nursing homes containing a brief explanation of the study and the website link; in addition, the occupational health service of the hospital and a member of staff at each site will contact their peers and send them an informative email with a brief explanation of the study and the website link. Consequently, any individuals interested in taking part in the study will find on the website all the details regarding the study procedures and aims, together with the rights of every participant and an explanation of the way in which their data will be processed.

Informed consent will then be documented via electronic signature, and complying candidates will undergo a basic screening process, which will consist of a checklist of the inclusion criteria. Candidates who meet these self-reported criteria will be presented with a schedule for the following 2 weeks and they will be asked to select a time for a 20-min video call or telephone call from a member of the research team who will conduct the complete screening of the participant, including the assessment of psychiatric diagnoses (MINI v7.0.2). Candidates who meet all the criteria will formally be included as participants in the study and will be offered information regarding the following procedures, which will start with a baseline evaluation including different self-reported questionnaires that the participant will complete using an online platform. The participant will receive via email or text message the link to the evaluation platform. Once the control phase is over, the participant will receive via email or text message their credentials (i.e., user and password) to access to the online intervention platform so they can start the MINDxYOU program. An external researcher, not related to this study, will conduct the randomization process to one of the three possible sequences using EPIDAT 4.2 (see Fig. 1).

Clear indications will be given on how to use the platform in the welcome module (see Table 1). Participants will be assessed for all the study variables every 8 weeks; the usability measure (System Usability Scale, SUS) [38] will be completed after finishing module 1 and module 4. The research team will record the usage that each participant makes of the platform: the number of times it was accessed, modules completed, and number of tasks delivered. Two external professionals, one in each province and unrelated to the research group, will be in charge of monitoring the intervention, providing feedback to the participants, and making weekly contact with participants via their preferred means (telephone call, text message, or email); this contact will be used to motivate participants to complete the intervention. Both professionals will previously have been trained in the contents and procedures of the MINDxYOU program by the research team. Once the intervention is complete, focus groups will be held in the provinces of Zaragoza and Malaga to qualitatively assess the acceptance and suitability of the intervention, and interviews will be conducted with managers and directors from each site to evaluate different aspects regarding implementation of the program.

Table 1 Brief outline of the MINDxYOU program

Intervention

The MINDxYOU online program is based on the principles of “third wave” psychotherapies, such as the promotion of wellbeing through the practice of mindfulness, compassion, acceptance, and spirituality [39]. The effectiveness of “third wave” interventions has been widely contrasted empirically, also in the case of health professionals [16], and the MINDxYOU program was designed based on the experience of the research group on previous projects, such as the Attachment-Based Compassion Therapy [40] and Smiling is Fun [41] online programs, and also considering the results of the HEROES [42] and the PSY-COVID projects, the latter being an international research project that evaluated the psychosocial impact of the COVID-19 pandemic on both the general population and specific populations (healthcare personnel, university students, etc.).

MINDxYOU is a self-guided online program particularly designed for healthcare workers in order to both monitor and support the individual’s mental health. This program will aid healthcare professionals to deal with their daily, work-related stress through the practice of mindfulness, compassion, and acceptance exercises, along with indications on how to apply these concepts to basic aspects of a healthy lifestyle (i.e., physical activity, sleep hygiene, balanced diet, and socialization).

The program is self-administered and will be delivered over the Internet and available via smartphone, tablet, and personal computer. During the first 8 weeks, there will be minimum support provided by an external professional unrelated to the research group, who will make weekly contact with participants via their preferred means (i.e., telephone, text message, email) to promote adherence to the intervention, encouraging the participant to keep on using the program and strengthening their commitment to the exercises. However, this contact will neither be used to add new educational content nor for assessment purposes, which will be entirely covered by the online platform with online links to the questionnaire(s) they need to complete; nor will it be used for counseling purposes.

The intervention begins with a welcome module that introduces the main aims and procedures of the platform. Participants are invited to share a written testimony of their main stressors in their workplace, and their feelings and personal needs. The external professional will read each testimony and respond to each one by offering a rationale for why the intervention can be beneficial for their case, enhancing motivation to complete the intervention. The testimonies will be analyzed and used to improve the contents of the intervention wherever possible.

After the welcome module, the program will be structured into 4 modules, each dedicated to a specific topic (see Table 1). Every module will contain animated videos with explanations of the aims and contents of the topic to be addressed, along with some examples of exercises that participants will then be able to practice on their own. Written material will also be provided to give participants a deeper understanding of the contents of the module, and audios with guided practices will be included for each exercise. PDF files and audios will be available for participants to download and save for use in the future or for practice outside of the web platform. Practice logs will also be available, and participants will be able to complete them on the online platform to enable the research team to check the completion of tasks. It is recommended that a period of 2 weeks be dedicated to each module, although participants will be free to adapt this to their needs. The first module, on mindfulness, will introduce most of the basic concepts of the intervention, and these will also be taken up in the following modules. The last module will focus on how to apply the concepts and practices of mindfulness and compassion concepts to key aspects related to a healthy lifestyle (i.e., physical activity, diet, sleep, and socialization). The estimated time required to complete the whole program is 8 weeks; after week 8, participants will no longer be contacted by the member of the research team. However, the program will continue to be available for use indefinitely.

Assessment plan

All potential candidates will undergo an initial screening assessment conducted by a member of the research team who is a certified psychiatrist or clinical/health psychologist; this assessment will include MINI v7.0.2 to identify psychiatric disorders. Later, a baseline evaluation will be conducted, including an ad hoc sociodemographic questionnaire, primary and secondary outcomes, process variables, and cost-utility measures. After the baseline evaluation, the participants will be assessed again every 8 weeks. During the intervention, the participants will complete the SUS twice.

Diagnostic interview: Mini International Neuropsychiatric Interview (MINI) Version 7.0.2

MINI v7.0.2 [43] is a brief structured diagnostic interview for DSM-5 psychiatric disorders. It assesses the 17 most common disorders in mental health, those considered the most important to identify in clinical and research settings: major depressive episode, major depressive disorder, suicidal tendencies, suicidal behavior disorder, maniac episode, hypomanic episode, bipolar disorder type I, bipolar disorder with psychotic traits, bipolar disorder type II, other bipolar disorder, panic disorder, agoraphobia, social anxiety disorder, obsessive–compulsive disorder, post-traumatic stress disorder, alcohol abuse disorder, and substance abuse disorder. It takes approximately 15 min to administer, and the present study will use the Spanish version to verify that the participants do not present with any severe conditions. MINI has shown excellent psychometric properties, including high interrater reliability (k = 0.90 or higher) [43].

Primary outcome: Perceived Stress Scale (PSS)

PSS [37] consists of 10 items in which participants are asked to rate how unpredictable, uncontrollable, and overloaded they have found their life over the past month on a 5-point Likert-type scale (e.g., “how often have you felt that you were unable to control the important things in your life?”). Scores range between 0 and 40, and higher scores reflect higher levels of perceived stress. PSS was validated in Spanish [44], and this version presented good psychometric properties, including internal consistency (α = 0.81), test–retest reliability (r = 0.77), and sensitivity to change.

Secondary outcomes

The Patient Health Questionnaire (PHQ-9) [45] is a 9-item scale aimed at screening for depression in primary care and other medical settings. It assesses the frequency of depressive symptoms during the last 2 weeks using a scale between 0 (not at all) and 3 (nearly every day). The total score ranges between 0 and 27, with higher scores indicating higher severity of depression. PHQ-9 has presented excellent psychometric properties, including reliability, validity, and responsiveness [46].

The Generalized Anxiety Disorder-7 (GAD-7) [47] is a 7-item, self-report measure to assess the intensity of anxiety symptoms. It refers to the period of the past 2 weeks; each item is scored on a 4-point Likert-type scale, resulting in a total score that can range between 0 and 21, with higher values reflecting more severe anxiety symptoms. Good psychometric properties have been reported for GAD-7 in different populations, also for the Spanish version [48].

The Brief Symptoms Inventory-18 (BSI-18) [49] is an adapted version of the original 53-item, self-report questionnaire [50] that is designed to offer rapid screening for the symptoms of psychological disorders. Each item is scored on a 5-point Likert-type scale, reflecting on the past 7 days. Scores on the 18 items are summarized on the Global Severity Index (GSI). The BSI-18 also includes three symptom scales: Somatization, Depression, and Anxiety, each comprising six items. The Spanish version [51] shows reliable and valid psychometric properties.

Process variables

The Connor-Davidson Resilience Scale (CD-RISC) [52] is a 10-item, self-report measure addressed at assessing resilience. Each item is scored on a 5-point Likert scale, and the total score, which ranges between 1 and 5, is calculated by averaging the scores of the items; higher scores indicate higher levels of resilience. The questionnaire has shown good psychometric properties in people with anxiety or stress-related disorders. The Spanish version [53] also presented good internal consistency (α = 0.86) and test–retest reliability (r = 0.87).

The Five Facets of Mindfulness Questionnaire -15 item version (FFMQ-15) [54] is an adaptation of the original 39-item FFMQ, a questionnaire measuring the five facets of mindfulness: observing (i.e., noticing internal and external experiences such as sensations, thoughts, or emotions), describing (i.e., labeling internal experiences with words), acting with awareness (i.e., focusing on one’s activities in the present moment as opposed to behaving automatically), nonjudging of inner experience (i.e., taking a non-evaluative stance toward thoughts and feelings), and non-reacting to inner experience (i.e., allowing thoughts and feelings to come and go, without getting caught up by them). Each subscale of FFMQ-15 includes 3 items, scored on a Likert-type scale (1–5). A score for each subscale, ranging between 5 and 15, can be computed by summing the items, where higher scores indicate higher levels of the mindfulness facet. FFMQ-15 presents good psychometric properties, similar to those of the original questionnaire, and the Spanish version also presented good internal consistency for every subscale [55].

The Sussex-Oxford Compassion Scales (SOCS) [56] consist of two self-report measures designed to evaluate self-compassion and other-compassion. Each measure consists of 20 items that are answered on a 5-point Likert-type scale. A total score of each scale can be calculated by summing the scores of the items; each total score ranges between 20 and 100, with higher scores indicating higher levels of compassion. The psychometric properties of the SOCS have proved to be good [56], and the Spanish adaptation is currently being validated by our research group.

The Acceptance and Action Questionnaire-II (AAQ-II) [57] is a common measure for assessing experiential avoidance, a key element of “third wave” psychotherapies that refers to the tendency to avoid thoughts, feelings, memories, sensations, and other internal experiences, and which is very commonly associated with worse mental health outcomes, as opposed to psychological flexibility. The scale presents 7 items scored on a 7-point Likert-type scale, and the total score is calculated by summing all the items. Higher scores mean higher experiential avoidance. AAQ-II shows good psychometric properties, including internal consistency (α = 0.84) and test–retest reliability (r = 0.79). The Spanish version has presented similar psychometric qualities [58].

Cost-utility variables

The five-level version of the EuroQol five-dimensional classification system (EQ-5D) [59] is a widely used health-related quality of life measure. First, participants are asked to report the severity (1 = no problems, 2 = mild, 3 = moderate, 4 = severe, 5 = extreme) of the problems they may present on the day of reporting with regard to each of the following five domains: mobility, self-care, usual activities, pain, and anxiety/depression. The combination of the answers given to these domains results in 3,125 different health states. The utility scores are obtained from the EQ-5D classification system and are used to rate patients’ health-related quality of life, which normally range between 0 (although it is possible to present negative scores) and 1 (i.e., “perfect health”). These utility values will be calculated using the Spanish tariffs of EQ-5D-5L [60]. EQ-5D utility values will be used to estimate quality-adjusted life-years (QALYs), a common measure to assess the outcomes associated with different treatments, both in terms of patients’ quality of life and survival [61]. EQ-5D also asks participants to use a visual analog scale (EQ-VAS) to record their current overall health status, ranging between 0 (worst imaginable health) and 100 (best imaginable health).

The Client Service Receipt Inventory (CSRI) [62] is one of the most widely used resource-use measurement tools; it describes and measures service utilization patterns as a basis for estimating associated costs across healthcare, social care, and community settings. In the present study, the Spanish version [63] will be used to collect retrospective data (last 2 months) on medication and service use. For medication intake, patients will be asked to bring their daily medication prescriptions, and the following information will be recorded: drug name, dosage, total number of prescription days, and daily dosage consumed. With regard to service use, patients will be asked about total visits to emergency departments, total days of general inpatient hospital admissions, number of diagnostic tests administered, and total visits to primary care physician, nurse, social worker, psychologist, psychiatrist, group psychotherapy, and other community healthcare professionals, specifying in each case if these services were provided by the public or by the private sector.

Implementation outcomes

In accordance with the Consolidated Framework for Implementation Research (CFIR) model, and taking into consideration the guidelines by Proctor et al. [64] and recommendations for implementation studies of online interventions by Hermes et al. [65], the present study will assess the following outcomes for the implementation process: acceptability, adoption, appropriateness, feasibility, fidelity, penetration, sustainability, and implementation costs, which will be measured along with the cost-utility variables. Treatment expectancies will be assessed before commencing the intervention, and participants will again be asked the same questions after completing the last module to assess their opinion [66]. In addition, focus groups will be held with the key players involved in the study in each province (i.e., Zaragoza and Malaga) once the intervention is complete (posttreatment) to qualitatively evaluate different domains related to the implementation.

The first of these, acceptability, defined as the perception among implementation stakeholders that the intervention is useful or satisfactory, will be measured using the following measures:

  • The System Usability Scale (SUS) [67], which is a 10-item questionnaire to measure the usability which is qualitatively related to the quality and acceptability of the intervention [68]. Usability is defined as the ease of use perceived by the users of the implemented technology. Use will be made of the Spanish version of SUS, which has presented good internal consistency (α = 0.81) [69].

  • The Attitudes towards Psychological Online Interventions (APOI) [70], which includes 16 items to assess attitudes toward online interventions. APOI explores four dimensions; “Skepticism and Perception of Risks,” “Confidence in Effectiveness,” “Technologization Threat,” and “Anonymity Benefits”. APOI has shown good psychometric properties with acceptable to good internal consistency (α = 0.77) [70].

  • The Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I) [71] is an 8-item, 4-point Likert scale questionnaire that assesses the general satisfaction of participants regarding the received intervention. The total score ranges between 8 and 32. The scale has shown excellent internal consistency (α = 0.93) [72].

Another construct that will be assessed in this regard is appropriateness, which is defined as perceived fit, relevance, or compatibility of the evidence-based practice for a given practice setting. We will use the Intervention Appropriateness Measure (IAM) [73], which includes 4 items designed to measure appropriateness of the intervention. The scale has shown good psychometric properties with high levels of internal consistency (α = 0.85 to 0.91) and test–retest reliability coefficients (r = 0.73 to 0.88).

The third domain that will be assessed in terms of implementation is adoption, defined as the intention, initial decision, or action to try or employ an evidence-based practice. In our study, adoption will be measured by the number of participants who agreed to use the program (considering the total number that were invited to use it) and actually accessed the online program. We will gather this information from the passive data collected by the online platform.

The feasibility of the intervention will also be studied. This is defined as the extent to which a new treatment can be successfully used or carried out within a given agency or setting. Feasibility will be measured with passive data gathered by the online platform regarding use of the program, specifically the frequency of use during the study. Use will also be made of the Feasibility of Intervention Measure (FIM) [73], which includes 4 items designed to measure the feasibility of the intervention. The scale has shown good psychometric properties with high levels of internal consistency (α = 0.85 to 0.91) and test–retest reliability coefficients (r = 0.73 to 0.88).

The fifth domain of implementation that will be analyzed is fidelity. In the context of online interventions, fidelity is defined as expected clinically meaningful use. This dimension will be measured with passive data collected by the online platform, specifically the number of modules and tasks completed. Penetration, in turn, will be measured considering the number of participants who agreed to use the intervention and completed at least 4 of the 5 modules of the program. Finally, sustainability, defined by the extent to which a new intervention is maintained, will be evaluated during the interviews and/or focus groups with the managers or directors of the different sites.

Data analysis

Effectiveness analysis

Both intention-to-treat analysis and per-protocol analyses (i.e., those participants who complete at least 4 of the 5 modules of the intervention) will be performed. The analyses will include the description and elementary head-to-head comparisons between both phases (i.e., control vs. intervention). Specifically, the variables for each of the phases will be described by using descriptive statistics (means and 95% confidence intervals in the case of quantitative variables with normal distribution, medians, and interquartile range in the case of quantitative variables with non-normal distribution). To confirm the main hypothesis, all of the effectiveness variables (i.e., PSS and secondary outcomes) will be compared between the intervention and control phases. A multilevel regression analysis will be performed including the cluster as a random effect, and multivariate multilevel regression analysis will be performed to adjust for possible confounders. The magnitude of the effect of the improvement and the Number Needed to Treat (NNT) will also be reported, using the cut-off points already established in the primary outcome as criteria for considering whether a participant is a responder.

Economic evaluation

The economic evaluation will be carried out following the recommendations of the Spanish guidelines for the economic evaluation of health technologies [74]. A societal perspective will be adopted, considering both direct and indirect costs. The cost-utility analyses will consider the effect of the intervention on the QALYs, and the results will be expressed in terms of the incremental cost-utility ratio (ICUR), calculated by dividing the difference in total costs between the intervention phase and the control phase by the difference in QALYs between both phases. A cost-effectiveness analysis will be also conducted, in this case considering the effect of the intervention on perceived stress (PSS). These results will be expressed in terms of the incremental cost-effectiveness ratio (ICER), calculated by dividing the difference in total costs between the treatment phase and the control phase by the difference in PSS scores between both phases.

Implementation analysis

In order to perform the analysis related to the implementation process of the intervention, different outcomes will be considered according to the adopted model [64, 65]. The analysis of the passive data consists of objective and direct counts (e.g., logins, frequency of use, modules completed, task completed, etc.). The quantitative data extracted from the questionnaires will be analyzed according to the appropriate methodology in each case. Some data will be collected qualitatively through interviews with the different stakeholders.

Ethics and dissemination

All procedures performed in this study will adhere to the 1964 Declaration of Helsinki and its most recent amendments (7th revision, adopted by the 64th World Medical Association General Assembly, Fortaleza, Brazil). Signed informed consent will be obtained from all participants, who will not receive any compensation, once they have been informed of the study procedures, potential risks, and their right to withdraw at any time from the study. The Research Ethics Committee of the Autonomous Community of Aragon (CEICA) and the Ethics and Research Committee of Northeast Malaga evaluated and approved the study protocol in July 2022 (PI22/341). Any important protocol modifications (e.g., changes to eligibility criteria, outcomes, analyses) will need to be approved by the ethics committees. To guarantee the confidentiality of the information, all the data collected by the platform will comply with the provisions of the Spanish Data Protection and Digital Rights Act 3/2018 (LOPD), which adapts Spanish legislation to the European Union’s General Data Protection Regulation (GRDP). A specific plan to guarantee maximum security of all collected data will be implemented for this project, and an assessment will be made of any possible risks and impacts on the information flow. The technology platforms will be separated into two completely independent systems, which will access two also independent databases. This will provide complete disaggregation of patient data, i.e., users’ personal data will be separated from their clinical record, multiplying by 2 all the security measures protecting end users. The data will remain separate on 2 different servers, with clinical data hosted on server 1 and personal data on server 2.

Once the study is complete, we will publish our results in international peer-reviewed biomedical journals and present them at national and international conferences. The implementation study will be conducted and reported according to the Standards for Reporting Implementation Studies (StaRI) Statement, indicated for reporting implementation studies focusing on enhancing the adoption and sustainability of the intervention [75]. A report will also be sent to the Instituto de Salud Carlos III (main funding body). The lead researcher will organize an end-of-study knowledge translation seminar. The main aim of this activity will be to share the study findings with stakeholders in order to discuss how to maximize uptake of the findings and to determine future research directions.

Discussion

Healthcare workers are in need of effective interventions to reduce their stress levels, which have been very significantly affected by the COVID-19 pandemic, in order to bring about a reduction in their risk of developing mental health disorders and, consequently, enhancement of their quality of life and job performance. The pandemic has helped to raise the visibility of healthcare workers as a risk group for developing mental health problems, which has facilitated the proliferation of different interventions in the recent years to address the needs of this population.

In this regard, online interventions have become particularly popular, both during and after the pandemic, as they offer a solution when face-to-face treatments are not available or recommended. Different programs have been adapted to the online format, mainly those based on classic CBT and “third wave” psychotherapies [20, 23,24,25,26,27,28,29,30]. Both types of interventions have been tested, with generally positive effects in stress reduction, although the effect sizes in many cases were small, which indicates that there is still room for improvement. The problem with some of these interventions might be that they deal with stress in a more generic way, rather than focusing on the kind of stressors that healthcare workers face on a daily basis. In this regard, a systematic review [76] suggested that the next step when developing online interventions targeting healthcare professionals was to adjust them to the particular needs of this population; therefore, an intervention that considers this aspect could generate both a higher degree of adherence and greater impact in terms of stress reduction.

MINDxYOU is a self-guided, Internet-delivered program that has been designed following the path drawn by previous studies offering online interventions to deal with different psychopathological symptoms (e.g., depression, anxiety, stress) [25, 26, 40, 41] and, using the feedback provided by different healthcare workers (i.e., doctors, nurses, psychologists), it adds a focus customized to the needs of these professionals, with examples that relate to common problems in their daily lives. The program offers different modules that enable the practice mindfulness, acceptance, and compassion, which constitute the core of the intervention, and a final module focusing on how to apply such skills to the main aspects of a healthy lifestyle (i.e., diet, sleep, exercise, socialization). This is justified by the importance of such aspects in mental health, and considering that many healthcare professionals have neglected their personal needs owing to their high levels of stress during the pandemic [2]. On the whole, MINDxYOU aims to cover the main needs of healthcare workers with regard to their stress management, and the present study will test whether the program is both effective and cost-effective.

Furthermore, the problem with many psychological interventions is the gap between research and practice. This means that, despite being proven effective, interventions generally take quite a number of years to be applied in the real-world practice [31]. Thus, implementation science focuses on the development of studies to reduce the know-do gap, focusing on the barriers to and facilitators for the implementation process [77]. Based on previous RCTs, we expect to adapt the intervention according to users’ experience and opinions [78, 79]. The implementation study will be conducted in order to assess the impact of the intervention on participants’ stress levels, the direct and indirect costs of the intervention, and the different factors influencing the implementation process: acceptability, appropriateness, adoption, feasibility, fidelity, penetration and sustainability.

If our hypotheses are confirmed, MINDxYOU will turn out to be both effective and cost-effective, and our results will provide information on the potential use of information and communications technologies (ICTs) related to the cost-effectiveness of Internet-based psychological interventions. Moreover, the study will show how implementation studies are useful to establish the framework with which to deal with the barriers to online psychotherapies and promote their implementation. In summary, this study could make a significant contribution to promote the use of online psychotherapy for reducing healthcare workers’ stress and reduce the gap between research and practice. 

Availability of data and materials

Upon completion of the study, the datasets generated and/or analyzed during the current study will be made available to researchers on request to the corresponding authors: Adrián Pérez-Aranda (aparanda@iisaragon.es) and Alberto Barceló-Soler (abarcelosoler@hotmail.com). No manuscripts related to this study have yet been published or submitted to any journal.

Abbreviations

AAQ:

Acceptance and Action Questionnaire

ACT:

Acceptance and Commitment Therapy

APOI:

Attitudes towards Psychological Online Interventions

BSI:

Brief Symptom Inventory

CBT:

Cognitive-Behavioral Therapy

CD-RISC:

Connor-Davidson Resilience Scale

CFIR:

Consolidates Framework for Implementation Research

CSQ-I:

Client Satisfaction Questionnaire adapted to Internet-Based interventions

CSRI:

Client Service Receipt Inventory

EQ-5D:

EuroQoL 5 Dimensions

FFMQ:

Five Facet Mindfulness Questionnaire

FIM:

Feasibility of Intervention Measure

GAD:

Generalized Anxiety Disorder

IAM:

Intervention Appropriateness Measure

ICER:

Incremental Cost-Effectiveness Ratio

ICT:

Information and communications technology

ICUR:

Incremental Cost-Utility Ratio

MBI:

Mindfulness-Based Interventions

NNT:

Number Needed to Treat

PC:

Primary Care

PHQ:

Patient Health Questionnaire

PSS:

Perceived Stress Scale

QALY:

Quality-Adjusted Life Year

RCT:

Randomized Controlled Trial

SOCS:

Sussex-Oxford Compassion Scales

SUS:

System Usability Scale

SW:

Stepped Wedge

WHO:

World Health Organization

References

  1. Pfefferbaum B, North CS. Mental Health and the Covid-19 Pandemic. N Engl J Med. 2020;383:510–2. https://doi.org/10.1056/nejmp2008017.

    Article  CAS  PubMed  Google Scholar 

  2. Greenberg N, Docherty M, Gnanapragasam S, Wessely S. Managing mental health challenges faced by healthcare workers during covid-19 pandemic. BMJ. 2020;368:m1211.

    Article  PubMed  Google Scholar 

  3. Karasu F, Öztürk Çopur E, Ayar D. The impact of COVID-19 on healthcare workers’ anxiety levels. J Public Heal. 2021;:1–11. doi:https://doi.org/10.1007/s10389-020-01466-x.

  4. Williamson V, Stevelink SAM, Greenberg N. Occupational moral injury and mental health: systematic review and meta-analysis. Br J Psychiatry. 2018;212:339–46. https://doi.org/10.1192/bjp.2018.55.

    Article  PubMed  Google Scholar 

  5. Cullen W, Gulati G, Kelly BD. Mental health in the COVID-19 pandemic. QJM. 2020;113:311–2. https://doi.org/10.1093/qjmed/hcaa110.

    Article  CAS  PubMed  Google Scholar 

  6. World Health Organization (WHO). Mental health considerations during COVID-19 outbreak. Geneva, Switzerland.; 2020.

  7. Sarabia-Cobo C, Pérez V, de Lorena P, Hermosilla-Grijalbo C, Sáenz-Jalón M, Fernández-Rodríguez A, et al. Experiences of geriatric nurses in nursing home settings across four countries in the face of the COVID-19 pandemic. J Adv Nurs. 2021;77:869–78. https://doi.org/10.1111/jan.14626.

    Article  PubMed  Google Scholar 

  8. Lewis C, Roberts NP, Andrew M, Starling E, Bisson JI. Psychological therapies for post-traumatic stress disorder in adults: systematic review and meta-analysis. Eur J Psychotraumatol. 2020;11:1729633. https://doi.org/10.1080/20008198.2020.1729633.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Kowalik J, Weller J, Venter J, Drachman D. Cognitive behavioral therapy for the treatment of pediatric posttraumatic stress disorder: a review and meta-analysis. J Behav Ther Exp Psychiatry. 2011;42:405–13.

    Article  PubMed  Google Scholar 

  10. Cuijpers P, Sijbrandij M, Koole S, Huibers M, Berking M, Andersson G. Psychological treatment of generalized anxiety disorder: a meta-analysis. Clin Psychol Rev. 2014;34:130–40.

    Article  PubMed  Google Scholar 

  11. Chiesa A, Serretti A. Mindfulness-based stress reduction for stress management in healthy people: a review and meta-analysis. J Altern Complement Med. 2009;15:593–600. https://doi.org/10.1089/acm.2008.0495.

    Article  PubMed  Google Scholar 

  12. Ledesma D, Kumano H. Mindfulness-based stress reduction and cancer: a meta-analysis. Psychooncology. 2009;18:571–9. https://doi.org/10.1002/pon.1400.

    Article  PubMed  Google Scholar 

  13. Carrière K, Khoury B, Günak MM, Knäuper B. Mindfulness-based interventions for weight loss: a systematic review and meta-analysis. Obes Rev. 2018;19:164–77. https://doi.org/10.1111/obr.12623.

    Article  PubMed  Google Scholar 

  14. Wakelin KE, Perman G, Simonds LM, Wakelin K. Effectiveness of self-compassion-related interventions for reducing self-criticism: A systematic review and meta-analysis. 2021.

  15. Phillips WJ, Hine DW. Self-compassion, physical health, and health behaviour: a meta-analysis. Health Psychol Rev. 2021;15:113–39. https://doi.org/10.1080/17437199.2019.1705872.

    Article  PubMed  Google Scholar 

  16. Kriakous SA, Elliott KA, Lamers C, Owen R. The effectiveness of mindfulness-based stress reduction on the psychological functioning of healthcare professionals: a systematic review. Mindfulness (N Y). 2021;12:1–28. https://doi.org/10.1007/s12671-020-01500-9.

    Article  Google Scholar 

  17. Rudaz M, Twohig MP, Ong CW, Levin ME. Mindfulness and acceptance-based trainings for fostering self-care and reducing stress in mental health professionals: a systematic review. J Context Behav Sci. 2017;6:380–90.

    Article  Google Scholar 

  18. Asuero AM, Queraltó JM, Pujol-Ribera E, Berenguera A, Rodriguez-Blanco T, Epstein RM. Effectiveness of a mindfulness education program in primary health care professionals: a pragmatic controlled trial. J Contin Educ Health Prof. 2014;34:4–12. https://doi.org/10.1002/chp.21211.

    Article  PubMed  Google Scholar 

  19. Aranda Auserón G, Elcuaz Viscarret MR, Fuertes Goñi C, Güeto Rubio V, Pascual Pascual P, Sainz de Murieta García de Galdeano E. Evaluation of the effectiveness of a Mindfulness and Self-Compassion program to reduce stress and prevent burnout in Primary Care health professionals. Aten Primaria. 2018;50:141–50. doi:https://doi.org/10.1016/j.aprim.2017.03.009.

  20. Mistretta EG, Davis MC, Temkit M, Lorenz C, Darby B, Stonnington CM. Resilience training for work-related stress among health care workers. J Occup Environ Med. 2018;60:559–68. https://doi.org/10.1097/JOM.0000000000001285.

    Article  PubMed  Google Scholar 

  21. Brinkborg H, Michanek J, Hesser H, Berglund G. Acceptance and commitment therapy for the treatment of stress among social workers: a randomized controlled trial. Behav Res Ther. 2011;49:389–98. https://doi.org/10.1016/j.brat.2011.03.009.

    Article  PubMed  Google Scholar 

  22. Gruber J, Prinstein MJ, Clark LA, Rottenberg J, Abramowitz JS, Albano AM, et al. Mental health and clinical psychological science in the time of COVID-19: Challenges, opportunities, and a call to action. Am Psychol. 2021;76:409–26. https://doi.org/10.1037/amp0000707.

    Article  PubMed  Google Scholar 

  23. Imamura M, Chen J, Matsubayashi SR, Targino RA, Alfieri FM, Bueno DK, et al. Changes in pressure pain threshold in patients with chronic nonspecific low back pain. Spine (Phila Pa 1976). 2013;38:2098–107.

    Article  Google Scholar 

  24. Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, et al. Effectiveness of a Mobile Phone-Based Intervention to Reduce Mental Health Problems in Healthcare Workers During the COVID-19 Pandemic: A Randomized Controlled Trial (PsyCovidApp Trial). SSRN Electron J. 2020;:In press. doi:https://doi.org/10.2139/ssrn.3749214.

  25. Ketelaar SM, Nieuwenhuijsen K, Gärtner FR, Bolier L, Smeets O, Sluiter JK. Effect of an E-mental health approach to workers’ health surveillance versus control group on work functioning of hospital employees: a Cluster-RCT. PLoS One. 2013;8(9):e72546.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Smoktunowicz E, Lesnierowska M, Carlbring P, Andersson G, Cieslak R. Resource-based internet intervention (med-stress) to improve well-being among medical professionals: randomized controlled trial. J Med Internet Res. 2021;23(1):e21445.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Nourian M, Nikfarid L, Khavari AM, Barati M, Allahgholipour AR. The impact of an online mindfulness-based stress reduction program on sleep quality of nurses working in covid-19 care units: a clinical trial. Holist Nurs Pract. 2021;35:257–63. https://doi.org/10.1097/HNP.0000000000000466.

    Article  PubMed  Google Scholar 

  28. Morrison Wylde C, Mahrer NE, Meyer RML, Gold JI. Mindfulness for novice pediatric nurses: smartphone application versus traditional intervention. J Pediatr Nurs. 2017;36:205–12. https://doi.org/10.1016/j.pedn.2017.06.008.

    Article  PubMed  Google Scholar 

  29. Hersch RK, Cook RF, Deitz DK, Kaplan S, Hughes D, Friesen MA, et al. Reducing nurses’ stress: a randomized controlled trial of a web-based stress management program for nurses. Appl Nurs Res. 2016;32:18–25. https://doi.org/10.1016/j.apnr.2016.04.003.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Lee D, Lee WJ, Choi SH, Jang JH, Kang DH. Long-term beneficial effects of an online mind-body training program on stress and psychological outcomes in female healthcare providers: a non-randomized controlled study. Medicine (Baltimore). 2020;99: e21027. https://doi.org/10.1097/MD.0000000000021027.

    Article  Google Scholar 

  31. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104:510–20. https://doi.org/10.1258/jrsm.2011.110180.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3:1–12. https://doi.org/10.1186/S40359-015-0089-9.

    Article  Google Scholar 

  33. Green BB, Coronado GD, Schwartz M, Coury J, Baldwin LM. Using a continuum of hybrid effectiveness-implementation studies to put research-tested colorectal screening interventions into practice. Implement Sci. 2019;14:1–9.

    Article  CAS  Google Scholar 

  34. Geoffrey M. Curran, PhD*, Mark Bauer, MD†, Brian Mittman, PhD‡, Jeffrey M. Pyne M, and Cheryl Stetler P. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact Geoffrey. Account Patient Saf Policy Reform. 2012;50:159–84.

  35. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: Rationale, design, analysis, and reporting. BMJ. 2015;350:h391.

    Article  CAS  PubMed  Google Scholar 

  36. Hemming K, Taljaard M. Sample size calculations for stepped wedge and cluster randomised trials: A unified approach. J Clin Epidemiol. 2016;69:137–46. https://doi.org/10.1016/j.jclinepi.2015.08.015.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385. https://doi.org/10.2307/2136404.

    Article  CAS  PubMed  Google Scholar 

  38. Brooke J. SUS: A “Quick and Dirty” Usability Scale. In: Jordan PW, Thomas B, Weerdmeester AB, McClelland IL, editors. Usability Evaluation In Industry. London, UK: Taylor & Francis; 1996. p. 189–94.

    Google Scholar 

  39. Kahl KG, Winter L, Schweiger U. The third wave of cognitive behavioural therapies: What is new and what is effective? Curr Opin Psychiatry. 2012;25:522–8. https://doi.org/10.1097/YCO.0b013e328358e531.

    Article  PubMed  Google Scholar 

  40. Campos D, Navarro-Gil M, Herrera-Mercadal P, Martínez-García L, Cebolla A, Borao L, et al. Feasibility of the internet attachmentbased compassion therapy in the general population: protocol for an open-label uncontrolled pilot trial. JMIR Res Protoc. 2020;9(8):e16717.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Montero-Marín J, Araya R, Pérez-Yus MC, Mayoral F, Gili M, Botella C, et al. An internet-based intervention for depression in primary care in Spain: a randomized controlled trial. J Med Internet Res. 2016;18(8):e231.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Mascayano F, van der Ven E, Moro MF, Schilling S, Alarcón S, Al Barathie J, et al. The impact of the COVID-19 pandemic on the mental health of healthcare workers: study protocol for the COVID-19 HEalth caRe wOrkErS (HEROES) study. Soc Psychiatry Psychiatr Epidemiol. 2022;57:633–45. https://doi.org/10.1007/s00127-021-02211-9.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Sheehan D V., Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. In: Journal of Clinical Psychiatry. 1998. p. 22–33. https://pubmed.ncbi.nlm.nih.gov/9881538/. Accessed 3 Feb 2022.

  44. Remor E. Psychometric properties of a European Spanish version of the Perceived Stress Scale (PSS). Span J Psychol. 2006;9:86–93. https://doi.org/10.1017/S1138741600006004.

    Article  PubMed  Google Scholar 

  45. Kroenke K, Spitzer RL. The PHQ-9: A New Depression Diagnostic and Severity Measure. Psychiatry Ann. 2002;32.

  46. Furukawa TA. Assessment of mood: guides for clinicians. J Psychosom Res. 2010;68:581–9. https://doi.org/10.1016/J.JPSYCHORES.2009.05.003.

    Article  PubMed  Google Scholar 

  47. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch Intern Med. 2006;166:1092–7.

    Article  PubMed  Google Scholar 

  48. García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes. 2010;8:8.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Derogatis LR. Brief Symptoms Inventory (BSI-18). Administration, scoring and procedures. Minneapolis: NCS Pearson; 2001.

  50. Derogatis LR, Melisaratos N. The brief symptom inventory: an introductory report. Psychol Med. 1983;13:595–605.

    Article  CAS  PubMed  Google Scholar 

  51. Miaja Ávila M, Moral de la Rubia J. Validación del Inventario Breve de Síntomas (BSI-18) en mujeres Mexicanas diagnosticadas con cáncer de mama. Psicooncologia. 2017;14:307–24. doi:https://doi.org/10.5209/PSIC.57088.

  52. Campbell-Sills L, Stein MB. Psychometric analysis and refinement of the Connor-Davidson Resilience Scale (CD-RISC): Validation of a 10-item measure of resilience. J Trauma Stress. 2007;20:1019–28. https://doi.org/10.1002/jts.20271.

    Article  PubMed  Google Scholar 

  53. Soler Sánchez MI, Meseguer de Pedro M, García Izquierdo M. Propiedades psicométricas de la versión española de la escala de resiliencia de 10 ítems de Connor-Davidson (CD-RISC 10) en una muestra multiocupacional. Rev Latinoam Psicol. 2016;48:159–66.

  54. Gu J, Strauss C, Crane C, Barnhofer T, Karl A, Cavanagh K, et al. Examining the factor structure of the 39-item and 15-item versions of the Five Facet Mindfulness Questionnaire before and after mindfulness-based cognitive therapy for people with recurrent depression. Psychol Assess. 2016;28:791–802. https://doi.org/10.1037/pas0000263.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Cebolla A, García-Palacios A, Soler J, Guillen V, Baños R, Botella C. Psychometric properties of the Spanish validation of the Five Facets of Mindfulness Questionnaire (FFMQ). Eur J Psychiatry. 2012;26:118–26. https://doi.org/10.4321/S0213-61632012000200005.

    Article  Google Scholar 

  56. Gu J, Baer R, Cavanagh K, Kuyken W, Strauss C. Development and Psychometric Properties of the Sussex-Oxford Compassion Scales (SOCS). Assessment. 2020;27:3–20. https://doi.org/10.1177/1073191119860911.

    Article  PubMed  Google Scholar 

  57. Bond FW, Hayes SC, Baer RA, Carpenter KM, Guenole N, Orcutt HK, et al. Preliminary psychometric properties of the acceptance and action questionnaire-II: a revised measure of psychological inflexibility and experiential avoidance. Behav Ther. 2011;42:676–88. https://doi.org/10.1016/j.beth.2011.03.007.

    Article  PubMed  Google Scholar 

  58. Ruiz FJ, Langer Herrera ÁI, Luciano C, Cangas AJ, Beltrán I. Midiendo la evitación experiencial y la infl exibilidad psicológica: Versión española del cuestionario de aceptación y acción - II. Psicothema. 2013;25:123–9.

    PubMed  Google Scholar 

  59. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20:1727–36. https://doi.org/10.1007/s11136-011-9903-x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Badia X, Roset M, Montserrat S, Herdman M, Segura A. [The Spanish version of EuroQol: a description and its applications. European Quality of Life scale]. Med Clin (Barc). 1999;112 Suppl:79–85. http://www.ncbi.nlm.nih.gov/pubmed/10618804. Accessed 23 Apr 2019.

  61. Stone G, Hutchinson A, Corso P, Teutsch S, Fielding J, Carande-Kulis V, et al. Understanding and using the economic evidence. In: The Guide to Community Preventive Services: What works to promote health? 2005. p. 449–63.

  62. Beecham J, Knapp M. Costing psychiatric interventions. In: Thronicroft G, Bewin C, Wing J, editors. Measuring Mential Health Needs. Oxford: Oxford University Press; 1992.

  63. Vázquez-Barquero J, Gaite L, Cuesta M, Garcia-Usieto E, Knapp M, Beecham J. Spanish version of the CSRI: a mental health cost evaluation interview. Arch Neuobiology. 1997;60:171–84.

    Google Scholar 

  64. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76.

    Article  PubMed  Google Scholar 

  65. Hermes EDA, Lyon AR, Schueller SM, Glass JE. Measuring the implementation of behavioral intervention technologies: Recharacterization of established outcomes. J Med Internet Res. 2019;21: e11752. https://doi.org/10.2196/11752.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Boß L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res. 2016;18(8):e234.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Bangor, Aaron, Kortum P, T. P, Miller, T. J. The System Usability Scale (SUS): an Empirical evaluation. Int J Hum Comput Interact. 2008;24:574.

  68. Brooke J. „SUS-A quick and dirty usability scale (in" Usability Evaluation in Industry", PW Jordan, B Thomas, I McLelland, BA Weerdmeester (eds))“. Usability Eval Ind (PW Jordan, B Thomas, BA Weerdmeester, AL Mcclell. 1996.

  69. Sevilla-Gonzalez MDR, Moreno Loaeza L, Lazaro-Carrera LS, Bourguet Ramirez B, Vázquez Rodríguez A, Peralta-Pedrero ML, et al. Spanish Version of the system usability scale for the assessment of electronic tools: development and validation. JMIR Hum Factors. 2020;7: e21161. https://doi.org/10.2196/21161.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Schröder J, Sautier L, Kriston L, Berger T, Meyer B, Späth C, et al. Development of a questionnaire measuring Attitudes towards Psychological Online Interventions-the APOI. J Affect Disord. 2015;187:136–41.

    Article  PubMed  Google Scholar 

  71. Bo L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res. 2016;18. doi:https://doi.org/10.2196/jmir.5952.

  72. Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5:233–7.

  73. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12:1–13.

    Article  Google Scholar 

  74. López Bastida J, Oliva J, Antoñanzas F, García-Altés A, Gisbert R, Mar J, et al. Propuesta de guía para la evaluación económica aplicada a las tecnologías sanitarias. Gac Sanit. 2010;24:154–70. https://doi.org/10.1016/j.gaceta.2009.07.011.

    Article  PubMed  Google Scholar 

  75. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for Reporting Implementation Studies (StaRI): explanation and elaboration document. BMJ Open. 2017;7: e013318.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Pospos S, Young IT, Downs N, Iglewicz A, Depp C, Chen JY, et al. Web-based tools and mobile applications to mitigate burnout, depression, and suicidality among healthcare students and professionals: a systematic review. Acad Psychiatry. 2018;42:109–20. https://doi.org/10.1007/S40596-017-0868-0.

    Article  PubMed  Google Scholar 

  77. Bauer MS, Kirchner JA. Implementation science: What is it and why should I care? Psychiatry Res. 2020;283. doi:https://doi.org/10.1016/J.PSYCHRES.2019.04.025.

  78. Castro A, López-del-Hoyo Y, Peake C, Mayoral F, Botella C, García-Campayo J, et al. Adherence predictors in an Internet-based Intervention program for depression. Cogn Behav Ther. 2018;47:246–61. https://doi.org/10.1080/16506073.2017.1366546.

    Article  PubMed  Google Scholar 

  79. Jesús MM, Prado-Abril J, Botella C, Fermin MC, Rosa B, Paola HM, et al. Expectations among patients and health professionals regarding Web-based interventions for depression in primary care: a qualitative study. J Med Internet Res. 2015;17(3):e67.

    Article  Google Scholar 

Download references

Acknowledgements

APA holds a “Sara Borrell” postdoctoral contract from the Instituto de Salud Carlos III (ISCIII; CD20/00181), not directly related to this project. We wish to thank the Mental Health Research Group (B17_20R) of the Government of Aragon (Spain) Department of Innovation, Research and University; the Research Network on Chronicity, Primary Care, and Health Promotion (RICAPPS, RD21/0016/0005), which is part of the Results Oriented Cooperative Research Networks in Health (RICORS) (Carlos III Institute of Health); and Feder Funds “Another way to make Europe.”

Funding

The study has been externally funded, after undergoing peer review, by the Instituto de Salud Carlos III (ISCIII), dependent on the Spanish Ministry of Science and Innovation, through a grant for health-related research projects (PI21/01338) and co-financed with European Union ERDF funds. The funder plays no role in study design, data collection and analysis, decision to publish, or manuscript preparation. The funding organization will audit trial conduct once a year.

Author information

Authors and Affiliations

Authors

Contributions

YLDH: Conceptualization, Resources, Project administration. SFM: Resources, Methodology. APA: Conceptualization, Methodology, Writing – original draft. ABS: Visualization, Resources, Writing – review & editing. JGP: Methodology, Writing – review & editing. EVM: Methodology, Writing – review & editing. DC: Methodology, Writing – review & editing. AMB: Visualization, Resources, Writing – review & editing. MBR: Resources, Writing – review & editing. BMK: Conceptualization, Supervision, Writing – review & editing. FMC: Conceptualization, Supervision, Writing – review & editing. JGC: Conceptualization, Supervision. All authors have read and approved the final manuscript.

Corresponding authors

Correspondence to Adrián Pérez-Aranda or Alberto Barceló-Soler.

Ethics declarations

Ethics approval and consent to participate

The Research Ethics Committee of the Autonomous Community of Aragon (CEICA) and the Ethics and Research Committee of Northeast Malaga evaluated and approved the study protocol in July 2022 (PI22/341). All study participants will sign an informed consent prior to randomization.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

López-del-Hoyo, Y., Fernández-Martínez, S., Pérez-Aranda, A. et al. Effectiveness and implementation of an online intervention (MINDxYOU) for reducing stress and promote mental health among healthcare workers in Spain: a study protocol for a stepped-wedge cluster randomized trial. BMC Nurs 21, 308 (2022). https://doi.org/10.1186/s12912-022-01089-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12912-022-01089-5

Keywords