Virtual patient simulation to improve nurses’ relational skills in a continuing education context: a convergent mixed methods study
BMC Nursing volume 21, Article number: 1 (2022)
Effective provider-patient communication is crucial to the delivery of high-quality care. Communication roadblock such as righting reflex is widely observed among providers and can lead to relational disengagement. In previous work, nurses felt ill-equipped to communicate effectively with HIV-positive patients to support medication adherence. Providing nurses with continuing education opportunities to improve their relational skills is a major target for optimizing the quality of care. Virtual patient simulation is one promising strategy that needs to be evaluated among graduate nurses. This study aimed to assess the acceptability of a virtual patient simulation to improve nurses’ relational skills in a continuing education context.
We conducted a convergent mixed methods study by combining a quantitative pre-experimental, one-group post-test design and a qualitative exploratory study. We used convenience and snowball sampling approaches to select registered nurses (n = 49) working in Quebec, Canada. Participants completed an online sociodemographic questionnaire, consulted the automated virtual patient simulation (informed by motivational interviewing), and filled out an online post-test survey. Descriptive statistics (mean, SD, median, interquartile range) were used to present quantitative findings. From the 27 participants who completed the simulation and post-test survey, five participated in a focus group to explore their learning experience. The discussion transcript was subjected to thematic analysis. At the final stage of the study, we used a comparison strategy for the purpose of integrating the quantitative and qualitative results.
Nurses perceived the simulation to be highly acceptable. They rated the global system quality and the technology acceptance with high scores. They reported having enjoyed the simulation and recommended other providers use it. Four qualitative themes were identified: motivations to engage in the simulation-based research; learning in a realistic, immersive, and non-judgmental environment; perceived utility of the simulation; and perceived difficulty in engaging in the simulation-based research.
The simulation contributed to knowledge and skills development on motivational interviewing and enhanced nurses’ self-confidence in applying relational skills. Simulation holds the potential to change practice, as nurses become more self-reflective and aware of the impact of their relational skills on patient care.
ISRCTN18243005, retrospectively registered on July 3 2020.
It widely documented that communication is an essential, core competency for all healthcare providers (HCP), including nurses . Effective HCP-patient communication is crucial to the delivery of high-quality care . Conversely, miscommunication can lead to medical errors, delayed treatment, low treatment adherence, medication errors, patient dissatisfaction, compromised patient safety, and so on [3,4,5]. One “communication roadblock” that is widely observed among HCP is the righting reflex . The righting reflex is a directive style of communication (or counselling), in which the HCP tries to convince a patient to take an action the HCP thinks is right (such as medication adherence). Although the righting reflex stems from a genuine intention to help, it can compromise the therapeutic relationship and lead to relational disengagement . The findings of our previous qualitative work  resonated with this communication roadblock. Indeed, when faced with people living with HIV non-adherence to their medication, nurses provided advice, information, and tips. However, nurses lacked tools to communicate effectively in supporting, convincing, and motivating their patients to “adhere” to their treatment. Providing nurses with continuing education (CE) opportunities to improve their relational skills is a major target for optimizing the quality of care.
Considering the barriers that keep some HCP from attending CE activities, such as staff shortage, cost, travel time, and competing demands , the use of virtual patient (VP) simulation in health professions education is a promising training modality to consider. VP simulation can be defined as an interactive, computerized simulation that relies on real-world patient scenarios for the training, education, and assessment of HCP .VP simulation makes learning flexible and convenient, as users choose the time and space where training occurs. It can also reduce the use of costly resources, and be disseminated to a broad range of HCP in various settings across far ranging geographical areas [10, 11]. Simulation gives nurses the opportunity to practice with a VP in a safe learning environment where they can make mistakes without causing real patients any harm .
Use of VP simulation in health professional training has grown exponentially in the past decade, as many literature reviews in the field have shown [13,14,15,16,17,18]. Medical education is the most targeted context in these reviews, followed by nursing. When the discipline of nursing is explored, the studies included in the reviews have mainly focused on undergraduate nursing students; the uptake of VP simulation for practicing nurses remains low. Evidence from a meta-analysis show that, compared to traditional methods in pre- and post-registration health-professionals’ education, VP simulation is more effective in improving skills (e.g. clinical reasoning, procedural skills, and mix of procedural and team skills) than knowledge . The use of VP simulation to improve relational skills is documented among students and other healthcare providers [13, 18], but it is less common among graduate nurses in a CE context . Simulation-based learning experience showed a smaller effect size of simulation-based research among staff nurses than among nursing students . One possible explanation of this difference lies in the challenges that nurses experience in the clinical setting (e.g. supplies, time, costs) that could hinder simulation-based research in practice and, consequently, reduce its potential effectiveness. More research is needed to assess the potential of VP simulation among staff nurses in improving their communication skills.
Aims and objectives
This study aimed to quantitatively and qualitatively assess the acceptability of a VP simulation to improve nurses’ relational skills in a CE context. This study is part of a doctoral dissertation . Acceptability is defined as “a multi-faceted construct that reflects the extent to which people delivering or receiving a healthcare intervention consider it to be appropriate, based on anticipated or experienced cognitive and emotional responses to the intervention” .
Our specific research objectives were: a) to measure the extent of the VP simulation nurses’ perceived acceptability in regards of the simulation design elements, of the global system quality and the technology acceptance, of the role simulation plays in supporting nurses’ professional practice, and of the achievement of learning objectives (quantitative objective); b) to explore nurses’ learning experience (qualitative objective); c) to deepen understanding of how the VP simulation contributed to nurses’ uptake of relational skills, to overall learning and its transfer into practice (mixed methods objective).
To fill the gap in this type of educational intervention available for nurses and to address the communication roadblock , we developed a VP simulation  aimed to improve nurses’ relational skills. The VP simulation was informed by a strengths-based nursing approach , motivational interviewing (MI) , and adult learning theories . The clinical case scenario is the story of Mr. Wilson, an HIV-positive man having difficulty taking his antiretroviral therapy. The clinical content is informed by MI : MI is one technique HCP can use to ensure the effective relational skills they need to open up conversation with patients about behavioural changes. MI is defined as a person-centered, collaborative, and guiding communication style (rather than a directive one) that elicits people’s motivation and commitment to change. The core of the simulation is then focused on applying relational skills consistent with MI (e.g., asking open-ended questions, using reflective listening), rather than acquiring HIV-specific knowledge. The screen-based simulation allows users to interact with a two-dimensional animated VP character (see Fig. 1). The simulation includes a prebriefing video and text, an electronic patient record, a glossary, and a simulated nurse-patient consultation. The narrative-simulated consultation encompasses quizzes and feedback loops, in which the learners’ choices and decisions can influence the VP’s speech. Green and red labels were also used as visual and theoretical cues to qualify the nurse-patient dialogue and acted as feedback. Key elements of the simulation are described elsewhere  and are summarized in Additional file 1 in the CONSORT-EHEALTH .
We conducted a convergent mixed methods study (Fig. 2). This mixed methods design is defined as the collection and analysis of quantitative and qualitative data integrated for results comparison and complementarity [26, 27]. Both types of data were collected separately and analyzed independently. A comparison strategy was then used to combine quantitative and qualitative results [28, 29], to interpret how the merged results agreed (correspondence, similarities), offered complementary information, or contradicted (disagreement or dissonance) . First, we carried out a pre-experimental study with a one-group post-test design  to measure nurses’ perceptions of the VP simulation. Second, we used a qualitative exploratory design  to describe nurses’ learning experience and to further nuance and deepen our understanding of the acceptability of the intervention by using complementary topics that were not covered by the quantitative component. The purpose of the integration of quantitative and qualitative findings was to compare and contrast both components to provide a comprehensive picture of the VP simulation’s contribution to nurses’ learning. The samples from the quantitative and qualitative components were interdependent as participants were required to complete the VP simulation and the post-test survey before being invited to take part in the focus group. Each method (quantitative and qualitative) aimed to answer complementary research questions. Some data collection questions targeted the same broad content while using different wordings: the role of the simulation to support nurses’ professional practice (quantitative component) and the concrete implications of having participated in the simulation for nursing practice (qualitative component). Other data collection questions were different such as the acceptance of the technology (quantitative component) and the motivation in participating in the whole study (qualitative component). One qualitative question was formulated from one quantitative statement to get a deeper understanding and explain what was meant for nurses to have gotten a useful learning experience helped to the VP simulation. During the integration phase of quantitative and qualitative findings, we identified content areas represented in both data sets  gained from survey and focus group - in order to compare and contrast them.
Sampling and recruitment
We used convenience and snowball sampling approaches to select registered nurses working in Quebec (Canada) who self-reported having basic computer literacy skills. Convenience sampling is advantageous in terms of affordability and of the immediate availability of the participants. Given the engagement required when participating in an online study, we were looking for nurses who had time and were motivated to complete the whole research process. Nurses were also asked to share the information with their colleagues (snowballing). Snowballing sampling is considered an effective and efficient approach to building a sample through the Internet . We used in-person and online recruitment strategies. We distributed leaflets at a conference involving nurses interested in HIV care and at clinical settings. Information was also communicated via online professional newsletters and e-mail to members of the Quebec order of nurses and of the HIV mentoring program . Table 1 contains a summary describing the nurses’ journey in the research process.
We collected quantitative data with LimeSurvey (LimeSurvey Project Team / Carsten Schmitz, 2020) with online post-test survey totalizing 80 closed questions on: 1) VP simulation design elements; 2) global system quality and technology acceptance; 3) role of simulation in supporting nurses’ professional practice; 4) achievement of learning objectives. We used a 4-points Likert scale (1 = Strongly disagree; 2 = Disagree; 3 = Agree; 4 = Strongly agree). We developed the majority of the questionnaire for this study (Additional file 2). Except items from the Technology Acceptance Model , which have been used almost entirely in their original form, all other items were inspired from existing tools that we modified to fit our simulation design elements, modality, content, role and learning objectives.
Virtual patient simulation design scale
This scale development was informed by Simulation Design Scale  to measure nurses’ perceptions of the simulation design elements (23 items): 1) context of the VP simulation / prebriefing (3 items); 2) glossary (4 items); 3) electronic patient record (1 item); 4) quizzes, feedback, and labels (11 items); 5) fidelity (4 items). Fidelity is the extent to which the VP simulation approaches reality .
Global system quality and technology acceptance
We slightly adapted the French-language version  of the Technology Acceptance Model  to measure these two main dimensions (total: 27 items): 1) global system quality and 2) technology acceptance. Global system quality is divided into the following constructs: system quality (5 items); information quality (3 items); service quality (3 items); and, user interface design quality (3 items). Technology acceptance includes: perceived usefulness (3 items); perceived ease of use (3 items); perceived enjoyment (3 items); and, intention to use (4 items). The higher the score (range of averages: 1–4), the greater the overall acceptance. The reliability of the original instrument  is demonstrated by a Cronbach alpha of 0.70 to 0.96 while our adapted version of the scale showed these psychometric properties: 0.68 to 1.00.
The simulation’s role in supporting nurses’ professional practice
Inspired by a validated French translation  of the Role of Simulation in Nurse Education Questionnaire  targeting the clinical preparation of students graduating from nursing programs, we developed a 22 item-tool. It evaluated the actual and anticipated impact of simulation on the nurse-patient relationship and on nurses’ communication skills, learning, and confidence in their ability to transfer the relational skills into practice.
Achievement of the learning objectives
This tool was developed to assess nurses’ agreement with the achievement of learning objectives (8 items) following their participation in the simulation.
Other questions, measures, and data
Open-ended questions were asked in the post-test survey to gain complementary insights about: a) the VP simulation design elements (e.g. comments about the electronic patient record, fidelity); b) the achievement of additional learning objectives; c) the most and least appreciated elements of the VP simulation; d) recommendations to improve the simulation.
The pre-intervention questionnaire included information on nurses’ sociodemographic characteristics, such as age group, gender, education level, workplace setting, previous MI training, and computer literacy skills.
A total of 30 participants was targeted to take part in the simulation and fill out the post-test survey. This sample size was determined according to recommendations for pilot studies [40, 41], considering that acceptability is often an element that is assessed in these type of studies .
Quantitative data analysis
A descriptive statistical analysis was conducted using Microsoft Excel version 2013 (Microsoft Corporation). Means (m), standard deviations (SD), median (med), and interquartile range (IQR) were calculated for continuous variables as well as for counts and percentages for the categorical variables. Fisher’s exact test was performed to compare the proportions of participants’ characteristics at baseline between completers and non-completers. This statistical test is justified when the sample size is small .
The focus group used a semi-structured conversational approach. It aimed to describe in greater detail and further nuance participants’ experience of the VP simulation as well as deepen our understanding of particular quantitative items, such as the utility of the VP simulation. The questions used to guide the focus group (Table 2) covered these topics: motivations, perceived difficulties for study participation, and concrete implications for nursing practice. The data collection was performed online through synchronous interactions using the Zoom videoconferencing platform (Zoom Video Communications Inc., 2016). The discussion was recorded after receiving participants’ consent.
Qualitative data analysis
The focus group recording was transcribed verbatim. Qualitative data analysis followed an inductive and iterative process. We thematically analyzed narratives from the focus group [44, 45]. Coding was led by GR and involved comparison across transcripts. The team members were involved in discussions of preliminary thematic findings and throughout the data interpretation process. NVivo Software Pro 12 (QSR International Pty Ltd., 2018) was used to facilitate data management and organization.
Credibility as a source of trustworthiness [46, 47] was ensured through prolonged engagement and observation. GR was immersed in the data by being directly involve in participant recruitment, data collection (leading the focus group) and analysis. GR listened the audio recording and transcribed the verbatim. She encouraged participants to express themselves freely, by sharing their true impressions of the VP simulation and not what they thought the student-researcher wanted to hear (in order to reduce the desirability bias). Peer debriefing was another way of ensuring the credibility, by discussing the preliminary qualitative findings as well as the mixed method interpretation findings with the team having expertise in qualitative and mixed methods research. A detailed and accurate descriptions of the sociodemographic characteristics of the nurses, the recruitment strategies and the settings was provided as a way of enhancing the transferability of the findings to other similar population and contexts.
Mixed methods integration
Once quantitative and qualitative data were collected and analyzed separately, both components were integrated using a comparison of results strategy . We first used a weaving technique, inspired by Fetter et al. [49, 50], that aims to narratively group both quantitative and qualitative findings under a mixed methods interpretation. For this interpretation process, we utilized the four stages of the pillar integration process (Fig. 3)  to visually compare quantitative and qualitative components and integrate them into a joint display (Additional file 3).
We reported the quantitative component, including the simulation description, with CONSORT EHEALTH  (Additional file 1). We used CHERRIES for Internet e-surveys  (Additional file 4); COREQ for the qualitative component  (Additional file 5), and GRAMMS for the mixed methods study  (Additional file 6).
Participants’ flow chart and characteristics
The participant flow chart of completers (n = 27) and non-completers (n = 22) is presented in Fig. 4.
Most of the completers held a bachelor’s degree. They had been working as nurses for an average of 18 years. Eighteen nurses (66.67%) had experience as HIV nurses. Eight nurses (30%) had previous MI training. Majority of participants (25/27, 93%) reported being confident in their computer skills.
From these 27 completers, five nurses took part in the qualitative component: two men and three women. They worked with different clienteles, including people living with HIV (PLHIV). Four nurses were trained in MI. Additional completers and non-completers’ characteristics are presented in Table 3.
All participants’ recruitment strategies
Participants were recruited in person (28/49, 57.14%), i.e. by being informed by a colleague or by the student-researcher; by e-mails sent by the Quebec order of nurses (11/49, 20%) and the HIV mentoring program (10/49, 20%).
Quantitative findings of completers
The detailed quantitative findings are presented in the additional files: the simulation design elements (Additional file 7), the global system quality and technology acceptance (Additional file 8), the role of the simulation (Additional file 9), and the learning objectives achievement (Additional file 10). Highlights are presented in each subsection.
Simulation design elements
A great majority (93%) of participants watched the video content and 78% read the corresponding text on the context of the simulation. Most of the participants (89%) felt that, to understand this context, it was key to have access to both text and video.
All participants agreed that the labels constructively supported their learning. Some 96% found that these cues were key to qualifying the content of the nurse-patient dialogue. All participants agreed that quizzes made them reflect on their nursing practice and they saw themselves in the quiz answers.
Almost all participants (96%) agreed that the feedback was provided in a timely manner (i.e. as the consultation progressed). All participants agreed that the feedback allowed them to make connections between the simulated situations and the theoretical elements of MI.
A majority of participants agreed with the simulation’s fidelity: the patient’s story (96%), the HIV-positive man’s appearance (96%), the nurse-patient interactions (93%), and the nurse’s office (85%) were all perceived as authentic.
Global system quality and technology acceptance
The mean score was rated a 3.65 (±0.48) for the service quality construct among participants who used the VP simulation support services (11/27). The interface design quality (3.54 ± 0.55) was the second construct with the highest score of the global system quality dimension, followed by system quality (3.51 ± 0.54) and by information quality (3.49 ± 0.50).
Participants had a good intention to use (3.53 ± 0.60) the VP simulation. Nurses perceived enjoyment (3.47 ± 0.57) and an ease of use (3.42 ± 0.67) with the simulation. The lowest mean score of the technology acceptance dimension was the perceived usefulness (3.35 ± 0.71), which is, above all, highly acceptable.
The role of simulation in supporting nurses’ professional practice
The items with the highest scores were: simulation led nurses to reflect on their practice in general, not just with PLHIV (3.58 ± 0.58); the content will lead nurses to improve their communication skills with clienteles other than PLHIV (3.50 ± 0.51), the health of PLHIV (3.50 ± 0.51), and the quality of therapeutic relationships with PLHIV (3.50 ± 0.51).
Achievement of learning objectives
Scores on the achievement of objectives ranged from 3.35 to 3.58, indicating a favourable assessment by participants. These two learning objectives had the highest scores: identification of traps within nursing interventions that can shut down communication with the patient (3.58 ± 0.50), and those that can optimize openness to the patient’s experience (3.54 ± 0.51).
Four main themes are presented: 1) Motivations to engage in the simulation-based research; 2) Learning in a realistic, immersive, and non-judgmental environment; 3) Perceived utility of the simulation; and, 4) Perceived difficulty in engaging in the simulation-based research.
Motivations to engage in the simulation-based research
Participants identified several reasons for taking part in the simulation-based research. First, the simulation offered accreditation and was free of charge, which were appealing incentives. Second, nurses reported that their interest and curiosity had been stirred by the learning modality, which was perceived as innovative, stimulating, and interactive, and by the way MI could be transposed into technology:
I was curious to see this new training modality because I have already followed MI training, and sometimes we’d practice with a coworker. I was curious to see how far we could get with the simulation. (Female nurse-manager)
Nurses perceived that the simulation could be applicable and coherent in their own practice with different clienteles (e.g. youth, people with hepatitis C), and, more broadly, to a variety of contexts:
[The simulation] was addressing the issue of adherence to HIV treatment and I felt that [the topic] fit in well with my practice. (Male assistant head nurse)
I thought [the simulation] was something that was interesting and not just about HIV [...] it was something that could be transferred to other areas of activity. (Female school nurse)
Finally, the desire to learn new knowledge or strengthen existing knowledge about MI and HIV were factors motivating nurses’ participation.
I found it important to do this training to learn things about HIV but also about motivational interviewing, which we do daily, enormously, at our office. (Female school nurse)
Learning in a realistic, immersive, and non-judgmental environment
Two nurses who were experienced in providing HIV care reported the VP’s story to be an uncommon one for non-adherence, but felt that it was nonetheless credible and realistic. What they felt to be most important was the nurse-patient interaction, which allowed to immerse themselves in the simulation:
Maybe this is because I’ve done a lot of work around the issue of taking antiretroviral treatment, so I found the [VP’s] situation ... maybe less typical... At the same time, I realized that it was not necessarily very important. Eventually, you forget about the situation, you know, because [the learning activity] is more about how to react to interactions with the patient [...] I was more focused on what he was saying than the image. I think it’s a really strong point of [the learning activity] that we got really into it. (Female nurse-researcher)
One nurse’s first impression was the VP’s resemblance to a puppet, which lead him to wonder about the seriousness of the learning activity. The patient’s appearance could have caused this participant to lose interest in the learning experience, but eventually this image of the VP gave way to a more human and realistic impression:
At first, I thought [the VP] looked like a puppet [...] I kind of wondered if [the simulation] was for real. I don’t really want to question its seriousness … Beyond the caricature, I could see the patient asking himself questions; he was squinting a little. Human beings do that. They’re not puppets [...] And as I went along doing the interview, I saw there was communication between the nurse and the patient. And [my impression] faded away. (Male nurse case-manager)
Two participants compared the simulation to physical presence-based group learning, where MI must be practiced through role-playing with a coworker. The simulation was seen as an advantageous way to reproduce a real interaction with a VP, reducing the discomfort and bias of practicing with someone, and fostering the learning progress:
In classic training activities, we practice with a coworker. I find that quite biased because we’ve both just learned the theory; we try to apply it; the other person has just learned the same thing so, in the end, well, we help each other only a little bit. But here, we were faced with a virtual character who is very realistic. I find it even more real than with, shall we say, another trainee. But for people who are shy in groups, [the simulation] is really very accessible and allows them to progress. (Male assistant head nurse)
Compared to group training activities, the simulation provides freedom while targeting individual learning and performance:
I think that doing it one by one, well, alone, allows something that is not necessarily possible in a group training activity. It’s even more in-tune with what you would actually do. There is no judgment. There are no right or wrong answers. [The simulation] allows you to answer more freely. (Female nurse-researcher)
Finally, this participant summed up her experience: “I feel like I got real practice.” (Female school nurse).
Perceived utility of the virtual patient simulation
We identified three sub-themes as part of this theme: developing reflective learning and transferring it to practice, being present and revisiting relational skills, and acquiring and consolidating motivational interviewing knowledge and skills.
Developing reflective learning and transferring it to practice
All the nurses mentioned the simulation’s capacity to promote mistake-based learning through quizzes and feedback loops:
It was fun because it’s like action/reaction. It was immediately obvious if you asked the question wrong, you could see the effect. I found it interesting because if you took a wrong action, you could get back on track. That way, we could understand why it was a mistake. (Female nurse-manager)
This participant, who did the entire virtual simulation twice, reported a progression of his learning, building on the mistakes he had made:
The first time, I made a lot of mistakes because I told myself that I was going to go with my knowledge and experience. The second time, I did it with my new knowledge. It gives you parallel vantage point onto yourself, onto your own beliefs. (Male nurse case-manager)
The simulation thus allowed participants to reflect and take a critical look at themselves and their practice, becoming aware of past mistakes and the impact of their interventions on their relationship and interactions with patients:
You’re never neutral in a MI. Yes, you’re the care provider, but you’re a person. It can set certain limits or can even make you get stuck in it. [The simulation] makes you aware of who you are through all this. (Male nurse case-manager).
Look, if patients don’t react or aren’t motivated, well, maybe it’s because I too am playing a part as the care provider: maybe I am not addressing them in the right way, maybe I am not considering them in their entirety, according to their beliefs and values. (Female nurse-manager)
The interactivity inherent to the simulation supports this reflexive process, which in turn can lead to transferring learning to real practice, and thus improve it:
When you’re one-on-one [with a young person], sometimes you’ll answer off the cuff because you’re in a hurry. If you’ve practiced [the situation] in simulation, you’re going to know that whatever you said was not so great, you know, you’re going to question yourself. So, you’re going to be more careful when a similar situation occurs in reality [...] I’m going to try saying it differently to help the person get a little further. It makes you better. (Female school nurse)
This participant questioned his past interventions, in which he hastily presumed the cause of non-adherence (e.g. relapse, substance abuse) when interacting with his clientele. After participating in the simulation, this nurse stated his intention of changing his way of intervening so that he better understands the patient’s situation, before drawing conclusions:
Do I go too fast sometimes? Telling myself that, well, he didn’t take it [his treatment], that he must have relapsed, always jumping to my conclusions first. Don’t I miss things sometimes, too? I was thinking that maybe now I will be more careful and try to understand the patient’s reasons and stop just saying ‘Ah, well, he didn’t take it.’ (Male assistant head nurse)
Being present and revisiting relational skills
The simulation helped to underscore the importance of listening to patients. This meant being present, available during the consultation and living in the “here and now”:
It helps nurses understand or realize that it’s important to listen, to be there in the here and now. More and more, we have our electronic medical records, we write in the record and don’t even look at the patient. We no longer take the time to actually look at the patient because we are so busy on our computer... It’s really worth it to sit down and look at the patient and just be present with them. (Female nurse-manager)
The simulation had a positive influence on revisiting ways of communicating and asking patients the right questions to support them in reflecting and identifying their own solutions:
I’d say it’s more in the way the questions are asked. It’s really focused on open-ended questions, and solutions that come from the patient. We [nurses] may have solutions, but they have to come from them [the patients], and that’s when they are most effective [...] How can we ask questions that bring out the best in the patient? (Male nurse case-manager)
The simulation alerted the nurses and raised their awareness of how they relate to patients, creating optimal conditions for successful relational practice and mobilizing communication skills that allow patients to express themselves and, especially, to find their own solutions.
Acquiring and consolidating motivational interviewing knowledge and skills
One participant with no prior MI training considered the simulation to be an effective and efficient way to achieve intensive learning:
I’d read a little about MI, but I’d never done any training. I didn’t expect to learn so much in such a short time. (Female nurse-researcher)
Moreover, for another participant, who had received training in MI and who does not practice directly with patients, the key lays in putting theoretical elements into action with the VP. Consequently, the simulation-facilitated practice helped reinforce her knowledge and feelings of competence in applying MI:
I had already had some MI training. [The simulation] reassured me a bit that, actually, I was competent and that I would have been good, face-to-face, with a patient. So, it just confirmed this for me. Because there’s always a doubt about MI being this huge thing. But in the end, you know, we just lack practice. And I found that the platform meant that I was able to strengthen my nursing practice and my past theoretical learning, since I don’t see patients every day. (Female nurse-manager)
For the other three participants who had previous MI training, the simulation helped them better understand the theory and refresh their knowledge, as well as learn how to better apply it. Simulation as a learning modality thus seemed to benefit nurses with various levels of MI training and knowledge.
Perceived difficulty in engaging in the simulation-based research
We asked participants in the focus group to reflect on the difficulties they experienced in completing the study, or those they heard their coworkers mention. Technical difficulties were noted as one of the main potential explanations of some participants’ withdrawal, either because of the complexity of creating an account, the delay between the characters’ words and movements, or the system’s slowness. Individual perseverance became important in this context:
I’m not saying the workflow was slow... but maybe that’s why some people didn’t finish the training activity. I’m not saying it was repetitive, but maybe if they feel it was too slow... When the patient talks, he moves his arms around, and sometimes there was a little delay. This was maybe a feeling I had, since I was persistent at first. (Female nurse-manager)
One participant did not like the simulation’s lack of progress indicators, which she felt might also have discouraged others. Individual and time-related elements were another hypothesis for some participants’ withdrawal:
Perhaps a lack of time or a drop in motivation along the way. When I start something, I like to finish it. So maybe it’s question of personality, too. (Female school nurse)
Integration of quantitative and qualitative findings into mixed method interpretations
Using the pillar integration process (see Additional file 3), we have combined both quantitative and qualitative data and categories (findings). The operationalization of this process as well as an excerpt of the joint display that provides a side-by-side comparison of both types of data and categories is presented in Additional file 11. Examples are provided to support the identification of the four mixed method interpretation findings resulting from our analysis: 1) Influence of the simulation’s fidelity on nurses’ impression of getting a real practice and of having an immersive learning experience; 2) Simulation’s perceived flexibility, efficacy, and control over one’s learning led to a positive learning experience; 3) Taping self-awareness and reflection in relational practice, 4) Acquiring new knowledge and building self-confidence. These findings describe how the VP simulation quality, its element designs and its role contributed to nurses’ learning experience.
Influence of the simulation’s fidelity on nurses’ impression of getting a real practice and of having an immersive learning experience
While various simulation design elements were assessed quantitatively as being realistic, the qualitative results provide insight into how fidelity contributed to nurses’ immersion in their learning experience, among other gains. The quantitative and qualitative results are therefore complementary. Participants were able to overcome the VP’s appearance and become immersed in the scenario to focus on the nurse-patient interactions. They also felt the simulation gave them an opportunity for real practice.
Simulation’s perceived flexibility, efficacy, and control over one’s learning led to a positive learning experience
As described in Additional file 8, global system quality and technology acceptance were rated with high scores. VP simulation offers flexibility for when and where learning occurs, it gives users control over their learning, and it was generally perceived as more effective than other types of training. These aspects all positively influenced participants’ learning experience. The qualitative findings supported the quantitative results. All participants in the focus group appreciated being able to use the simulation during or outside of work hours and even from home. The flexibility of the learning modality allowed them to consult the simulation more than once. Compared to face-to-face training that requires trainees to practice with a colleague, the simulation gave them practice with the VP that was both more realistic and less intimidating. This modality therefore allows users to express a sincere response, without fear of making a mistake in front of a group. Simulation also facilitates the evaluation of individual knowledge and performance, rather than collective ones.
Taping self-awareness and reflection in relational practice
The quantitative results indicate that the high scores in favour of the role of simulation, quizzes, and feedback prompted the participants to reflect on their nursing practice, make connections between theory and practice, learn from mistakes, and raise their awareness of elements that can facilitate or hinder therapeutic relationships with patients. The qualitative results also enriched the quantitative results when nurses gave concrete examples of their own communication styles that had been less effective in the past (e.g. leading the consultation, making recommendations to the patient without asking permission, jumping to conclusions too quickly) and that could be improved. The nurses said that practicing with the VP and getting synchronous feedback that mirrors their actual practice would help them avoid replicating ineffective patterns. The simulation therefore contributed to educating and raising awareness of self, as nurse, and of others (i.e. patients), and underscored the importance of nurses’ presence, open-mindedness, availability, and good listening.
Acquiring new knowledge and building self-confidence
By assessing the role of simulation in supporting nursing practice, participants reported having learned something new. They also expressed having built self-confidence. Indeed, they felt capable of applying communication skills and of facing similar situations with PLHIV and other clienteles in the future. The qualitative results reinforce these findings, reflecting the simulation’s influence on nurses feeling better prepared and equipped to apply MI with their clienteles, to consolidate their practice, and thus to reinforce their sense of confidence and competence.
Statement of main findings
Overall, nurses perceived that VP simulation is highly acceptable, if we consider that the great majority of means were above 3, on the 4-point Likert scale. The quantitative results were highly consensual in favour of simulation design elements, global system quality and technology acceptance, the simulation’s role in supporting nursing practice, and the achievement of learning objectives. The qualitative results nuanced, deepened, and even added new elements (e.g. motivation to participate and difficulties encountered) to the quantitative results. The integration of quantitative and qualitative findings drew a full portrait of the continuum of the nurses’ simulation-based experience, the VP elements that contributed to their immersive learning, and its potential transfer to their practice. Four mixed method interpretations were described: 1) Influence of the simulation’s fidelity on nurses’ impression of getting a real practice and having an immersive learning experience; 2) Simulation’s perceived flexibility, efficacy, and control over one’s learning led to a positive learning experience; 3) Taping self-awareness and reflection in relational practice; 4) Acquiring new knowledge and building self-confidence.
Comparison with existing literature
The qualitative theme motivations to engage in the simulation-based research brings a new element to the quantitative results, which could not capture this perspective. We created and reinforced what Moore et al.  call a teachable moment in order to influence the enrollment, participation, and engagement of nurses in this learning activity. To do so, we shared information about the project via different channels, with a view to reaching a variety of nurse profiles. The majority of nurses who completed the VP simulation was recruited through in-person strategy (16/27). The social influences, such as recommendations by colleagues, are recognized as facilitators to the self-directed learning of physicians in CE, alongside with affective attitude (e.g. professional interest, motivation, willingness) and training accreditation . Learners’ characteristics/qualities, like perseverance and the desire to finish what has been started, are also considered to be elements that can affect both learning and the motivation to engage in the simulation [36, 56].
VP simulation led to immersive, realistic, active, and constructive learning experiences. Prebriefing was planned in the VP simulation codevelopment process  and is considered a best practice [57, 58]. Prebriefing is known as a facilitation method and a preparatory activity to ready learners for the simulation-based experience. The red and green labels were cues that served as feedback, and were also used as a facilitation method to orient learners through the simulation .
Feedback is considered to be the most important feature of effective learning [16, 59]. Indeed, it can support learners’ self-assessment of their skills and allow the progression, development, and maintenance of those competencies . In the quantitative results, participants reported appreciating the timing of the synchronous feedback. The timing of feedback can indeed play a role in learning, as can its source (how it is provided, by whom) and type (i.e. outcome or process-based). One other meaningful feature of the simulation is the opportunity of deliberate and repetitive practice, which can impact learning [59, 60], unlike other educational interventions, such as conferences or lectures in which learners are often passive recipients of “inert” information .
In light of our results and when comparing with the literature, it is reasonable to believe that different modes of fidelity, be it physical, conceptual, emotional/experiential [62, 63], not only affected participants’ learning, but also their engagement, overall experience, and immersion in the simulation [59, 62,63,64]. Physical fidelity refers notably to the VP’s appearance and the nurse’s virtual office. In the VP simulation, conceptual fidelity is illustrated by the if/then concept [62, 63]: if the learner adopts a relational skill consistent with MI (e.g. a guiding style of counseling vs a directing one), then it will open the dialogue with the patient. One participant clearly expressed this mode of fidelity by the “action/reaction principle.” Finally, participants in the focus group reported the sense of having truly practiced; they found that simulation was an effective learning approach, reflecting the emotional and experiential mode of fidelity . This mode refers to the learner’s emotions, feelings, and beliefs relating to their entire experience of participating in the simulation [62, 63].
It would seem that the benefits of the simulation extend beyond the learning of the recommended communication techniques to encompass a relational component: being present, taking the time to listen carefully and understanding patients without falling prematurely into professional preconceptions. This relational component is aligned with partnership, one of the vital aspects of MI  that has to do with seeing the world through the patient’s eyes without imposing the nurse’s view. Another important finding of this study is the reflective learning developed during nurses’ simulation-based experience. Reflection was indeed central to the learning experience because it played a double role, as both a reflective methodology (i.e. synchronous feedback used to promote reflection and to learn from one’s mistakes) and a learning outcome  (i.e. VP simulation allowed nurses to develop reflective learning). Reflection is considered to be a means of supporting professional development  and has the potential to transform experience into new learning . A meaningful finding of our study is that nurses raised their awareness of themselves, and of others (i.e. patients), by acknowledging how some elements (such as their own communication styles) could influence positively or negatively therapeutic relationships with patients. Perceptual skills have been reported as a communication skills learning outcome in a qualitative synthesis (n = 168) targeting health professions workers . Perceptual skills (self-awareness, awareness of others and context) accounted for 9% of learning outcomes throughout the whole papers and were considered of importance in reflective, self-regulating health providers. Bennett-Levy  sheds light that interpersonal perceptual skills are fundamental to effective therapeutic practice, and include these three attributes: empathy, mindfulness and reflection-in-action. By bringing into the table these notions of reflection and perceptual skills, in agreement with Denniston et al. , we argue that relational skills are complex. Training interventions aimed to address relational skills, including communication between healthcare providers and patients, should consider an array of relational skills learning outcomes in their evaluation design.
Our findings also corroborate those of a qualitative studies [11, 68] and those reported in an integrative review (n = 38 articles)  that explored how VP simulation influenced the non-technical skills (such as communication) of undergraduate nursing students  and undergraduate health professional education . The findings suggest that students acknowledged the importance of communication and listening to their patients. The VP “opened their eyes” to the impacts either effective or poor communication had on healthcare . VP simulation exerted a positive influence on students, by reinforcing or teaching new communication skills, providing opportunities for practicing those skills and for building their confidence in applying them, developing specific verbal and nonverbal communication skills  and developing awareness of non-technical skills including but not limited to communication .
Strengths and limitations
As far as we are aware, this is the first study to examine the acceptability of a VP simulation informed by MI to improve nurses’ relational skills in a CE context. This mixed methods study led to a gain in complementary and rich data, providing a comprehensive picture of nurses’ learning experience. However, two main challenges were encountered during the integration. The first was dealing with huge numbers of items and data in the quantitative survey (n = 80 items) and comparing these to the qualitative data that was focused on circumscribed thematic. The second was the unequal sample size (27 in the quantitative component; and 5 in the qualitative part). Use of the joint display was helpful in putting together data and results that can be compared and discussed. Limitations of one method associated with the present of the other method are described in further details in Additional file 6.
The convenience sampling and snowball approaches used in both quantitative and qualitative components allowed us to recruit nurses with various profiles who work in different settings, thus adding to the richness of the findings. However, those sampling approaches have their own limitations. The findings may have been tainted with participation bias, given our sample had to complete 100% of the simulation and the post-test survey. It means that only the volunteers and the most “motivated” and willing nurses participated. This could explain the consensual findings in favour of the VP simulation. It would have been useful to also explore the reasons why nurses abandoned the simulation; but this was unfortunately not possible. The nonrepresentativeness of this captive sample of nurses may have limited the transferability of the findings to other contexts and population . In addition, the five-participant sample size was relatively small (focus group).
An important limitation of our study is that we did not follow the best practices for scale development and validation, such as those suggested by Boateng et al. . This is consistent with the exploratory lens of our study. With the exception of the Global System Quality and Technology Acceptance instrument , the other tools were not validated. However, to ensure content clarity and ease of navigation, we conducted two rounds of survey pre-tests with potential end users before launching the study.
Implications for research
The variety of the nurses’ profiles provided insight into the transferability of the VP simulation beyond the field of HIV care. In the post-test survey, nurses recommended the simulation to other healthcare professionals. Further research could focus on simulation acceptability for a variety of providers. Subsequent research would be necessary to explore the influence of contextual enablers/barriers (e.g. resources, structure, organizational culture) and those that are related to healthcare professionals themselves (e.g. role, work structure, habits, competing demands) when using VP simulation to apply the resulting MI-inspired relational skills. Considering that reflective learning was an important finding, this aspect could be deepened by exploring the underlying causal mechanisms that lead users to improve their relational skills and to put these latter into practice. Future work could be guided by these research questions: What are the contexts and mechanisms that allow healthcare professionals to integrate MI-consistent relational skills into their professional practice? How does the VP simulation produce different outcomes? Additional research could be conducted to examine how simulation-based intervention change nurses’ practice and how practice-change behaviour translates into patient outcomes (e.g. comfort and quality of life, empowerment, medication adherence).
Relational skills are fundamental to high-quality nursing care. Findings from this mixed methods study provided critical insight into nurses’ perception of the simulation’s high acceptability. It holds potential to change practice, as nurses become more self-reflective and aware of the impact of their relational skills on patients. VP simulation particularly contributed to knowledge development on MI, on how self-confidence in applying relational skills can be increased by practicing with the VP. Nurses’ participation in the simulation contributed to immersive, positive, and constructive learning experiences. The study highlights the value and novelty of VP simulation for CE in nursing.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
CHEcklist for Reporting Results of Internet E-Surveys
Canadian Institutes of Health Research
Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth
Consolidated Criteria for Reporting Qualitative Studies
Good Reporting of A Mixed Methods Study
International Nursing Association for Clinical Simulation and Learning
People living with HIV
Denniston C, Molloy E, Nestel D, Woodward-Kron R, Keating JL. Learning outcomes for communication skills across the health professions: a systematic literature review and qualitative synthesis. BMJ Open. 2017;7(4):e014570. https://doi.org/10.1136/bmjopen-2016-014570.
Blackmore A, Kasfiki EV, Purva M. Simulation-based education to improve communication skills: a systematic review and identification of current best practice. BMJ Simulation Technol Enhanced Learning. 2018;4(4):159–64. https://doi.org/10.1136/bmjstel-2017-000220.
Foronda C, MacWilliams B, McArthur E. Interprofessional communication in healthcare: an integrative review. Nurse Educ Pract. 2016;19:36–40. https://doi.org/10.1016/j.nepr.2016.04.005.
The Joint Commission (2015). Sentinel Event Data Root Causes by Event Type 2004–2014 [http://www.jointcommission.org/assets/1/18/Root_Causes_by_Event_Type_2004-2014.pdf]. Accessed 26 October 2020.
Vermeir P, Vandijck D, Degroote S, Peleman R, Verhaeghe R, Mortier E, et al. Communication in healthcare: a narrative review of the literature and practical recommendations. Int J Clin Pract. 2015;69(11):1257–67. https://doi.org/10.1111/ijcp.12686.
Miller WR, Rollnick S. Motivational interviewing: helping people to change. 3rd ed. New York: The Guilford Press; 2013.
Rouleau G, Richard L, Côté J, Gagnon M-P, Pelletier J. Nursing practice to support people living with HIV with antiretroviral therapy adherence: a qualitative study. J Assoc Nurses AIDS Care. 2019;30(4):e20–37. https://doi.org/10.1097/JNC.0000000000000103.
Coventry TH, Maslin-Prothero SE, Smith G. Organizational impact of nurse supply and workload on nurses continuing professional development opportunities: an integrative review. J Adv Nurs. 2015;71(12):2715–27. https://doi.org/10.1111/jan.12724.
Ellaway R, Candler C, Greene P, Smothers V, & MedBiquitous. (2006). An architectural model for MedBiquitous Virtual Patient [http://groups.medbiq.org/medbiq/display/VPWG/MedBiquitous+Virtual+Patient+Architecture]. Accessed 14 May 2020.
Cant RP, Cooper SJ, Sussex R, Bogossian F. What’s in a name? Clarifying the nomenclature of virtual simulation. Clin Simul Nurs. 2019;27:26–30. https://doi.org/10.1016/j.ecns.2018.11.003.
Peddle M, Mckenna L, Bearman M, Nestel D. Development of non-technical skills through virtual patients for undergraduate nursing students: an exploratory study. Nurse Educ Today. 2019;73:94–101. https://doi.org/10.1016/j.nedt.2018.11.008.
Kaplonyi J, Bowles K-A, Nestel D, Kiegaldie D, Maloney S, Haines T, et al. Understanding the impact of simulated patients on health care learners’ communication skills: a systematic review. Med Educ. 2017;51(12):1209–19. https://doi.org/10.1111/medu.13387.
Bracq M-SM, Michinov E, Jannin P. Virtual reality simulation in nontechnical skills training for healthcare professionals: a systematic review. Simul Healthc. 2019;14(3):188–94. https://doi.org/10.1097/SIH.0000000000000347.
Cant RP, Cooper SJ. Simulation in the internet age: the place of web-based simulation in nursing education. An integrative review. Nurse Educ Today. 2014;34(12):1435–42. https://doi.org/10.1016/j.nedt.2014.08.001.
Consorti F, Mancuso R, Nocioni M, Piccolo A. Efficacy of virtual patients in medical education: a meta-analysis of randomized studies. Comput Educ. 2012;59(3):1001–8. https://doi.org/10.1016/j.compedu.2012.04.017.
Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and Meta-analysis. Acad Med. 2010;85(10):1589–602. https://doi.org/10.1097/ACM.0b013e3181edfe13.
Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N, et al. Virtual patient simulations in health professions education: systematic review and Meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(7):e14676. https://doi.org/10.2196/14676.
Peddle M, Bearman M, Nestel D. Virtual patients and nontechnical skills in undergraduate health professional education: an integrative review. Clin Simul Nurs. 2016;12(9):400–10. https://doi.org/10.1016/j.ecns.2016.04.004.
Cantrell MA, Franklin A, Leighton K, Carlson A. The evidence in simulation-based learning experiences in nursing education and practice: an umbrella review. Clin Simul Nurs. 2017;13(12):634–67. https://doi.org/10.1016/j.ecns.2017.08.004.
Rouleau G. [Development and evaluation of a virtual patient simulation aimed to improve nurses’ relational skills in a continuing education context] Développement et évaluation d’une simulation numérique visant à améliorer les habiletés relationnelles des infirmières dans un contexte de formation continue Québec. Canada: Université Laval; 2020.
Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res. 2017;17(1):88. https://doi.org/10.1186/s12913-017-2031-8.
Rouleau G, Pelletier J, Côté J, Gagnon M-P, Martel-Laferrière V, Lévesque R, et al. Codeveloping a virtual patient simulation to foster nurses’ relational skills consistent with motivational interviewing: A situation of antiretroviral therapy nonadherence. J Med Internet Res. 2020;22(7). https://doi.org/10.2196/18225.
Gottlieb L. Strengths-based nursing care: health and healing for person and family. New York: Springer Publishing Company; 2013.
Clapper TC. Beyond Knowles: what those conducting simulation need to know about adult learning theory. Clin Simul Nurs. 2010;6(1):e7–e14. https://doi.org/10.1016/j.ecns.2009.07.003.
Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and Mobile health interventions. J Med Internet Res. 2011;13(4):e126. https://doi.org/10.2196/jmir.1923.
Creswell JW, Creswell JD. Research design: qualitative, quantitative, and mixed methods approaches. 5th ed. Los Angeles: Sage Publications; 2018.
Fetters MD, Molina-Azorin J. Utilizing a mixed methods approach for conducting interventional evaluations. J Mix Methods Res. 2020;14(2):131–44. https://doi.org/10.1177/1558689820912856.
Johnson RE, Grove AL, Clarke A. Pillar integration process: a joint display technique to integrate data in mixed methods research. J Mix Methods Res. 2019;13(3):301–20. https://doi.org/10.1177/1558689817743108.
Pluye P, Kaur N, Granikov V, Garcia BE, Tang D. Mixing phases, results and data in patient oriented research. Int J Multiple Res Approaches. 2018;10(1):41–56. https://doi.org/10.29034/ijmra.v10n1a3.
Campbell C, Stanley J. Experimental and quasi-experimental designs for research. Chicago: Rand McNally; 1963.
Deslauriers J-P, Kérisit M. Le devis de recherche qualitative. [Qualitative research design.]. In: Poupart J, Deslauriers J-P, Groulx L-H, Mayer R, Pirès A, editors. La recherche qualitative: Enjeux épistémologiques et méthodologiques [Qualitative research: Epistemological and methodological issues]. Montréal: Gaëtan Morin; 1997. p. 85–111.
Patton MQ. Qualitative research & evaluation methods. 4th ed. Thousand Oaks: Sage Publications; 2015.
Le Programme national de mentorat sur le VIH et les hépatites. Le Programme [https://pnmvh.org/a-propos/le-programme/]. Accessed 5 May 2020.
SimforHealth (2020). MedicActiV - Virtual simulation platform for the training of health professionals [http://www.medicactiv.com/en/]. Accessed 23 May 2020.
Cheng Y-M. The effects of information systems quality on nurses’ acceptance of the electronic learning system. J Nurs Res. 2012;20(1):19–31. https://doi.org/10.1097/JNR.0b013e31824777aa.
Jeffries P. A framework for designing, implementing, and evaluating: simulations used as teaching strategies in nursing. Nurs Educ Perspect. 2005;26(2):96–103.
Fontaine G, Cossette S, Heppell S, Boyer L, Mailhot T, Simard M-J, et al. Evaluation of a web-based E-learning platform for brief motivational interviewing by nurses in cardiovascular care: a pilot study. J Med Internet Res. 2016;18(8):e224. https://doi.org/10.2196/jmir.6298.
Simoneau IL, Paquette C, Fortin F. Traduction et validation en langue française du McCaughey-Traynor Role of Simulation Questionnaire: une étude pilote. (Étude pilote PA2012–015-141EX). Sherbrooke: Cégep de Sherbrooke; 2012.
McCaughey CS, Traynor MK. The role of simulation in nurse education. Nurse Educ Today. 2010;30(8):827–32. https://doi.org/10.1016/j.nedt.2010.03.005.
Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 2008;31(2):180–91. https://doi.org/10.1002/nur.20247.
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10(1):1. https://doi.org/10.1186/1471-2288-10-1.
Sidani S, Braden CJ. Testing the Acceptability and Feasibility of Interventions. In: Sidani S, Braden CJ, editors. Design, Evaluation, and Translation of Nursing Interventions. John Wiley & Sons, Ltd; 2011. p. 163–96. https://doi.org/10.1002/9781118785553.ch12.
Kim H-Y. Statistical notes for clinical researchers: chi-squared test and Fisher's exact test. Restor Dent Endod. 2017;42(2):152–5. https://doi.org/10.5395/rde.2017.42.2.152.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa.
Paillé P, Mucchielli A. L'analyse qualitative en sciences humaines et sociales. 4th ed. Paris: Armand Colin; 2016. https://doi.org/10.3917/arco.paill.2016.01.
Cypress BS. Rigor or reliability and validity in qualitative research: perspectives, strategies, reconceptualization, and recommendations. Dimens Crit Care Nurs. 2017;36(4):253–63. https://doi.org/10.1097/DCC.0000000000000253.
Lincoln YS, Guba EG. Naturalistic inquiry. Thousand Oaks: Sage Publications; 1985.
Pluye P, Bengoechea EG, Granikov V, Kaur N, Tang DL. A world of possibilities in mixed methods: review of the combinations of strategies used to integrate qualitative and quantitative phases, Results and Data. IJMRA. 2018;10(1):41–56. https://doi.org/10.29034/ijmra.v10n1a3.
Fetters MD, Curry LA, Creswell JW. Achieving Integration in Mixed Methods Designs: Principles and Practices. Health Serv Res. 2013;48(6pt2):2134.
Fetters MD, Freshwater D. Publishing a methodological mixed methods research article. J Mix Methods Res. 2015;9(3):203–13. https://doi.org/10.1177/1558689815594687.
Eysenbach G. Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3). https://doi.org/10.2196/jmir.6.3.e34.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042.
O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. https://doi.org/10.1258/jhsrp.2007.007074.
Moore J, Donald CK, Sherman L, Pavan M. A conceptual framework for planning and assessing learning in continuing education activities designed for clinicians in one profession and/or clinical teams. Med Teach. 2018;40(9):1–10. https://doi.org/10.1080/0142159X.2018.1483578.
Jeong D, Presseau J, ElChamaa R, Naumann DN, Mascaro C, Luconi F, et al. Barriers and facilitators to self-directed learning in continuing professional development for physicians in Canada: a scoping review. Acad Med. 2018;93(8):1245–54. https://doi.org/10.1097/ACM.0000000000002237.
Billett S. Conceptualizing learning experiences: contributions and mediations of the social, personal, and brute. Mind Cult Act. 2009;16(1):32–47. https://doi.org/10.1080/10749030802477317.
International Nursing Association for Clinical Simulation and Learning Standards Committee. Standards of best practice: SimulationSM simulation design. Clin Simul Nurs. 2016;12:S5–S12. https://doi.org/10.1016/j.ecns.2016.09.005.
Page-Cutrara K. Use of Prebriefing in nursing simulation: a literature review. J Nurs Educ. 2014;53(3):136–41. https://doi.org/10.3928/01484834-20140211-07.
Issenberg SB, Mcgaghie WC, Petrusa ER, Gordon DL, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10–28. https://doi.org/10.1080/01421590500046924.
Chee J. Clinical simulation using deliberate practice in nursing education: a Wilsonian concept analysis. Nurse Educ Pract. 2014;14(3):247–52. https://doi.org/10.1016/j.nepr.2013.09.001.
Bennett-Levy J. Therapist skills: a cognitive model of their acquisition and refinement. Behav Cogn Psychother. 2006;34(1):57–78. https://doi.org/10.1017/S1352465805002420.
Dieckmann P, Gaba D, Rall M. Deepening the theoretical foundations of patient simulation as social practice. Simul Healthc. 2007;2(3):183–93. https://doi.org/10.1097/SIH.0b013e3180f637f5.
Rudolph JW, Simon R, Raemer DB. Which reality matters? Questions on the path to high engagement in healthcare simulation. Simul Healthc. 2007;2(3):161–3. https://doi.org/10.1097/SIH.0b013e31813d1035.
Chiniara G, Cole G, Brisbin K, Huffman D, Cragg B, Lamacchia M, et al. Simulation in healthcare: a taxonomy and a conceptual framework for instructional design and media selection. Med Teach. 2013;35(8):e1380–95. https://doi.org/10.3109/0142159X.2012.733451.
Moon J. Reflection in learning and professional development: theory and practice. London: Kogan Page; 2000.
Gustafsson C, Fagerberg I. Reflection, the way to professional development? J Clin Nurs. 2004;13(3):271–80. https://doi.org/10.1046/j.1365-2702.2003.00880.x.
Tremblay M-C, Richard L, Brousselle A, Beaudet N. Learning reflexively from a health promotion professional development program in Canada. Health Promot Int. 2014;29(3):538–48. https://doi.org/10.1093/heapro/dat062.
Peddle M. Participant perceptions of virtual simulation to develop non-technical skills in health professionals. J Res Nurs. 2019;24(3–4):167–80. https://doi.org/10.1177/1744987119835873.
Given LM. The sage encyclopedia of qualitative research methods. Thousand Oaks: Sage Publications; 2008. https://doi.org/10.4135/9781412963909.
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front Public Health. 2018;6(149). https://doi.org/10.3389/fpubh.2018.00149.
We are grateful to all the nurses who participated in the simulation based-research. We truly appreciated the Quebec HIV mentoring program’s support in disseminating information about the study. We are grateful to Confluence Communication for their participation in editing this manuscript, and to Marc-André Reid for his support in creating the post-test survey on LimeSurvey. The funding organizations did not influence the design, collection, analysis and interpretation of data, the writing of the paper, nor the decision to submit the paper for publication. Thank you to Jack Tchuente for having performed a part of the descriptive statistics. This paper is part of a thesis dissertation work led by GR under the supervision of MPG and JC.
The development of this simulation was funded by a student project grant from Quebec Network on Nursing Intervention Research, by a planning and dissemination grant from the Canadian Institutes of Health Research (CIHR) , and by a doctoral scholarship allocated to GR from the CIHR . GR received doctoral scholarships from the Quebec Network on Nursing Intervention Research, the Fonds de recherche du Québec santé, the CIHR, and Quebec’s Ministère de l’Éducation et de l’Enseignement supérieur.
Ethics approval and consent to participate
The project received ethics approval from the Institutional Review Board of the University of Montreal Hospital Center [#18.243]. A process for the evaluation of multicenter projects was undertaken to recruit nurses working in various settings [MP-02-2019-8082]. Participation in the study was voluntary. Two online consents were obtained prior to nurses (i.e. participants have to tick a box “I consent to participate): 1) answering the pre-intervention questionnaire; and 2) participating in the focus group. The study was a low risk ethics concern. The ethics committee approved this form of online consent.
Consent for publication
JC, the University of Montreal Hospital Research Centre and SimforHealth are linked by a partnership contract. SimforHealth owns the MedicActiv platform that supported the simulation and was involved throughout the co-development process. GR, JP, and JC developed the VP simulation.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
CONSORT-EHELTH_UNformat_citations - Reporting guidelines - CONSORT-EHEALTH (Quantitative component of the study).
Questionnaire – Online questionnaire developed for the study.
Pillar integration process – Description of the integration of quantitative and qualitative data and categories.
CHERRIES_UNformat citations - Reporting guidelines – CHERRIES (for the online questionnaire).
COREQ_UNformat citations - Reporting guidelines – COREQ (Qualitative component of the study).
GRAMMS_UNformat citations- Reporting guidelines – GRAMMS (Mixed methods component of the study).
VP simulation design elements – Quantitative findings.
Techno acceptance – Quantitative findings based on the Technology Acceptance Model.
Simulation’ role to support practice – Quantitative findings.
Achievement learning objectives – Quantitative findings.
Operationalization integration – Application of the pillar integration process and an excerpt of the joint display.
About this article
Cite this article
Rouleau, G., Gagnon, MP., Côté, J. et al. Virtual patient simulation to improve nurses’ relational skills in a continuing education context: a convergent mixed methods study. BMC Nurs 21, 1 (2022). https://doi.org/10.1186/s12912-021-00740-x
- Computer simulation
- Nursing continuing education
- Motivational interviewing
- Mixed method
- Relational skills
- Simulation training
- Virtual patient simulation