On-site clinical mentoring as a maternal and new-born care quality improvement method: evidence from a nurse cohort study in Nepal

Background We describe an on-site clinical mentoring program aimed at improving emergency obstetrical and new-born care (EmONC) in Nepal and assess its effectiveness on nurses’ knowledge and skills. In Nepal, both the maternal mortality ratio (MMR, 239/100,000 live births) and the neonatal mortality rate (NMR, 21/1000 live births) were among the highest in the world in 2016, despite impressive progress over recent decades considering the challenging environment. Methods From September 2016 to April 2018, three experienced nurses conducted repeated mentoring visits in 61 comprehensive or basic EmONC centers and birthing centers located in 4 provinces of Nepal. Using updated national training manuals and teaching aids, these clinical mentors assessed and taught 12 core EmONC clinical skills to their nurse-mentees. Clinical mentors worked with management mentors whose goal was to improve the nurses’ working environment. We assessed whether the cohort of nurse-mentees performed better as a group and individually performed better at the end of the program than at baseline using relevant tests (chi-square test, Wilcoxon matched-pairs signed-rank test, and Kruskal-Wallis equality-of-population rank test). Results In total, 308 nurses were assessed, including 96 (31.2%), 77 (25.0%) and 135 (43.8%) who participated in all three, two or only one mentoring session, respectively. In total, 225 (73.0%) worked as auxiliary nurse-midwives (ANMs), while 69 (22.4%) worked as nurses. One hundred and ninety five (63.3%) were trained as skilled birth attendants, of which 45 (23.1%) were nurses, 141 (72.3%) were auxiliaries and 9 (4.6%) had other positions. The proportion of ANMs and nurse-mentees who obtained a knowledge assessment score ≥ 85% increased from 57.8 to 86.1% (p <  0.001). Clinical assessment scores increased significantly for each participant, and therefore for the group. SBA-trained mentees had better knowledge of maternal and new-born care and were better able to perform the 12 core clinical skills throughout the program. Conclusions Our study suggests that on-site clinical mentoring of nurses coupled with health facility management mentoring can improve nurses’ clinical competences in and performance of maternity and new-born care. Assessing evidence of impact on patient safety would be the next stage in evaluating this promising intervention.


Introduction
Nepal is a landlocked, low-income country with geographical conditions that make it one of the most disasterprone countries worldwide (earthquakes, river floods and landslides) [1]. Nepal also endured years of civil war, from 1996 to 2006 [2,3]. However, this country has made impressive progress over recent decades in child survival and maternal health, despite these major challenges [4]. The maternal mortality ratio (MMR) decreased between 1996 and 2016 from 539 to 239 per 100,000 live births [4]. The neonatal mortality rate (NMR), estimated at 50 per 1000 live births in 1996, was still among the highest in the world in 2016 (21/1000) [4,5].
The observed progress can be attributed to government efforts, which, after the civil war in the 1990s, demonstrated a consistent policy focus on maternal and child health, with sustained financial commitment and funding for maternal health through the national program 'Aama Surkshya'. This program was implemented by the Family Health Division (FHD) of the Ministry of Health and Population (MoHP) in 2005 and offers cash to women who deliver at a health facility and to health care providers who attend the deliveries [6], which contributed to the increase in institutional deliveries (from 18% in 2006 to 57% in 2017 [4,7]). Improving the quality of care became a focus of national health strategies in 2016 [8] and remains a current goal in the context of Nepal's commitment to universal health coverage [9]. Substantial progress is still needed to achieve the country's sustainable development goals for 2030 (a MMR of 70/100,000 and a NMR of 12/1000) [4,5].
Mentorship of health workers was introduced in Nepal by the FHD/MoHP in 2014 with the support of the Deutsche Gesellschaft fur International Zusammernarbeit (GIZ), which channels the German technical cooperation, and its bilateral Nepal-Germany program aimed at improving maternal and new-born care (MNC). The decision to implement this program was based on the need to follow-up on a MNC provider training program run by the Nick Simons Institute and the FHD of the MoHP, as well as on preceding experience in mentoring to expand abortion care [10]. The initial phase of this program lasted from July 2014 to July 2016 and was initiated due to the stagnation of the MMR and NMR and the poor performance of staff working in maternity settings. Its core intervention was the mentoring of emergency obstetrical and new-born care (EmONC) health staff by national senior peers through a bundled approach combining mentorship of both clinical and management teams working in EmONC facilities. This initial phase was shortened due to a strategy reorientation after the major earthquake in April 2015, which killed 8897 people and destroyed 83.5% of public health facilities in the most affected area [1].
A second phase of this mentoring program was implemented in 2016 at the request of the MoHP. This phase included some districts affected by the earthquake, expanded the mentorship to the staff working in birthing centers and promoted the use of tools developed by the National Health Training Center for skilled birth attendants (SBAs). The objectives of this second phase were to i) improve the clinical capacity of MNC service providers, ii) improve the quality of management in health facilities, and iii) support referral networks. Mentoring was considered a comprehensive teaching and learning process in which senior health workers, who served as the mentors, provided personalized support to juniors, who were the mentees, based on the mentees' needs, to strengthen their ability to provide quality health care [11].
Different mentoring models have been used in diverse country contexts for supporting clinical staff, ranging from field-based teams of mainly non-physician health workers to highly skilled mentors (specialists in obstetrics/gynaecology and paediatrics) and nurse mentors in India and Rwanda [11][12][13]. However, the effectiveness of mentoring programs for health staff is not yet well documented in the scientific literature. Available studies mostly focus on institutional mentoring for medical or nursing students [14][15][16][17]. Several studies have examined mentors' and mentees' perceptions of their experiences, challenges and needs [18,19]. A recent study on the effectiveness of on-site mentoring in Indian maternity clinics examined the impact of mentoring on nurses' knowledge and clinics' readiness to provide services but did not consider the effectiveness of mentoring on provider practices [20]. A Cochrane review noted the need for further evaluations in different settings and contexts [21].
The objective of this paper is to describe a mentoring program aimed at improving emergency MNC in Nepal and to assess its effectiveness on nurses' knowledge and skills.

Methods
We prospectively performed repeated cross-sectional assessments of the knowledge and clinical skills of a cohort of nurses working in comprehensive or basic EmONC centers (CEONCs and BEONCs) and birthing centers in Nepal. These nurses work either as 'staff nurses' (they should have received at least a 3-year preservice nursing training) or as auxiliary nurse-midwives (ANMs) (their pre-service training lasts only 18 months). Some ANMs and staff nurses were certified as SBAs after completion of a 2-month SBA training. Both types of nurses, SBA certified or not, should be able to attend to women in labour, managing both normal and complicated deliveries (before transfer to a higher facility for appropriate care whenever it is possible). In remote birthing centers, ANMs often work alone.
The mentoring program lasted from September 2016 to April 2018 in seven CEONCs, 19 BEONCs and 35 birthing centers in seven/72 districts of Province no. 7, 5, 6 and 3 in Nepal. These health facilities represent, respectively, 6.7, 11.9 and 2.0% of all CEONCs, BEONCs and birthing centers existing in 2015 [8]. Districts and health facilities were purposely selected by the MoHP to represent the diversity of Nepal's geographical and economic environment. Some of these districts belong to the poorest and most remote areas in Nepal. Three of them were also particularly affected by the 2015 Nepal earthquake. All CEONC and BEONC centers in these districts were included in the program, as well as the birthing centers having the highest patient flow or the highest number of pregnancies.

Clinical mentoring organization
The clinical mentors organized three mentoring sessions (each lasting three to four days) in each targeted health facility between December 2016 and March 2018. A total of 12 MNC topics were covered (some of them introduced during the second mentoring session only): completing a normal delivery and resuscitating a new-born, plotting and interpreting a partograph, managing shock due to a post-partum haemorrhage, safely referring a woman or a new-born to a higher-level health facility, performing a vacuum delivery, performing manual aspiration, promoting kangaroo mother care, managing an eclampsia case, decontaminating used medical equipment and using sterile gloves.
In accordance with the standards-based management and recognition approach developed by Jhpiego, which has been used since 1997 and has been implemented in more than 30 countries [22], the mentoring sessions were organized as follows; Mentoring sessions began with an introductory meeting and a visit of the health facility ( Fig. 1). Then, each nursementee was individually assessed regarding her knowledge in MNC, as well as her ability to perform 12 clinical skills (after they had been trained in these skills), using quality improvement (QI) tools approved by the MoHP. Feedback was then given to each mentee before the mentor demonstrated the correct steps using manikins. Poorly performing mentees were invited to repeat the procedure. Where possible, mentoring was reinforced through engagement with real patients. Mentoring visits concluded with a meeting with the mentees, the health facility management team and the respective district health authorities whenever possible. A joint action plan was then prepared, developing actions to be taken to close the observed gaps until the next mentoring session, which occurred five to six months later. Mentorship followup was offered by phone when necessary. The 14 clinical mentors were nursing graduates with extensive maternity and public health and training experience, selected with the help of the Ministry of Health based on technical criteria and their communications skills and experience; four of them were SBA trainers. They were trained on the mentoring approach and QI tools in October 2016 and were supervised by the program team.

Complementary management mentoring
Clinical mentors worked in pairs with management mentors who supported health facility management teams. In a given health facility, clinical and management mentors jointly conducted mentoring sessions. The management mentors worked with the health facility heads and management committees and covered eight quality care domains: health facility management, infrastructure, patient dignity, staffing, supplies and equipment, drugs, and infection prevention (Additional file 1). Concomitant mentoring activities were also organized regarding advanced MNC practices with the help of specialized mentors for CEONC and neonatal care staff.

Knowledge and skills assessments
Standardized assessments using multiple choice questions and an objective structured clinical examination were used to measure the changes in knowledge and clinical skills during the program.
The nurse-mentees' knowledge was assessed at the beginning of each mentoring session using an autoadministered questionnaire in the Nepali language extracted from the national SBA training manual [23]. It consisted of 26 questions on the following topics: antenatal care, partograph, normal delivery, vacuum delivery, complicated procedures, new-born resuscitation, eclampsia, and infection prevention (detailed list shown in Additional file 2). Knowledge assessment scores were calculated as the proportion of correct answers to the 26 questions.
The mentees were also assessed before each mentoring session for 12 core clinical skills using the QI tools extracted from the same SBA training manual. Each QI tool consisted of a set of questions to answer or procedures to demonstrate grouped into 'standards' with verification criteria. The expected answers, procedure steps and verification criteria were detailed in checklists attached to the QI tools. For each assessed skill, a skill score was computed as the number of standards fully and correctly performed over the total number of standards; these results were presented as a percentage. Managing an eclampsia crisis was not assessed among non-SBAs during the first mentoring session. The referral procedure and condom tamponade were introduced during the second mentoring session only. Manual vacuum aspiration and vacuum delivery were assessed in only CEONCs and some BEONCs where the necessary equipment was available. We called the 'overall clinical score' the mean of results obtained for the 12 clinical skills assessed, expressed as percentages.

Analyses
The clinical mentors recorded the mentees' assessment scores using an Excel sheet or the KoBo Toolbox [24]. Data sets were anonymized using a unique identifier for each mentee. Characteristics of the mentees are presented as medians and interquartile ranges (IQRs) for continuous data and frequencies and percentages for categorical data.
We assessed mentees' knowledge and skills at baseline and determined whether they performed better according to their nursing education, their SBA status, their function and place of employment using the chi-square test. Mentees who were not engaged in daily clinical practice (i.e., not working as ANMs or nurses) were removed from the analyses. The National Health Training Center and the program team defined good knowledge and performance scores as scores equal to or above 85%.
We then assessed whether the mentees performed better as a group at the end of the program than at baseline. We therefore compared the proportion of ANMs and nurses with scores ≥85% between the first and last mentoring sessions using the chi-square test.
We completed the analysis by assessing the change in mentees' assessment scores from baseline to the end of the program according to their SBA status using the Kruskal-Wallis equality-of-population rank test. The mentees who became SBAs during the program were removed from the analysis comparing SBAs with non-SBAs.
Finally, since not all mentees attended all mentoring sessions, we also assessed whether they individually performed better at the end of the program using the Wilcoxon matched-pairs signed-rank test. In this test, each mentee was matched with herself at baseline and at the end of the program, and her first assessment was used as the point of reference for comparisons.
All analyses were performed using Stata 13 (Stat Corp., College Station, TX, USA) software. Statistical significance was defined at a 5% threshold (p < 0.05).

Ethical considerations
This article does not report on a health research involving human subject as stated by the Nepal Health Research council (file:///C:/Users/Acer/Downloads/ERB_Guideline_ 2019-final-_29-Sep-1.pdf). It is a review of an educational program jointly implemented with the MoHP of Nepal. Delivery and evaluation of this educational program were approved by and implemented with the MoHP. All nurses involved in this education program provided their oral consent to participate in the program, and were allowed to drop out anytime. Their consent was recorded in the anonymized program follow-up database. Knowledge and skills assessments were considered part of the learning experience. Databases that recorded assessment results were anonymized, with access restricted to data analysts and program managers.

Results
The clinical mentoring program reached a total of 308 nursing staff. Of them, 196 benefitted from the first mentoring session, 197 from the second session and 184 from the last one. A total of 96 mentees (31.2%) participated in all three mentoring sessions, 77 (25.0%) received two mentoring sessions, and 135 (43.8%) were reached only once ( Table 1).

Mentees' assessments at baseline
During the first mentoring session, 185 ANMs and nurses completed the initial knowledge assessment, with 107 (57.8%) of them correctly answering 85% or more of the 26 questions (Table 2). SBAs had a higher level of knowledge than non-SBAs (64.8% of SBAs obtained a score ≥ 85%, vs 34.9% among non-SBAs, p = 0.001). Staff working in CEONCs also had better knowledge than staff in BEONCs and birthing centers (72.5, 67.7 and 38.9%, respectively, p < 0.001). We did not find evidence that staff with higher nursing education performed better in the knowledge assessment than less educated staff or that nurses had better knowledge than ANMs.
Of the 186 ANMs and nurses assessed for at least one clinical skill at baseline, 27 (14.5%) obtained a good ANMs should be support staff nurses, but they often work alone overall clinical score ( Table 2). Nurses performed better than ANMs (27.8% of the nurses obtained a good score vs 11.3% of ANMs, p = 0.01); SBAs performed better than non-SBAs (17.5% vs 4.6%, respectively, p = 0.03), as well as staff working in CEONCs (35.3% vs 9.7% in BEONCs and 4.1% in birthing centers, p < 0.001). We did not find evidence of any association between the level of nursing education and clinical performance at baseline. The mentees' clinical performance varied widely at baseline, from 91.5% of them knowing how to put on sterile gloves correctly to 13.4% of them being able to manage a case of shock due to a post-partum haemorrhage (Table 3). Less than one-fourth of the mentees  were able to correctly manage an eclampsia case (13.9%), conduct a normal delivery following national standards (18.2%) and interpret a partograph (20.2%). At baseline, our data did not show any difference in performance between the nurses and the ANMs, except for 'managing an eclampsia case' (26.7% of nurses obtained a good score vs 10.7% of ANMs, p = 0.02; Table 3). The skills performed better by SBAs than non-SBAs at baseline were the following: new-born resuscitation, interpretation of a partograph and decontamination of instruments (data not shown).

Mentoring effectiveness on knowledge
The proportion of mentees who obtained a good knowledge assessment score increased to 86.1% at the third mentoring session, indicating a significant and positive increase of 28.3% in knowledge (p < 0.001) ( Table 2). The mentees who gained the most knowledge were those working in birthing centers (+ 52.0%), those not trained as SBAs (+ 36.1%, p < 0.001), those with the lowest level of nursing education (+ 30.3%), and those working as ANMs (+ 28.8%).

Mentoring effectiveness on clinical performance
The overall clinical performance increased at the end of the program (+ 62.7%, p < 0.001). The greatest improvement was observed in the birthing centers (+ 80.7%) among SBAs (+ 72.3%), ANMs (+ 67.6%) and staff with the lowest level of nursing education (+ 67.3%). During the last mentoring session, significant improvements were observed for all skills assessed ( Table 3). The skills with the greatest improvements were the management of haemodynamic shock (+ 57.2% of mentees performed well compared with those at baseline), management of eclampsia (+ 56.9%), new-born resuscitation (+ 56.7%) and completion of a normal delivery (+ 55.0%). The skills that improved the least (decontaminating instruments and putting on sterile gloves) were those that were already mastered at baseline. Comparisons of changes in clinical assessment scores did not show any significant differences between SBAs and non-SBAs (Additional file 3: Table S1). This result means that although this mentoring program used tools that were initially designed for SBAs, both SBAs and non-SBAs improved in their knowledge and ability to demonstrate clinical skills. Similar comparisons between ANMs and staff nurses showed that ANMs' gains in clinical performance were higher than those of nurses for the two following skills: managing shock due to post-partum haemorrhage and conducting a normal delivery (+ 30.0% among ANMs vs + 21.0% among nurses, p = 0.005; + 21.0% among ANMs vs + 18% among nurses, p = 0.03, respectively) (Additional file 4: Table S2).

Mentoring effectiveness at the individual nurse level
The improvement of knowledge and clinical skills was also confirmed at the individual level (Table 4). Of the 172 mentees who underwent at least two knowledge assessments, 75.0% increased their scores. A significant positive increase was also found for all clinical skills assessed. More than 90% of mentees who were assessed at least twice improved their scores for the following skills: completing a normal delivery, managing postpartum haemodynamic shock, resuscitating a new-born, and managing eclampsia. The two skills which least improved were those best performed at baseline (using sterile gloves: 91.4% of mentees obtained a score > 85% at baseline; decontaminating instruments: 79.6% with high scores at baseline). In most cases, when the scores were lower at the end of the program, the drops in performance were minimal. For instance, the mean drop for the seven staff getting lower scores for managing an eclampsia case was − 6%.

Main findings
Our study showed that prior to the mentoring program, the nursing teams in charge of deliveries in CEONCs, BEONCs and birthing centers of Nepal demonstrated limited ability to perform several life-saving procedures in emergency obstetrical and neonatal care, regardless of their initial training. In particular, these teams were poorly able to manage shock due to post-partum haemorrhage or an eclampsia crisis. Additionally, common procedures, such as completing a normal delivery according to the national standards or correctly interpreting a partograph and taking action, were not mastered by the maternity staff in charge of deliveries. These results may be explained by the limited clinical experience and practice exposure of these young nurses, as illustrated by the fact that nurses working in CEONCs (with a higher caseload) had better knowledge and skills than those of other staff. Nevertheless, this result raised questions about the quality of their pre-service training. These findings corroborate a recent study that concluded the presence of "limited capacities of maternity staff in Nepal, despite important investments from the Government" [25]. However, we found that ANMs and nurses trained as SBAs had better knowledge and better overall clinical scores than their non-trained peers, which shows the impact of the efforts by the MOH since the mid-2000s to upgrade its active workforce in maternity settings.
Notable improvement was observed after this structured mentoring program was provided by senior nurse peers. The mentees significantly improved their knowledge in MNC, with 86.1% of them reaching a 'good level of knowledge', as defined by the MoHP of Nepal. The mentees also improved their ability to perform the 12 core MNC clinical skills of this program, with marked improvements in the following three procedures that save mothers' and new-borns' lives: managing eclampsia, managing post-partum haemorrhage shock, and resuscitating a new-born.
This mentoring program appeared to have greatest impact among the staff with the lowest level of nursing education and in the lowest level of health facilities providing maternity care, namely, the birthing centers, where most of the staff are ANMs who work in isolated and remote areas despite their limited education. Interestingly, the mentoring program had a strong positive effect on non-SBAs, although the QI tools used were primarily designed for SBAs. We strongly believe that this is due to the individual-to-individual approach used in the programme, which allowed mentors to tailor their input according to the needs of each mentee.

Limitations of the study
The lack of a control group means that we are unable to determine to what degree the changes in outcome are solely the result of the intervention. Moreover, we assessed the nurses' clinical skills mostly using case scenarios and simulation models due to the limited caseload in some health facilities, which do not replace clinical observations during the provision of care to actual patients. This difficulty is inherent to QI programs in EmONC. Emergencies in maternity settings are often unpredictable and rare in places with low caseloads [26]. Moreover, the data we collected did not allow us to identify the potential impact of the program on maternal and new-born outcomes. Another limitation of this study is that the validity and the reliability of the knowledge and skills assessments are not known.

Sustainability
The institutionalization of this mentoring program within the MoHP in a sustainable form was initiated as early as the beginning of the program by the FHD/ MoHP and the National Training Health Center. More than 90 government mentors were already trained and certified by the FHD/MoHP and the National Health Training Center at the end of the program. Funds have been secured by the MoHP to cover their travel and accommodation costs, as well as to fund extra staff to fill the gaps created while mentors are away from their post to perform their mentoring activities.
However, the overall sustainability of the program remains unclear, mainly due to the local political context, as the health system is undergoing a complete restructuring to fit into a newly implemented federal system. The program team suggested that the MoHP explore the feasibility of developing 'cascade-mentoring', with local Score change was computed for each mentee as the difference between the first and last assessments. A higher score at the end indicated that the mentee improved her knowledge or clinical skill a Significance was assessed using the Wilcoxon matched-pair signed-rank test; P was estimated for two-sided test, testing for both positive and negative increases at the end of the programme. The test showed that none of the loss in performance score was statistically significant b For skilled birth attendants only mentors further trained and mentored by provincial mentors, as successfully implemented in Uganda in the area of MNC [26]. Nepal has, however, taken a very promising step forward by launching a midwifery education program [27][28][29]. It is expected that the soon-to be deployed midwives will help improve the clinical skills of ANMs and nurses.

Lessons learned
Given the geo-economical context of Nepal and its current transition from a central government to a regional system, it will take time before all birthing centers have a trained midwife or SBA-registered nurse. Meanwhile, and because SBAs are better equipped to address normal deliveries and obstetrical complications than their non-trained peers, SBA training is still provided and should be scaled-up and accelerated for all employed maternity staff and combined with regular refresher courses and supervision/mentoring by experienced nurses. Plans are currently being made by the MoHP to reshape the existing SBA course into a 'bridging course' to become a certified 1-year midwifery course for all staff who have previously completed the 3year PCL nursing course. Trained staff will be registered as midwives in the Nepal Nursing Council. Among the lessons learned from this program, we found that on-site clinical mentoring in remote locations requires motivated mentors, engaged mentees and support from hospitals and health facility management committees. During qualitative focus group discussions held during the closing project meeting, the program team, including the mentors, also observed that mentoring builds teamwork, staff confidence, motivation, and a sense of ownership and pride. This finding may be due to one innovative feature of this mentoring program: the bundle approach linking clinical and management mentoring. Given that high-quality clinical services strongly depend on management support for the provision of basic infrastructure, water, power, equipment and supplies, as well as regular cleaning and maintenance, all clinical mentoring activities were organized jointly with the management mentors, promoting synergy, interaction and collaboration between the clinical and management teams, as seen elsewhere [26]. At the facility level, clinical and management teams developed the habit of discussing context-specific needs and of developing and effectively implementing gap-closing plans.
Among the factors that adversely affected the implementation of the program, in addition to the low caseload mentioned in the limitations section, we note the difficulty of maintaining the cohort of nurse-mentees due to frequent staff transfers and vacancies (mainly training opportunities or pregnancy leaves). An additional challenge was the changing political context, as Nepal's constitution was revised during the program, and the MoHP moved from a centralized ministry to a federal context.

Conclusion
Quality of care has recently gained much attention, as it is now clear that universal health coverage will not be reached without improvement in the quality of health services [30]. Despite its limitations, our study adds to the body of evidence on methods of quality care improvement. This on-site clinical mentoring program, coupled with health facility management mentoring, effectively contributed to increasing clinical competences and performance of health workers in maternity settings. Still the study did not measure the programme's impact on patient safety. This program functioned as an integral part of the MNC program of the MoHP through the use of tools developed by the National Health Training Center. Nevertheless, the sustainability of this program is being challenged by the current restructuring of the overall health system toward a federal government. However, this health system restructuring offers an unprecedented window of opportunity that should be utilized to strengthen human resources for MNC. One-site mentoring could be a good option, as commonly implemented in GIZ-supported programs, to help change clinical practices at the frontline to deliver high-quality healthcare "at the right time, in the right place, by the right care provider, while minimizing harm and resource waste and leaving no one behind" [31].