Skip to main content
Download PDF

Standards for evaluating the quality of undergraduate nursing e-learning programme in low- and middle-income countries: a modified Delphi study

BMC Nursing volume 22, Article number: 73 (2023) Cite this article

Abstract

Background

The lack of standards for evaluating the quality of undergraduate nursing programmes hampers the evaluation of e-learning programmes in low- and middle-income countries. Fragmented approaches to evaluation coupled with a lack of uniform criteria have been a major deterrent to the growth of e-learning. Adopting standards from high-income countries has contextual challenges in low- and middle-income countries (LMICs). Holistic approaches coupled with uniform standards provide information to stakeholders hence the quality of the programmes is measurable. The e-learning situation in low-and middle-income countries provided an impetus to develop and validate these standards.

Design

A modified Delphi technique.

Review methods

Fourteen experts with experience and expertise in e-learning and regulation of undergraduate nursing from fourteen countries from LMICs participated in three rounds of the modified Delphi process. A pre-described set of standards was shared electronically for independent and blinded ratings. An 80% threshold was set for consensus decisions. The standards were modified based on experts’ comments, and two subsequent rounds were used to refine the standards and criteria.

Results

At the end of round one, the expert consensus was to keep 67, modify 39 and remove three criteria. At the end of the second round, the consensus was to modify 38 and remove one criteria. In the third round, experts agreed that the standards were feasible, usable, and practical in LMICs. A total of six broad standards with 104 criteria were developed.

Conclusion

The Technological bloom permeating all spheres of society, including education is an essential component in the development of e-learning programmes. E-learning in nursing education requires critical evaluation to ensure quality in undergraduate nursing programmes. The intricacies of the Low and middle-income context were taken into consideration in developing the standards to offer sustainable evaluation of the quality of e-learning in LMICs, and local solutions to local problems.

Peer Review reports

Introduction

The mere availability of nurses is insufficient to meet the global needs for high-quality nursing staff to respond effectively to current health challenges [1]. They in fact also need the required competencies and must be effectively trained to meet the health needs of communities [2]. There is an increased demand for high-quality degreed nurses with improved skill sets across various contexts, including low- and middle-income settings [3]. A range of evidence supports Choi et al. (2021); Porat- Dahlerbruch et al. (2022); Drasiku et al. (2021) and Harrison et al. (2019) [4,5,6,7], nurses with a degree-level qualification offer better nursing care and improved patient health outcomes, including reduced preventable hospital deaths. Therefore, numerous strategies are used to accelerate the upgrading of nurses from diploma or associate degree qualifications to bachelor’s degrees [8]. Most of these upgrading programmes are offered through e-learning or open distance learning approaches, taking advantage of the nurses’ prior knowledge and skills [9]. The flexibility provided by e-learning allows student nurses to navigate various competing interests such as work, family, and professional development [10].

E-learning has challenged conventional teaching approaches and revolutionised learning with ease and accessibility otherwise unimagined in education [11]. E-learning is an approach to facilitate the application of knowledge and information and enhance learning using personal computers, CD-ROMs’ and the internet for education [12]. The critical components of an e-learning programme are divided into four broad categories: curriculum, learners, educators, and, support, and evaluation (Misut & Pribilova, 2014; Gautam & Tiwari, 2016) [13, 14]. Nursing education curricula have also been significantly influenced by technological advances Risling, (2017) and will be shaped by e-learning in the future as the technological landscape is growing with advances in all spheres of human interaction [15]. Murebwanyire et al. (2015); Harerimana & Mtshali (2020) describe how e-learning was identified as a large-scale method to upgrade nurses to a diploma and higher diplomas [16, 17]. The demand for nurses with a degree has resulted in unprecedented increase in the number of nurses upgrading qualifications from diploma to degree. This gap has been met by e-learning, enabling many nurses to access opportunities for learning [18]. E-learning has been instrumental in developing the capacity for an alternative learning environment for continuing professional development and lifelong learning, especially for those whose work schedules offer limited time to travel over long distances to acquire knowledge [19, 20]. Regmi and Jones (2020) state that the ability of e-learning to transcend barriers and enhance learning has modelled it to be a mainstream approach in health sciences [21].

The shifting dynamics of the learner, educator, and environment have led to unprecedented and transformative approaches to teaching and redefined nursing education, thus shaping e-learning [22, 23]. Using e-learning programmes, therefore, provides appropriate learning experiences that can support clinical learning experiences with demonstratable teamwork, leadership, practical knowledge, and an emphasis on work readiness [24]. The delicate balance between what constitutes a successful e-learning programme and quality evaluation in undergraduate nursing programmes provides an avenue for deliberations and developing quality assessment tools applicable in LMICs.

Africa is experiencing significant growth in e-learning, especially since the COVID-19 pandemic, even though e-learning is still in its infancy in many LMICs [25, 26]. The proliferation of internet usage in Africa is dissimilar to undergraduate learners who consider e-learning as a mode of study; whilst the former figures are surging greatly, the latter is slowly gaining familiarity among undergraduate learners in LMICs [27]. E-learning in Africa developed in 1996 with centres in Ethiopia, Ghana, Kenya, Uganda, and Zimbabwe. The initial programmes were on science, engineering, and business [27]. In Kenya e-learning programmes in nursing education were offered through a distance learning model using textbooks, as a computer-based approach to e-learning was a challenge [28]. However, there was no public and institutional policy and standards to support e-learning initiatives in most LMICs during those early days. This lack of support was further complicated by a lack of infrastructure, poor internet proliferation, a lack of capacity among institutions, educators and learners, limited familiarity with technology in education and a negative attitude to e-learning [29, 30].

Standards offer a yardstick for evaluating the quality of an educational programme. Thus, e-learning programmes must be evaluated to determine their quality [31]. Standards in e-learning are documented agreements containing technical specifications including rules, guidelines, and definitions of characteristics to ensure that materials, products, processes, and services are fit for their purpose with specific parameters in order to ensure that e-learning courses are apt and useful for their user [32]. Quality standards for assessing e-learning programmes should be anticipated while developing the programme to provide methodological, valid, and objective assessment criteria [33]. Delva (2019) describes that standards provide a roadmap to assess the quality of e-learning [34]. The quality of an e-learning programme includes learners who are ready to learn and supported to learn, environments with adequate resources, content reflected in relevant curricula for the acquisition of relevant skills, educators able to carry out roles in skill acquisition, assessment and outcomes that demonstrate knowledge, effective use of ICT by learners and educators, and skills and attitudes linked to desired results [21, 35, 36]. Standards for evaluating the quality of undergraduate e-learning programmes can be adapted for nursing education and broadened to include clinical components. These standards could, therefore, redefine how quality in nursing education should be evaluated specifically in LMICs [21, 37]. Standards in use in LMICs are adopted from High income countries and these offer contextual challenges in implementation [9]. Focus should be on context specific standards developed in LMICs.

Evaluating the effectiveness of e-learning programmes can guide the development of nursing education in order to determine its value and importance. This can be best done by making use of a set of standards. This set of standards will inform stakeholders on the excellence of these programmes [38, 39]. As e-learning programmes develop, so does the need to monitor the quality of e-learning and its components to provide an effective programme. However, the evaluation of e-learning should not consider constituent parts only, but a holistic evaluation of the e-learning programme [40]. A common pitfall of most evaluations in LMICs is the lack of uniform criteria for evaluating the quality of the e-learning programmes, coupled with the lack of standards.

This has led to the inability of regulatory bodies to evaluate such programmes in nursing education [40]. The lack of holistic standards has been a stumbling block in establishing quality evaluation tools in e-learning programmes in LMICs, as most evaluations focus more on the components of e-learning than on the entire programme collectively.

The use of fragmented approaches to assess e-learning in LMICs is rooted in an over-emphasis on specific areas of e-learning rather than on the programme as a whole. This fragmentation is attributed to a lack of standards [29, 41, 42]. Barteit et al. (2020) further identified that most evaluations of e-learning in LMICs are small-scale, and more rigorous evaluation methods are required to understand the strengths and shortcomings of e-learning programmes [29]. Baker et al. (2021) describe that standards provide a clear delineation of the requirements of programmes to meet the global needs of high-quality nursing personnel able to respond effectively to current health challenges [43]. Therefore, the lack of direct standards for evaluating e-learning programmes across LMICs can explain the intransigence of evaluating these programmes.

The need for context-specific standards for e-learning to meet the specific needs of the programme whilst matching with international standards of quality is required in LMICs [41, 44]. The paucity of standard evaluation tools for e-learning in LMICs limits the validity of evaluation results [29, 41]. However, e-learning can be evaluated against established standards which already exist and might not be specific to LMICs [34]. We contend that adopting evaluation criteria from developed countries to be applied in LMICs offers context-specific challenges to the development of e-learning. The adopted evaluation criteria, albeit contextually sound in the developed country, could mollify other features that may potentially inform more robust e-learning systems in LMIC contexts. The novelty of these developed and validated standards in LMICs, keeping in mind the contextual challenges compounding LMICs, offers feasible, usable, and practical techniques to holistically evaluate the quality of undergraduate e-learning nursing programmes.

Methods

Aim

This paper reports on a consensus study on standards for evaluating the quality of undergraduate nursing e-learning programmes in LMICs.

Design

This paper is drawn from an overarching project which sought to develop, validate, and test standards for evaluating the quality of undergraduate nursing e-learning programmes in Kenya. Preliminary work was conducted in an earlier phase of the study through an integrative review which identified six broad thematic areas with a total of 109 criteria for consideration (see Additional file 1). The standards were divided into six categories: curriculum planning, proficiency of the educator, learner proficiency and attitude, infrastructure for learning, support, and evaluation. Each of the standards had different criteria. The modified Delphi is similar to the full Delphi in respect to intent and procedure. The modification consists of beginning the process with standards developed from integrative reviews [45]. The modified Delphi has additional benefits of reduced bias, anonymity, and controlled feedback to participants [45, 46]. This study applied a modified Delphi technique used to achieve consensus on standards or quality indicators, especially in areas with no recommended guidelines for a specific problem [47, 48]. For a modified Delphi to be successful and maintain quality, it was important to identify the problem, meticulously select the panel of experts, ensure the anonymity of experts, obtain controlled feedback, conduct iterative Delphi rounds, gain consensus on the criteria, analysis of the consensus, closing criteria and result stability [49]. In line with Brown and Crookes (2016), equal anonymous contributions for minimising power differentials and reducing bias were applied in the modified Delphi [50]. Consensus-building was developed by:

  • applying the three steps of identifying potential experts for inclusion in the expert panel,

  • conducting a modified Delphi and.

  • presentation of the final standards.

Step 1: Identifying the experts to include in the panel

Experts were selected based on the following inclusion criteria: experience in e-learning in LMICs through teaching and/or policy formulation related to e-learning, recent publication history in e-learning in LMICs, a nursing or health sciences background and technical knowledge of e-learning. This selection involved administrators, digital campus, and e-learning support personnel from a number of LMICs.

The inclusion criteria were used in the selection process to ensure that objectivity was maintained throughout the process.

Firstly, the authors from the studies identified in the integrative review stage were included. The total number of experts in this first phase was ten. Secondly, both authors identified individuals within their networks who met the inclusion criteria. The total of 15 experts identified were composed of specialists in nursing education from Africa and Asia. These 34 identified experts were invited to be part of the study through an email with a participation information leaflet and a consent form.

The experts were also requested to nominate other experts who met the inclusion criteria, and these nominated experts were also invited participate in the study. The 14 experts who volunteered were included in the modified Delphi.

Step 2: Data collection

The data collection tool for the first round was developed using the REDCap application ® and Survey Monkey ®. A total of six standards with a total of 109 criteria were used. The modified Delphi had three rounds of ratings, reviews, and reflections by the experts over sixteen weeks between May and August 2022.

Modified Delphi round 1

Round 1 of the modified Delphi had an online questionnaire, The experts were presented with all six standards and 109 criteria and asked to state whether they agreed, disagreed or were not sure about a stated criterion. We used 3-point Likert scale questions and additional space for the experts to provide their opinions on the standards in their own words, this would assist in a clear interpretation and actionable results for the next phase of the study. Experts had two weeks to complete the questionnaire. Fourteen responses were received, translating to a 100% response rate. The consensus was set at 80%, with areas of disagreement providing an opportunity for a second round [46, 51]. Responses to the modified Delphi, phase 1, were analysed by the authors over two weeks. Any question reaching the consensus threshold was regarded as finalised and was not included in the second round. In cases where at least 80% consensus was not reached by the experts, if they were unsure about a standard and-or were unsure about a standard and or its criterion, the standard and its criterion were reformulated based on the feedback from this round and included in the questionnaires for the second round.

Modified Delphi round 2

The questionnaire for the second round required the experts to review the standards that did not reach the consensus threshold. They could either keep or, discard standards and criteria or suggest alternatives or modifications. The standards were presented in RED Cap format with a similar 3-point Likert scale approach. At the end of the second round, the standards proposed by the experts were deliberated, the decisions to retain or remove were made in line with the expert opinion, and the final standards were developed. The consensus reached in the second phase was considered final.

Modified Delphi round 3

This final round aimed to provide the experts with the finalised standards after the iterative process. Experts were requested to comment on their feasibility, usability, and practicality in LMICs. The experts agreed on all aspects.

Data analysis

Data from the modified Delphi rounds were analysed quantitatively by descriptive statistics and qualitatively by themantic analysis.

Data analysis involved analysing each criterion for consensus [52]. After each rating round, analysis was done on each standard, and those standards with a consensus of 80% across all the responses were moved to the final draft. The standards and criteria that did not reach 80% consensus and areas of disagreement were rerated in the second round, and the recommendations and comments made by experts were considered. Responses from the second round provided ratings with areas of agreement with modifications, and the standards moved to the final draft.

All areas of agreement from the second round were incorporated, and those that did not reach the threshold were eliminated from the standards. The third round presented the final standards through the iterative process.

Rigour

Ethical approval

was sort from the Health Sciences Research Ethics Committee (HSREC) and National Commission for Science and Technology (NACOSTI). Trustworthiness reflects the rigour of a study and concerns ensuring its credibility, dependability, confirmability, and transferability [53]. The initial findings of the study were discussed to identify opinions and feedback and to determine dependability The views of experts were sought who are leaders in nursing education and e-learning and are familiar with the evaluation of e-learning. Rigour was thus maintained by selecting a heterogeneous sample of experts with different backgrounds but relevant experience and the capacity to validate expert opinion [54]. All activities were documented to determine the confirmability of the findings, and a report of the process was prepared. The results were shared amongst individuals of similar situations and experiences (LMICs) to determine transferability, and the results were confirmed to be feasible and useable. A blinded review process ensured that experts’ identities were hidden to avoid group bias [54]. Rigour is further maintained by consistency and meticulous adherence to the process [54].

Results

Most of the participating experts from Africa and Asia were female n = 8 (57%) with roles in nursing education regulation, directors of e-learning, nursing, and midwifery education. The male participants n = 6 (43%) had roles in nursing and midwifery education, health professions education backgrounds in nursing education and e-learning (see Table 1.)

Table 1 Demographic characteristics of experts
Full size table

A total of 14 experts consented to participate (see Fig. 1).

Fig. 1
figure 1

Expert flow throughout the study

Full size image

A total of 109 criteria were presented to the experts at the start of the modified Delphi process. At the end of round 1, experts agreed to keep 67, modify 39 and remove three criteria. The 67 criteria were included in the final standards and were not entered into the second round. The remaining 39 criteria were amended based on the experts’ choice and proceeded to the second round. At the end of the second round, the expert panel reviewed 39 criteria from the six different domains and either agreed, or disagreed, with the criteria and had the option to offer alternative criteria. The panel suggested the modification of 38 criteria and removal of one. The expert panel reached consensus on all the presented standards, and a final draft was developed. The third stage was the presentation of the final standards to the experts for final approval. A total of eight (57%) experts responded to the final survey and they were required to reflect on the usability, feasibility, and practicality of the standards. The consensus on the three aspects was 100% (see Fig. 2).

Fig. 2
figure 2

The flow of standards throughout the process

Full size image

In the modified Delphi process, 104 standards across six domains were developed. The feasibility, usability, and practicability across LMICs were at 100%, with the experts agreeing that the standards were applicable in LMICs (see Table 2).

Table 2 Summary of standards through the stages of the study
Full size table

The standards were divided into six categories namely: curriculum planning, proficiency of the educator, learner proficiency and attitude, infrastructure for learning, support, and evaluation. Each of the standards had different criteria (see additional file 1 shows this in more detail).

Discussion

The success of e-learning in high-income countries is anchored on a combination of models such as Delone and McLean’s information systems success model, user satisfaction models and technology acceptance models [55, 56]. Certain challenges were reported regarding the model of Delone and Mclean (2003) due to complexity for implementors [57]. Several revisions were made to the model with evident integration challenges. The technology acceptance models have been widely used to evaluate e-learning success. However, criticism has been directed towards the application of these models and attempts to expand these models leading to theoretical challenges and confusion [58]. User satisfaction models, focusing on learner satisfaction have been extensively used in various disciplines, including health sciences [59]. The standards developed in this study differ extensively from those readily utilized in high-income countries. Despite this, the authors’ specific focus was the LMIC context, which differs immensely in terms of curriculum, availability of resources, contextual culture, infrastructure, educators, learners, and technical support. A plethora of models are available for evaluating e-learning quality in high-income countries with comprehensive criteria that are successful in that context. The standards developed in high-income countries, are applicable in contextually similar settings and have been challenging to apply in LMICs [60]. This is the essential focus behind the development of these standards namely to provide quality standards for evaluating undergraduate nursing programmes in LMICs and involvement of experts within LMICs.

The urgency to develop standards for e-learning in LMICs is anchored in its rapid expansion, especially post-pandemic, and the need to maintain quality in the education of nurses who are critical determinants of health outcomes in communities and the health of the patients they serve [61]. Currently, there are no standards for evaluating the e-learning quality in LMICs. The central focus in developing such standards was to bridge a gap and develop a holistic evaluation of e-learning, as opposed to a fragmented approach model [36]. The focus was on highlighting the challenges regarding e-learning in LMICs, the available opportunities, and best practices across various LMIC settings [62,63,64]. These perspectives shaped the foundation of the study and grounded the process of developing e-learning standards in LMICs. This approach was pragmatic in developing the quality of e-learning, especially in LMICs which are beset by a host of challenges. It offers a systematic perspective for university boards of management approach regarding the needs of e-learning rather than a fragmented perspective which obscures the development of e-learning [65, 66]. Widyanti et al. (2020) found that most university management structures realized the value of e-learning educators when the COVID-19 pandemic led to the global closure of universities [67]. This led to a higher cognitive workload in e-learning compared to face-to-face learning, coupled with the weight of designing content, teaching, evaluating learners, and infrastructural challenges [66]. The ensuing confusion led to some universities opting to close indefinitely, while those that managed to remain open had difficulty manoeuvring the e-learning landscape with no standards to guide the transition process [68]. The timing of these standards is interwoven with the desire of most universities to actualise e-learning as a mode of study. It is quickly becoming a reality in the health science departments and the quality of the programmes is critical.

The current study reports the findings of a modified Delphi on standards for evaluating the quality of undergraduate e-learning programmes. The study accentuates expert opinion and consensus on the standards in LMICs and highlights various spheres of concern to experts across the different criteria. Shifting learning practices and the advent of e-learning and virtual learning are shaping contemporary approaches to understanding the curriculum [69]. The curricula in use were developed in previous years and were subject-centred. While it was essential at the time, the shift to learner-centred approaches in e-learning demands a change in the approach to curriculum design [70, 71]. The two interacting constructs of societal needs and learners are critical factors in shaping a curriculum. The former is the foundation of the traditional curriculum, while the latter shapes the e-learning curriculum [69]. Integrating technology into the curriculum, blending thinking, innovation, and ICT skills is inevitable in engaging the millennial learner who is central to the organisation of learning whilst the educator supports learning. The use of a traditional face-to-face curriculum in e-learning has therefore been a huge impediment to the growth of e-learning, and undermines the progress made toward improving e-learning quality in LMICs [72]. A curriculum specific to e-learning is a starting point for quality undergraduate e-learning programmes, as the curriculum would provide all stakeholders with a measurable structural plan for the programme’s accountability and social responsibility, especially in LMIC settings [69, 73]. Furthermore, the novelty, clarity and comprehensiveness of these standards will enable the development of an e-learning curriculum. We argue that with the shifting dynamics of the learners, content, teaching approaches and demands of modern-day education, a rigid traditional face-to-face curriculum would be impractical in e-learning.

The second standard regarding the proficiency of educators in is a common but often overlooked concept in e-learning. The following aspects are important: the attitude of the educator; learners who consider educators as role models, and the educators’ attitude toward e-learning that rigorously affects the learners’ attitude [59, 73]. Experts agree that the increased workload associated with e-learning, class size issues, teaching resource availability, mental workload and educator motivation are critical factors in educators’ perceptions of e-learning [74, 75]. Almahasees et al. (2021) identified differences in training and e-learning delivery between high-income and low-income countries. The lack of teaching resources available to educators, reduced knowledge of ICT and the challenges of class control emerged as areas of agreement among most experts [76]. This has been a problem in the development of e-learning in low-income countries [26, 74, 77]. The educators’ motivation for the use of e-learning, teaching styles, their perception of e-learning and control of technology will shape the behaviour of learners and significantly improve learner satisfaction [78, 79]. The experts agreed on the use of e-learning to advance the continuing professional development (CPD) of educators to ensure a commitment to continuously improve their knowledge and skills. This is supported by Mlambo et al. (2021) who found that the accessibility and availability of CPD through e-learning for educators greatly improved their competence in the use of e-learning tools [80]. The role played by educators in shaping learners’ perceptions cannot be overestimated, especially in e-learning.

The third standard, the relationship between learner proficiency and attitude and the use of e-learning and consequent success in e-learning is significant and leads to the satisfaction of learners [55, 81]. Perceived satisfaction of learners regarding the use of e-learning is an essential measure of the success of e-learning and will lead to the optimal use of the system [82]. Usefulness is a key determinant of the success of e-learning among learners, because if they perceive e-learning to be useful, it will increase to the likelihood of their use of e-learning and significantly influence satisfaction [82, 83]. The ability of learners to objectively evaluate teaching and learning was an area of concern as most of the learners were not actively involved in the evaluation of e-learning programmes compared with learner challenges limiting the objective evaluation of e-learning. However, this contrasts with Al-Fraihat et al. (2021); Penna and Stara (2007), who identified that learner-centred evaluation is essential to the quality of e-learning [55, 84]. Experts agreed that learners should be prepared in advance for e-learning and notified on how to navigate the course and its expectations. The developed standards focus on the learner preparedness, equipment, and evaluation of e-learning to ensure satisfaction and actualise the maximum benefits. Furthermore, sufficient, concise, and clear information contributes immensely to satisfaction.

Fourthly, the myriad of infrastructural e-learning challenges faced by universities in LMICs requires contextually sound standards and comprehensive evaluation, although progress in this regard is evident [85]. Policy issues surrounding e-learning infrastructure were an area of contention as most of the experts agreed that operationalising policies in e-learning is challenging in LMICs. Commitment by key stakeholders is often tenuous. This view was similarly shared by Kibuku et al. (2020) and Mbugua (2014), who identified that policy gaps have hampered higher education in the Kenyan context [74, 86, 87]. The impetus to develop these standards with a focus on policy is to ensure that stakeholder commitment is captured from the start when developing e-learning programmes. The availability of infrastructure to support e-learning was described as challenging for most LMICs, with funding and systemic challenges overshadowing meaningful progress to improve e-learning. Despite the promising internet penetration in LMICs, especially in Africa, the concentration is mostly in urban areas. In addition, the cost of internet access is prohibitive [85]. The learning management systems (LMS) in most developed countries are customised to suit their specific needs, but in LMICs, reliance on open source LMS limits the ability to customisable options [87,88,89]. The ease of use, system quality and satisfaction of learning management systems (LMS) by educators and learners are critical in improving e-learning [88, 89]. We argue for the commitment of governments and university management boards to strengthen the resolve toward improving e-learning which has enormous potential to scale up health workers.

Fifthly, support systems for educators and learners were an area of concern. In most LMICs, there’s is not adequate e-learning support for educators and learners [90]. Wang et al. (2018) identified that support for e-learning resources is an area of interest in e-learning [91]. Experts agreed that e-learning support is a critical and essential standard for evaluating e-learning, and its accessibility is crucial for learners’ satisfaction. Experts agree that e-learning support is essential to the operation of e-learning. There is a need to improve internet bandwidth, although better bandwidth doesn’t necessarily mean that learners and educators will use this for learning purposes. The need for control of content and site restriction is crucial in ensuring appropriate utilisation. According to the findings of Chanboualapha and Islam (2012), who compared the relationship between internet usage and learning and found that the internet has a positive relationship with learners’ knowledge [92]. The absence of social media in their study can be attributed when the study was conducted. However, Waweru (2018) found that learners spend most of their time on social media, during the time which they could be studying, leading to poor academic performance academically [93]. The site access restriction should be so that learning is not impaired. These are some of the areas in which the panel of experts expressed concern, and it was worth noting that the changes to the standards improved the quality and applications to LMICs.

Lastly, these standards will be critical for evaluating the quality of undergraduate e-learning nursing programmes. They will also assist in shaping the intricate capacity to develop local solutions to local problems in LMICs [23]. Based on the iterative process, the experts reached consensus on six broad categories: curriculum planning, proficiency of the educator, learner proficiency and attitude, infrastructure for learning, support, and evaluation, with 104 criteria for evaluating the quality of e-learning in undergraduate nursing programmes in LMICs. These standards will provide schools of nursing seeking to evaluate the quality of e-learning with a guide to practical evaluation. The standards are comprehensive and broader covering wider range of criteria than those available in developed countries. They align with nursing education at a degree level. Consequently, educators and learners will have better learning experiences with e-learning leading to quality in e-learning in nursing education hence improving patient outcomes. The feasibility, usability, and practicability of the standards’ are crucial in ensuring they are easily usable and practical for the context-specific challenges in LMICs. The complete agreement (100%) by the experts (100%) on the final standards’ feasibility, usability, and practicability is critical in applying these standards in LMICs. The crux of this study is standards for quality evaluation. The feasibility, usability, and practicality of these standards have a high propensity to improve the quality of e-learning holistically in LMICs.

Limitations

Despite numerous efforts to recruit more experts from more LMICs, this did not materialise. This hesitation to volunteer may be due to busy work schedules and unavailability and can be explained by the critical roles of most of the experts identified in nursing education and e-learning. A further limitation was that the experts drawn from different LMICs had to be proficient speak in English.

Conclusion

The opportunities provided by e-learning outweigh the challenges as it presents no geographic barriers, flexibility, while accommodating flexibility, creativity and critical thinking. It utilised online resources, offers an effective approach to transferring clinical knowledge, and improves the teaching experience [22, 94, 95]. However, many challenges impact the implementation of e-learning in LMICs. The salient challenges include the lack of a relevant curriculum, lack of infrastructure, lack of ICT knowledge, weakness of content development, educators and learners’ culture and lack of regular online training and seminars for educators and students to support the application of e-learning [94, 96] The rather inconspicuous challenges affecting e-learning are financial, managerial, insufficient professional development, copyright issues, and stakeholder motivation[46, 62, 97, 98]. This last aspect is rarely addressed but has an immense impact on the growth of e-learning in LMICs [63, 64, 97, 98]. Critical reflection of these challenges towards developing sustainable solutions for e-learning is vital in improving the quality of e-learning in LMICs. The standards are essential to evaluating the quality of undergraduate nursing programmes. The identification of these standards culminated in an iterative process with experts drawn from various LMICs with expertise in e-learning and quality assurance and an understanding of the challenges in the field.

The quality of an e-learning programme is evaluated using standards. Undergraduate nursing programmes have a theoretical and clinical component that further complicates such evaluation. The modified Delphi reported in this article refined standards for evaluating the quality of undergraduate nursing e-learning programmes in LMICs. The final product consisted of six standards with 104 criteria. The six standards identified were curriculum planning, proficiency of the educator, learner proficiency and attitude, infrastructure for learning, support and evaluation. These offer critical domains in evaluating the quality of undergraduate nursing programmes, and these are essential areas when evaluating the quality of e-learning. The intricacies of the LMIC context require a thorough understanding of the challenges to meticulously develop feasible, usable, and practical standards. These standards further demonstrate adaptability in low-resource settings. The standards developed are recommended for use in LMICs to evaluate the quality of undergraduate e-learning programmes. Additionally, the standards provide researchers with a basis for further research, with the evolving technological landscape, it will be thought-provoking to see the transitional changes that these standards take over time. Therefore, these developed standards are contextualised to offer sustainable evaluation of the quality of e-learning in LMICs as well as local solutions to local problems. The next steps for these standards are piloting in a local university for feasibility, usability and practicality.

Data Availability

All data generated or analysed during this study are included in this published article [and its supplementary information files].

Abbreviations

COVI9-19:

Corona Virus 2019

CPD:

Continuous Professional Development

ICT:

Information, Communication, and Technology

LMS:

Learning ManagementSsystem

LMIC:

Low- and Middle-Income Countries

References

  1. Drennan V, Ross F. Global nurse shortages the facts, the impact and action for change. Br Med Bull. 2019;130:25–7. https://doi.org/10.1093/bmb/ldz014.

    Article  PubMed  Google Scholar 

  2. World Health Organization. (2016). Global strategy on human resources for health: Work force 2030. https://apps.who.int/iris/rest/bitstreams/1061558/retrieve

  3. Azad A, Min J, Syed S, Anderson S. Continued nursing education in low-income and middle-income countries: a narrative synthesis. BMJ Global Health. 2020;5(2):e001981. https://doi.org/10.1136/bmjgh-2019-001981.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Choi S, Cho E, Kim E, Lee K, Chang SJ. Effects of registered nurse staffing levels, work environment, and education levels on adverse events in nursing homes. Scientific Reports. 2021 Nov 2;11(1). https://doi.org/10.1038/s41598-021-00946-8

  5. Porat-Dahlerbruch J, Aiken LH, Lasater KB, Sloane DM, McHugh MD. Variations in nursing baccalaureate education and 30-day inpatient surgical mortality. Nurs Outlook. 2021 Nov. https://doi.org/10.1016/j.outlook.2021.09.009.

  6. Drasiku A, Gross JL, Jones C, Nyoni CN. Clinical teaching of university-degree nursing students: are the nurses in practice in Uganda ready? BMC Nursing. 2021 Jan 4;20(1). https://doi.org/10.1186/s12912-020-00528-5

  7. Harrison JM, Aiken LH, Sloane DM, Brooks Carthon JM, Merchant RM, Berg RA, et al. In hospitals with more nurses who have Baccalaureate Degrees, Better Outcomes for patients after cardiac arrest. Health Aff. 2019 Jul;38(7):1087–94. https://doi.org/10.1377/hlthaff.2018.05064.

  8. Blegen M, Goode C, Park S, Vaughn T, Spetz J. Baccalaureate education in nursing and patient outcomes. JONA: The Journal of Nursing Administration. 2013;43(2):89–94. https://doi.org/10.1097/NNA.0b013e31827f2028.

    Article  Google Scholar 

  9. Yangoz ST. The use of e-learning program in nursing education. New Trends and Issues Proceedings on Humanities and Social Sciences. 2017 Dec 5;4(2):230. https://doi.org/10.18844/prosoc.v4i2.2752

  10. Agu C, Stewart J, McFarlane-Stewart N, Rae T. COVID‐19 pandemic effects on nursing education: looking through the lens of a developing country. Int Nurs Rev. 2021;68(2):153–8. https://doi.org/10.1111/inr.12663.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Khalil R, Mansour AE, Fadda WA, Almisnid K, Aldamegh M, Al-Nafeesah A et al. The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a qualitative study exploring medical students’ perspectives. BMC Medical Education. 2020 Aug 28;20(1). https://doi.org/10.1186/s12909-020-02208

  12. Valverde-Berrocoso J, Garrido-Arroyo M, del Burgos-Videla C, Morales-Cevallos C. MB. Trends in Educational Research about e-Learning: A Systematic Literature Review (2009–2018). Sustainability. 2020 Jun 24;12(12):5153. https://doi.org/10.3390/su12125153

  13. Misut M, Pribilova K. Measuring of Quality in the Context of e-Learning. Procedia - Social and Behavioral Sciences. 2015 Apr;177:312–9. https://doi.org/10.1016/j.sbspro.2015.02.347

  14. Gautam S, Tiwari M. Components and benefits of E-learning system. International Research Journal of Computer Science (IRJCS) Issue [Internet]. 2016;01. Available from: http://irjcs.com/volumes/Vol3/iss1/03.JAJCS10082.pdf

  15. Risling T. Educating the nurses of 2025: Technology trends of the next decade. Nurse Education in Practice. 2017 Jan;22(22):89–92. https://doi.org/10.1016/j.nepr.2016.12.007

  16. Murebwayire M, Biroli I, Ewing H, Abstract. The Evolution of the Nursing Profession Post Genocide.Rwanda Journal. 2015 Nov17;2(2):106. https://doi.org/10.4314/rj.v2i2.44F

  17. Harerimana A, Mtshali NG. Conceptualisation of e-learning in nursing education in the context of Rwanda. Journal of Nursing Education and Practice. 2020 Mar 5;10(6):26. https://doi.org/10.5430/jnep.v10n6p26

  18. Rouleau G, Gagnon M-P, Côté J, Payne-Gagnon J, Hudson E, Dubois C-A et al. Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews. Journal of Medical Internet Research [Internet]. 2019 Oct 2;21(10):e15118. https://doi.org/10.2196/15118

  19. Paul Maichiki M, Abdulkadir J, Kogi Cl. E-Learning as an Alternative Platform for Enhancing Knowledge and Managerial Skills: Evidence from Ahmadu Bello University in Sub-Saharan Africa. International Journal of Social, Political and Economic Research. 2022 Apr 9;9(1):1–25. https://doi.org/10.46291/IJOSPERvol9iss1pp1-25

  20. Sarli L, Artioli G, Bogotto S, Labelli E, Pittella F, Guasconi M, Simone R, De Luca E, Rossi S, D’Apice C. From classroom training to e-learning: a journey through the quality of learning life of nurse students in post-graduate education - a longitudinal qualitative study. Acta Bio-Medica: Atenei Parmensis. 2022;93(Suppl2):e2022192. https://doi.org/10.23750/abm.v93iS2.13080.

    Article  CAS  PubMed  Google Scholar 

  21. Regmi K, Jones L. A systematic review of the factors – enablers and barriers – affecting e-learning in health sciences education.BMC Medical Education. 2020 Mar30;20(1). https://doi.org/10.1186/s12909-020-02007-6

  22. Li W, Gillies R, He M, Wu C, Liu S, Gong Z et al. Barriers and facilitators to online medical and nursing education during the COVID-19 pandemic: perspectives from international students from low- and middle-income countries and their teaching staff. Human Resources for Health. 2021 May 12;19(1).https://doi.org/10.1186/s12960-021-00609-9

  23. Bvumbwe T, Mtshali N. Nursing education challenges and solutions in Sub Saharan Africa: an integrative review. BMC Nursing. 2018 Jan 31;17(1). https://doi.org/10.1186/s12912-018-0272-4

  24. Thomson P, Richardson A, Foster G. Collaborative learning in the COVID-19 pandemic: a change to the delivery of undergraduate nursing education. Nurs Praxis Aotearoa New Z. 2021 Dec;37(3):34–6. https://doi.org/10.36951/27034542.2021.032.

  25. Kyari S, Adiuku-Brown ME, Abechi H, Adelakun T. E-Learning in tertiary education in Nigeria: where do we stand? Int J Educ Evaluation. 2018;4(9):1–10.

    Google Scholar 

  26. Tarus JK, Gichoya D, Muumbo A. Challenges of implementing e-learning in Kenya: A case of Kenyan public universities.The International Review of Research in Open and Distributed Learning. 2015 Jan20;16(1). https://doi.org/10.19173/irrodl.v16i1.1816

  27. Kotoua S, Ilkan M, Kilic H. The Growing of Online Education in Sub Saharan Africa: Case Study Ghana. Procedia - Social and Behavioral Sciences. 2015 Jun;191:2406–11. https://doi.org/10.1016/j.sbspro.2015.04.670

  28. Nguku A. (2005). Nursing the future: e-learning and clinical care in Kenya. Africa Research Institute- a policy voices series. https://www.africaresearchinstitute.org/newsite/wpcontent/uploads/2013/03/PV-Nursing-the-Future.pdf

  29. Barteit S, Guzek D, Jahn A, Bärnighausen T, Jorge M, and Neuhann F. Evaluation of e-learning for medical education in low- and middle-income countries: A systematic review. Computers & Education. 2020: 145:103726. doi.org/10.1016/j.compedu.2019.103726

  30. Syed S, Rastogi A, Bansal A, Kumar A, Jindal A, Prakash A et al. Future of e-Learning in Medical Education—Perception, Readiness, and Challenges in a Developing Country. Frontiers in Education. 2021 Mar 5;6. https://doi.org/10.3389/feduc.2021.598309

  31. McDonald EW, Boulton JL, Davis JL. E-learning and nursing assessment skills and knowledge – an integrative review. Nurse Educ Today. 2018 Jul;66(1):166–74.

  32. International Standards Organisation 9001 [ISO]. (2015). Quality management systems requirements [internet]. Available from: https://ch97a3a0kw31390rw3nfrzf3-wpengine.netdna-ssl.com/wp-content/uploads/2020/11/ISO-9001-2015-Fifth-Edition.pdf

  33. Roddy C, Amiet DL, Chung J, Holt C, Shaw L, McKenzie S et al. Applying Best Practice Online Learning, Teaching, and Support to Intensive Online Environments: An Integrative Review. Frontiers in Education. 2017 Nov 21;2(59). https://doi.org/10.3389/feduc.2017.00059

  34. Delva S, Nkimbeng M, Chow S, Renda S, Han H-R, D’Aoust R. Views of regulatory authorities on standards to assure quality in online nursing education. Nurs Outlook. 2019 Nov;67(6):747–59. https://doi.org/10.1016/J.OUTLOOK.2019.06.011.

  35. Harerimana A, Mtshali NG. E-learning in nursing education in Rwanda: A middle-range theory. Journal of Nursing Education and Practice. 2021 Mar 30;11(7):78https://doi.org/10.5430/jnep.v11n7p78

  36. Timbi-Sisalima C, Sánchez-Gordón M, Hilera-Gonzalez JR, Otón-Tortosa S. Quality Assurance in E-Learning: A Proposal from Accessibility to Sustainability. Sustainability. 2022 Mar 5;14(5):3052. https://doi.org/10.3390/su14053052

  37. Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals.BMC Medical Education]. 2017 Oct10;17(183). https://doi.org/10.1186/s12909-017-1022-0.

  38. Abuatiq A. E-Learning in nursing: Tool Development for evaluating virtual patient Learning Systems. Teach Learn Nurs. 2019;14(4):291–7. https://doi.org/10.1016/j.teln.2019.06.010.

    Article  Google Scholar 

  39. Garba Y, Afendee Mohamed M, Almaiah M, Abdullahi F. Evaluation of mobile learning framework. International Journal of Research and Innovation in Social Science. 2022. 2454–6186. Available from: https://www.rsisinternational.org/journals/ijriss/Digital-Library/volume-6-issue-1/791-799.pdf

  40. Prosen M, Karnjuš I, Ličen S. Evaluation of E-Learning Experience among Health and Allied Health Professions Students during the COVID-19 Pandemic in Slovenia: An Instrument Development and Validation Study. International Journal of Environmental Research and Public Health. 2022 Apr 14;19(8):4777 https://doi.org/10.3390/ijerph19084777

  41. Frehywot S, Vovides Y, Talib Z, Mikhail N, Ross H, Wohltjen H et al. E-learning in medical education in resource constrained low- and middle-income countries.Human Resources for Health. 2013 Feb4;11(1). https://doi.org/10.1186/1478-4491-11-4

  42. Rawashdeh AZA, Mohammed EY, Arab ARA, Alara M, Al-Rawashdeh B. Advantages and Disadvantages of Using e-Learning in University Education: Analyzing Students’ Perspectives.Electronic Journal of e-Learning. 2021 May17;19(3):107–17. https://doi.org/10.34190/ejel.19.3.2168

  43. Baker C, Cary A, da Conceicao Bento M. 2021. Global standards for professional nursing education: The time is now. Journal of Professional Nursing.2021: 37(1), 86–92.https://doi.org/10.1016/j.profnurs.2020.10.001

  44. Wasfy NF, Abouzeid E, Nasser AA, Ahmed SA, Youssry I, Hegazy NN et al. A guide for evaluation of online learning in medical education: a qualitative reflective analysis. BMC Medical Education. 2021 Jun 10;21(1). https://doi.org/10.1186/s12909-021-02752-2

  45. Cluster R, Scarcella J, Stewart R. The modified Delphi- A rotational modification.Journal of vocational and technical education. 1999; 15(2). https://files.eric.ed.gov/fulltext/EJ590767.pdf

  46. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000 Oct;32(4):1008–15. https://doi.org/10.1046/j.1365-2648.2000.t01-1-01567.x.

  47. Niederberger M, Spranger J. Delphi Technique in Health Sciences: A Map.Frontiers in Public Health. 2020 Sep22;8. https://doi.org/10.3389/fpubh.2020.00457

  48. Vishwanath S, Begum H. Modified Delphi methodology [internet]. Available from: https://www.monash.edu/__data/assets/pdf_file/0010/1757134/Registry-SIG_15032019_Modified-Delphi-Methodology.pdf

  49. Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness.World Journal of Methodology. 2021 Jul20;11(4):116–29. https://doi.org/10.5662/wjm.v11.i4.116

  50. Brown R, Crookes P. What level of competency do experienced nurses expect from a newly graduated registered nurse? Results of an australian modified Delphi study. BMC Nurs. 2016;15(1). https://doi.org/10.1186/s12912-016-0166-2.

  51. Stewart D, Gibson-Smith K, MacLure K, Mair A, Alonso A, Codina C et al. A modified Delphi study to determine the level of consensus across the European Union on the structures, processes and desired outcomes of the management of polypharmacy in older people. Leong C, editor. PLOS ONE. 2017 Nov 20;12(11):e0188348. https://doi.org/10.1371/journal.pone.0188348

  52. Woodcock T, Adeleke Y, Goeschel C, Pronovost P, Dixon-Woods M. A modified Delphi study to identify the features of high quality measurement plans for healthcare improvement projects.BMC Medical Research Methodology. 2020 Jan14;20(1). https://doi.org/10.1186/s12874-019-0886-6

  53. Adebiyi B, Mukumbang F, Okop K, Beytell A. A modified Delphi study towards developing a guideline to inform policy on fetal alcohol spectrum disorders in South Africa: a study protocol. BMJ Open. 2018;8(4):e019907. https://doi.org/10.1136/bmjopen-2017-019907.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Keeney S, Hasson F, Mckenna HP. The Delphi technique in nursing and health research. West Sussex, U.K.; Ames, Iowa: Wiley-Blackwell;: Chichester; 2011.

    Book  Google Scholar 

  55. Al-Fraihat D, Joy M, Masa’deh R, Sinclair J. Evaluating E-learning systems success: an empirical study. Comput Hum Behav. 2020;102:67–86. https://doi.org/10.1016/j.chb.2019.08.004.

    Article  Google Scholar 

  56. Alotaibi R, Alshahrani S. An extended DeLone and McLean’s model to determine the success factors of e-learning platform. PeerJ Comput Sci. 2022;8:e876. https://doi.org/10.7717/peerj-cs.876.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Delone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: a ten-year update. J Manage Inform Syst. 2003 Apr;19(4):9–30. https://doi.org/10.1080/07421222.2003.11045748.

  58. Abdullah F, Ward R. Developing a General Extended Technology Acceptance Model for E-Learning (GETAMEL) by analysing commonly used external factors. Comput Hum Behav. 2016;56:238–56. https://doi.org/10.1016/j.chb.2015.11.036.

    Article  Google Scholar 

  59. Sun P-C, Tsai RJ, Finger G, Chen Y-Y, Yeh D. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction.Computers & Education. 2008May;50(4):1183–202. https://doi.org/10.1016/j.compedu.2006.11.007

  60. Olayemi E, Asare EV, Benneh-Akwasi Kuma AA. Guidelines in lower-middle income countries.British Journal of Haematology. 2017 Mar14;177(6):846–54.https://doi.org/10.1111/bjh.14583

  61. Naciri A, Radid M, Kharbach A, Chemsi G. E-learning in health professions education during the COVID-19 pandemic: a systematic review.Journal of Educational Evaluation for Health Professions. 2021 Oct29;18:27. https://doi.org/10.3352/jeehp.2021.18.27

  62. O’Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, McGrath D. Barriers and solutions to online learning in medical education – an integrative review. BMC Med Educ. 2018;18(1). https://doi.org/10.1186/s12909-018-1240-0. Jun 7.

  63. Shahmoradi L, Changizi V, Mehraeen E, Bashiri A, Jannat B, Hosseini M. The challenges of e-learning system: higher educational institutions perspective. J Educ Health Promotion. 2018;7:116. https://doi.org/10.4103/jehp.jehp_39_18.

    Article  Google Scholar 

  64. Shrivastava S, Shrivastava P. Challenges Associated with the introduction of E-learning in Medical Education and Strategies to Counter them. MAMC J Med Sci. 2019;5(1):43. https://doi.org/10.4103/mamcjms.mamcjms_17_19.

    Article  Google Scholar 

  65. Oliver R. Quality assurance and e-learning: blue skies and pragmatism. ALT-J. 2005 Oct;13(3):173–87. https://doi.org/10.1080/09687760500376389.

  66. Zalat MM, Hamed MS, Bolbol SA. The experiences, challenges, and acceptance of e-learning as a tool for teaching during the COVID-19 pandemic among university medical staff. PLoS ONE. 2021 Mar;26(3):e0248758. https://doi.org/10.1371/journal.pone.0248758.

  67. Widyanti A, Hasudungan SH, Park J. E-learning readiness and perceived learning workload among students in an Indonesian university. Knowledge Management & E-Learning: An International Journal. 2020: 12(1):18–29. https://doi.org/10.34105/j.kmel.2020.12.002

  68. Nyerere J. Kenya’s university students and lecturers face huge challenges moving online [Internet]. The Conversation. Available from: https://theconversation.com/kenyas-university-students-and-lecturers-face-huge-challenges-moving-online-136682

  69. Viana J, Peralta H. Online learning: from the curriculum for all to the curriculum for each individual. J New Approaches Educational Res. 2021;10(1):122–36. https://doi.org/10.7821/naer.2021.1.579.

    Article  Google Scholar 

  70. Adirika B, Okolie CV. Examining models of curriculum development and processes: implications for african educational heritage and review. Social Sci Humanit J. 2020;17(6):325–42. https://sshj.in/index.php/sshj/article/view/44/36.

    Google Scholar 

  71. Togar Alam Napitupulu YC. Evaluation of Student Satisfaction in Using the Learning Management System for Online Learning at XYZ University. Turkish Journal of Computer and Mathematics Education (TURCOMAT). 2021 Apr 5;12(6):2810–6. https://doi.org/10.17762/turcomat.v12i6.5788

  72. Susam T, Durnali M, Orakci Ş. Administering Education and Training Through a Web-Based System. Utilizing Technology, Knowledge, and Smart Systems in Educational Administration and Leadership. 2020;20–141. https://doi.org/10.4018/978-1-7998-1408-5.ch002

  73. Al Shdaifat S, Alazemi A, Al-Jamal D, Zafarani P. e-Curriculum as perceived by Teachers: Jordan as an Example. Educ Res Int. 2022;2022:1–12. https://doi.org/10.1155/2022/8508216.

    Article  Google Scholar 

  74. Kibuku R, Ochieng D, Wausi A. e-Learning challenges faced by universities in Kenya: A literature review. Electron J e-Learning. 2020;18(2). https://doi.org/10.34190/EJEL.20.18.2.004.

  75. Lwoga E. Critical success factors for adoption of web-based learning management systems in Tanzania. International Journal of Education and Development using ICT [Internet]. 2014 May 6 [cited 2021 Sep 14];10(1). Available from: https://www.learntechlib.org/p/147447/

  76. Almahasees Z, Mohsen K, Amin M. Faculty’s and Students’ Perceptions of Online Learning During COVID-19. Frontiers in Education. 2021;6. doi.org.10.3389/feduc.2021.638470

  77. Hasan M, Maarop N, Samy GN, Baharum HI, Abidin WZ, Hassan NH. Developing a success model of Research Information Management System for research affiliated institutions. 2017 International Conference on Research and Innovation in Information Systems (ICRIIS). 2017 Jul; https://doi.org/10.1109/ICRIIS.2017.8002444

  78. Goh CF, Hii PK, Tan OK, Rasli A. Why do University Teachers use E-Learning Systems? The International Review of Research in Open and Distributed Learning. 2020 Apr 8;21(2):136–55. https://doi.org/10.19173/irrodl.v21i2.3720

  79. Mailizar M, Almanthari A, Maulina S. Examining Teachers’ Behavioral Intention to Use E-learning in Teaching of Mathematics: An Extended TAM Model. Contemporary Educational Technology.2021 Feb13;13(2):ep298. https://doi.org/10.30935/cedtech/9709

  80. Mlambo M, Silén C, McGrath C. Lifelong learning and nurses’ continuing professional development, a metasynthesis of the literature. BMC Nursing. 2021 Apr 14;20(62):1–13. https://doi.org/10.1186/s12912-021-00579-2

  81. McGill TJ, Klobas JE. A task–technology fit view of learning management system impact. Comput Educ. 2009 Feb;52(2):496–508. https://doi.org/10.1016/j.compedu.2008.10.002.

  82. Hassanzadeh A, Kanaani F, Elahi S. A model for measuring e-learning systems success in universities. Expert Syst Appl. 2012 Sep;39(12):10959–66. https://doi.org/10.1016/j.eswa.2012.03.028.

  83. Islam AKMN. Investigating e-learning system usage outcomes in the university context. Comput Educ. 2013 Nov;69:387–99. https://doi.org/10.1016/j.compedu.2013.07.037.

  84. Penna M, Stara V. The failure of e-learning: why should we use a learner centred design. Journal of e-Learning and Knowledge Society [Internet]. 2007 [cited 2022 Oct 18];3(2):127–35. Available from: https://www.learntechlib.org/p/43414

  85. Joseph M. Challenges of educational digital infrastructure in Africa: A tale of hope and disillusionment.Journal of African Studies and Development. 2019 Jul31;11(5):59–63. https://doi.org/10.5897/JASD2019.0539

  86. Mbugua M. Elearning in Kenya: the Challenge of Policy Making. Internet]. [cited 2022 Oct 18]; Available from: https://www.academia.edu/11545537/Elearning_in_Kenya_the_Challenge_of_Policy_Making

  87. Esmaily HM, Savage C, Vahidi R, Amini A, Zarrintan MH, Wahlstrom R. Identifying outcome-based indicators and developing a curriculum for a continuing medical education programme on rational prescribing using a modified Delphi process.BMC Medical Education. 2008 May30;8(1). https://doi.org/10.1186/1472-6920-8-33

  88. Thuseethan S, Kuhanesan S. Usability evaluation of learning management systems in sri lankan universities. Global J Comput Sci Technol. 2014;15(1):1–13. https://doi.org/10.48550/arXiv.1412.0197.

    Article  Google Scholar 

  89. Chandra Y, Napitupulu TA. Evaluation of student satisfaction in using learning management system for online learning at XYZ university. Turkish J Comput Math Educ. 2021;12(6):2810–1816. https://doi.org/10.17762/turcomat.v12i6.5788.

    Article  Google Scholar 

  90. Nawaz A, Zubair Khan M. Issues of Technical Support for e-Learning Systems in Higher Education Institutions. Int J Mod Educ Comput Sci. 2012 Mar;9(2):38–44. https://doi.org/10.5815/ijmecs.2012.02.06.

  91. Wang Y, Liu X, Zhang Z. An overview of e-learning in China: History, challenges and opportunities.Research in Comparative and International Education. 2018Mar;13(1):195–210. https://doi.org/10.1177/1745499918763421

  92. Chanboualapha S, Islam M. Internet usage for improvement of learning. Unpublished Master’s degree. Boras: University of Boras. 2012 http://www.diva-portal.org/smash/get/diva2:1308488/FULLTEXT01.pd

  93. Waweru GW. Effects Of Social Media On Students Academic Performnace In Tertiary Education Institutions In Thika Municipality, Kiambu County, Kenya [Internet]. erepository.uonbi.ac.ke. 2018 [cited 2022 Oct 18]. Available from: http://erepository.uonbi.ac.ke/handle/11295/105295

  94. Gama LC, Chipeta GT, Chawinga WD. Electronic learning benefits and challenges in Malawi’s higher education: A literature review. Education and Information Technologies. 2022 May 3. https://doi.org/10.1007/s10639-022-11060-1

  95. Maatuk AM, Elberkawi EK, Aljawarneh S, Rashaideh H, Alharbi H. The COVID-19 pandemic and E-learning: challenges and opportunities from the perspective of students and instructors. Journal of Computing in Higher Education. 2021 May 3;34(1). https://doi.org/10.1007/s12528-021-09274-2

  96. Eli-Chukwu NC, Igbokwe IC, Ifebude B, Nmadu D, Iguodala W, Uma U et al. Challenges confronting e-learning in higher education institutions in Nigeria amid Covid – 19. Journal of Applied Research in Higher Education. 2022 Feb 11; https://doi.org/10.1108/JARHE-09-2021-0346

  97. Aung T, Khaing S. Challenges of implementing e-Learning in developing countries: a review. Adv Intell Syst Comput. 2015;405–11. https://doi.org/10.1007/978-3-319-23207-2_41.

  98. Aiken L, Sloane D, Bruyneel L, Van den Heede K, Griffiths P, Busse R, et al. Nurse staffing and education and hospital mortality in nine european countries: a retrospective observational study. The Lancet. 2014;383(9931):1824–30. https://doi.org/10.1016/S0140-6736(13)62631-8.

    Article  Google Scholar 

Download references

Acknowledgements

Prof. Ruth Albertyn: Critical reading.

Annamarie du Preez: Language editing.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of Nursing, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa

    Moses Mutua Mulu & Champion N. Nyoni

Authors
  1. Moses Mutua Mulu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Champion N. Nyoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Moses Mutua Conceptualisation, methodology, writing original draft, writing-review, and editing. Champion Nyoni Conceptualisation, methodology, writing original draft, writing-review, and editing.

Corresponding author

Correspondence to Moses Mutua Mulu.

Ethics declarations

Ethical approval

The research confirms that all methods were carried out in accordance with the Declaration of Helsinki. Ethical approval was sought from the University of Free State, Health Sciences Research Ethics Committee (HSREC) clearance number UFS-HSD2021/0370/2809. The experts signed an informed consent form to participate in the study (see Additional file 2) and were free to withdraw from the panel at any time during the discussions without victimization.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no known conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mulu, M.M., Nyoni, C.N. Standards for evaluating the quality of undergraduate nursing e-learning programme in low- and middle-income countries: a modified Delphi study. BMC Nurs 22, 73 (2023). https://doi.org/10.1186/s12912-023-01235-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12912-023-01235-7

Keywords

BMC Nursing

ISSN: 1472-6955

Contact us