Skip to main content
Download PDF

Testing the validity and reliability of the Korean nursing surveillance scale: a methodological study

BMC Nursing volume 23, Article number: 709 (2024) Cite this article

Abstract

Background

Because of the importance of nursing surveillance, there is a need to develop a scale to measure nursing surveillance that reflects the roles of nurses in South Korea. This study aimed to develop a scale to measure surveillance by Korean nurses and to test its reliability and validity.

Methods

In the development phase, a literature review was conducted to verify the attributes of nursing surveillance, and preliminary items were developed based on the surveillance activities in the Nursing Intervention Classification (NIC) and the interviews of Korean nurses and modified through content validation and a pilot study. In the psychometric testing phase, two surveys were conducted with Korean nurses working in acute hospitals, using the preliminary scale in exploratory factor analysis (EFA, n = 220) and confirmatory factor analysis (CFA, n = 219). Data were analyzed through EFA, CFA, correlation, and reliability analyses to verify convergent validity, discriminant validity, criterion validity, and reliability. To verify the validity of the preliminary scale, the exploratory factor analysis and confirmatory factor analysis convergent validity, discriminant validity, criterion validity, and reliability were performed.

Results

In the EFA, 16 items were grouped into four factors, accounting for 70.1% of the cumulative variance. In the CFA, the model exceeded the criteria for all fit indices (χ2 = 155.62 [df = 94, p < .001], CMIN = 1.65, SRMR = .048, RMSEA = .055, GFI = .921, NFI = .916, TLI = .955, CFI = .964) and was acceptable. The convergent validity, discriminant validity, criterion validity, and reliability were verified. The final Korean nursing surveillance scale consists of four factors: ‘anticipation of problems and decision-making’ with six items; ‘systematic assessment’ with five items; ‘recognition of patterns’ with three items; and ‘identification of the patient’s self-care and coping strategies’ with two items.

Conclusion

The Korean nursing surveillance scale developed in this study comprised questions that included NIC’s surveillance activities and empirical data from Korean nurses; based on the attributes of nursing monitoring derived from concept analysis, its validity and reliability were verified. This study can provide precedent to motivate the development of nursing surveillance scales in other countries, and ultimately stimulate studies on nursing surveillance, which is essential for patient safety.

Peer Review reports

Background

In hospitals, nurses are the most suitable providers to prevent patients’ complications, identify risks, and respond appropriately [1, 2]. Nightingale used the term ‘monitoring’ to describe nurses who collect data through observation to protect patients’ lives, prevent risks, and improve outcomes [3]. However, monitoring excludes cognitive analysis and decision-making. Surveillance differs from monitoring based on its objectives, approaches, data sources, and analyses. Surveillance includes the evaluation of monitoring indices, and acquisition, integration, and interpretation of information from sources such as caregivers, healthcare teams, databases, and clinical decision support systems [2, 4].

Surveillance by nurses is considered a defense mechanism to protect patients from harmful incidents [5]. Nurses frequently perceive negative changes in patients by checking and monitoring vital signs before discovering objective evidence of their worsening condition [6]. Systematic surveillance by nurses assists them in recognizing and interrupting dangerous situations [7]. Therefore, nursing surveillance is a core intervention for early detection of harmful incidents and error prevention [8].

NIC [9] defines nursing surveillance as ‘the purposeful and ongoing acquisition, interpretation, and synthesis of patient data for clinical decision-making.’ Specifically, nursing surveillance in hospitals during the acute phase refers to the process of collecting, reviewing, interpreting, analyzing, and evaluating data to identify and prevent potential complications [4]. While performing surveillance, nurses interact with patients and their families and continually collect and evaluate various types of data, including physiological indices, responses to intervention, and laboratory and diagnostic test results [4]. Therefore, surveillance is a goal-oriented, systematic process focused on early detection of worsening conditions, recognition of changes, and rapid and appropriate interventions [3, 6].

In a recent concept analysis of nursing surveillance, the attributes associated with nursing surveillance were ‘systematic process,’ ‘pattern recognition,’ ‘anticipation of problems,’ ‘coordinated communication,’ and ‘decision-making’ [8].

In previous literature reviews, nursing competence [10], knowledge, training, education, professionalism, confidence, and certification have been suggested as antecedents of nursing surveillance [5, 11, 12]. Furthermore, a culture of surveillance support, interdisciplinary communication, having nurses, a clinical ladder, nurse staffing, communication tools, interdisciplinary protocols, and usability of emergency services have been suggested as work environment-related antecedents affecting nursing surveillance [5, 13].

Appropriate nursing surveillance can result in decreases in the incidence of complications, hazardous events, and nurse burnout, and increased nurse and patient satisfaction [8, 14]. Although increased nursing surveillance leads to additional costs for hospitals, patients who receive nursing surveillance interventions at least 12 times per day exhibit significantly lower incidence of falls, and a reduction in treatment costs owing to falls [2, 15]. Additionally, cardiac arrest survival was higher among patients who received nursing surveillance and had their vital signs checked frequently according to surveillance-related nursing diagnosis [4]. Patients who received a surveillance-related nursing diagnosis had their vital signs measured more often and showed a higher recovery rate from cardiac arrest [16].

Kelly and Vincent conducted a literature review and analyzed the nursing surveillance concept [17]. Based on their findings, they defined surveillance in hospitals in the acute phase as the primary identification of patient health and safety risks through intentional and ongoing acquisition, interpretation, and synthesis of patient data for clinical decision-making. Additionally, they recognized the multidimensional characteristics of nursing surveillance and reported that surveillance consisted of behavioral factors, such as examination, diagnosis, and intervention, and cognitive factors, such as clinical judgment and decision-making [17]. Dresser described the attributes of surveillance as ongoing examination, observation, perception of changes, interpretation, and decision-making, and reported that the cognitive ability to recognize patterns and detect threats or risks to patients is the core factor of surveillance [4]. Jahrsdoerfer [18] suggested that attributes of clinical surveillance were attention, timeliness, recognition, analysis, action, and collaboration. Conversely, Giuliano differentiated nursing surveillance from monitoring, describing the surveillance’s essential factors as ongoing observation, recognition, interpretation, and decision-making [6]. Subsequently, based on a literature review, Halverson and Tilley derived the attributes of nursing surveillance to be ‘systematic process,’ ‘pattern recognition,’ ‘anticipation of problems,’ ‘coordinated communication,’ and ‘decision-making.’ [8]. They claimed that early alert systems that anticipate problems reflect the attributes of nursing surveillance.

Recently, Kim and Cho [19] conducted a concept analysis of nursing surveillance in Korean nurses using the hybrid model approach of Schwartz-Barcott and Kim [20] and reported that nurses were able to mentally recall an image of their patients and identify the required data to understand the overall scenario [21]. Nursing surveillance’s attributes were reported to be ‘systematic assessment,’ ‘pattern recognition,’ ‘anticipation of problems,’ ‘effective communication,’ ‘decision-making,’ and ‘nursing practice.’ Nurses reported that, by performing appropriate surveillance, they could prevent critical situations, respond rapidly to emergencies, aid patient recovery, reduce length of stay, prevent unnecessary treatment, increase nursing satisfaction, increase self-confidence and satisfaction, and improve trust in hospitals [19].

Despite the recognition of the importance of nursing surveillance, there is a shortage of methods to empirically measure surveillance [22]. Kelly measured surveillance activities using NIC-validated activities as an instrument to examine surveillance as a multidimensional concept [3]. However, the validity of this instrument was not reported, and the authors emphasized the need to develop an instrument that reflects the multidimensional attributes of nursing surveillance [17]. In particular, considering the social and technical differences in nurses’ roles within healthcare organizations between countries [23], differences in the content and extent of nursing surveillance activities are expected. Therefore, necessitating the development of an instrument to measure nursing surveillance that reflects the roles of nurses in South Korea, where the term ‘surveillance’ as an NIC intervention is not commonly used [19]. Consequently, this study aimed to develop an instrument to measure surveillance by Korean nurses and to test its reliability and validity.

Methods

Research design and procedure

This study design was a methodological approach. Following DeVellis’ guidelines [24] for scale development, this study was divided into a development phase, which involved the development of an instrument to measure nursing surveillance, and a psychometric testing phase, which tested the validity of the instrument.

Development phase

Verifying the conceptual framework and selecting constituent factors

The research team performed a literature review of the concept analysis of nursing surveillance [4, 8, 17,18,19]. The attributes of nursing surveillance were integrated into the following five factors: ‘systematic assessment,’ ‘pattern recognition and problem prediction,’ ‘decision making,’ ‘communication,’ and ‘nursing performance’ to develop the Korean nursing surveillance scale.

Preliminary item composition

The preliminary items for the Korean nursing surveillance scale were developed based on the 46 activities whose validity has been verified in NIC 7th edition [9]; these activities have been used to measure the surveillance activities of Korean nurses in a previous study [2]. Furthermore, reflecting the surveillance of Korean nurses, the research team extracted 12 activities expressing the attributes of nursing surveillance, which were not part of the 46 items, from interview data on the surveillance experience of nurses in acute care hospitals [19].

Content validity

An expert panel consisting of two nursing professors with experience in scale development, four nurse managers, and four clinical experts was formed, and the panel rated the content validity of the 58 preliminary items on a Likert scale ranging from 1 (‘unrelated’) to 4 (‘related’). Items with a content validity score below 0.80 were removed [25].

Pilot study

A pilot study was conducted with 10 nurses working at two general hospitals and two advanced general hospitals. They were administered a questionnaire that included items on general characteristics, the preliminary Korean nursing surveillance scale, and the 21 activities of surveillance included in the NIC 8th edition [26], recently revised and shortened, to test convergent validity. The items were scored on a Likert scale ranging from 0 (‘never performed’) to 4 (‘always performed’).

Psychometric testing phase

Participants and data collection

Convenience sampling was used to select two general hospitals and two advanced general hospitals with at least 300 beds. The minimum number of participants required to verify the validity of the tool was calculated based on reports suggesting that at least five participants per item are required for exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) [27], and that a sample size of at least 200 participants is suitable [24]. A total of 462 participants were recruited, assuming a dropout rate of 5%. Two hundred thirty one participants were assigned to Samples 1 and 2 through random sampling using the SPSS program. For EFA, 11 insincere responses were excluded, and the remaining 220 responses were included in the analysis. For CFA, 12 insincere responses were excluded, and the remaining 219 responses were included in the analysis. The number of participants in this study was sufficient to verify validity.

The objectives and methods of the study were explained, and permission for a questionnaire survey was obtained from the hospitals. Subsequently, explanation sheets were distributed to potential participants in the nursing units, and the study objectives and methods were explained. The participants included nurses, who performed nursing directly while working in the hospital. Outpatient nurses and nurse managers who did not directly care for patients were excluded. Participants who voluntarily consented to participate in the survey were provided with IRB-approved explanation sheets and questionnaires and were asked to complete them anonymously and submit them in separate envelopes. Respondents’ phone numbers were collected separately to provide a small token of appreciation, after which the numbers were deleted.

Research instruments

The examined general characteristics included sex, age, education, marital status, total career, current department career, and position. The examined work characteristics included working department, nurse staffing (number of beds/number of registered nurses), and nursing care delivery method.

To test the criterion validity of the Korean nursing surveillance scale, an instrument consisting of surveillance activities from the NIC 8th edition [26], validated as a standardized classification system for nursing interventions in North America, was used. The authors and publishers granted permission to use 21 activities from the revised NIC 8th edition. A professional translation service translated the English items into Korean, reverse-translated the items back into English to verify the validity of the translation, and the Korean expression of items was reviewed by a Korean literature Professor. The 21 items were measured on a Likert scale ranging from 0 (‘never performed’) to 4 (‘always performed’), with higher scores indicating a higher level of performance of surveillance activities. Regarding reliability, Cronbach’s α was 0.92 in this study.

Data analysis

The collected data were analyzed using IBM SPSS AMOS and Statistics 27.0 program (IBM, Armonk, NY, USA). The participants’ general characteristics were analyzed using descriptive statistics. The mean, standard deviation, skewness, and kurtosis of each item were analyzed to verify the data’s suitability. Before performing factor analysis, the KMO statistic and Bartlett’s test were used to verify whether the data satisfied the minimal conditions. During the exploratory factor analysis, principal component analysis was performed, in which the principal components (factors) that could maximally explain the total variance in the measured variables were extracted, along with Varimax rotation, in which the factor structure is rotated while maintaining the orthogonality (independence) of the factors. While verifying whether any factors for surveillance attributes and any items were significantly associated with certain factors [25], items for which the communality and factor loading were both ≥ 0.40 were extracted [27]. During the CFA, the model fit was determined based on the chi-square minimum/degree of freedom (CMIN/DF), goodness-of-fit-index (GFI), normed fit index (NFI), Tucker-Lewis index (TLI), comparative fit index (CFI), standardized root-mean-square residual (SRMR), and root mean square error of approximation (RMSEA) [28, 29]. The modification index was examined and the covariance of the error was analyzed to improve the model fit [30]. Convergent validity was analyzed using the normalized factor loading values, average variance extracted (AVE), and construct reliability (CR) of the items extracted from the CFA. Discriminant validity was analyzed using the AVE of the items and the squares of the correlation coefficients between each factor to determine the differentiability between the constituent concepts [31]. Regarding criterion validity, Pearson’s correlation coefficients were used to analyze the correlations with the 21-activity list from the NIC [26]. Cronbach’s α was used to verify the internal consistency of the scale. Statistical significance was set at < 0.05.

Ethical considerations

This study was conducted after receiving approval from the institutional review board of the authors’ affiliated institution, C-University (IRB No. 7001066-202304-HR-016). The explanation sheet provided to the nurses included the study’s objectives and methods and clarified that participation was voluntary, that anonymity would be guaranteed, that participation could be withdrawn at any time without consequences, and that the data would only be used for research purposes. Questionnaires were only distributed to nurses who read the explanation sheet and voluntarily consented to participate. The nurses who participated in the survey were provided with a small gift as a token of appreciation. The collected data were anonymously coded using an Excel file in accordance with a coding book. The coding file was password-protected, and the original questionnaire responses were stored in a locked cabinet.

Results

Development phase

Preliminary item composition

The preliminary items for the Korean nursing surveillance scale were developed based on the 46 activities and 12 activities from interview data of the surveillance experience of Korean nurses [19]. Twelve items were added into five factors as follows: two items (‘Through handover, the patient’s condition is determined,’ ‘During the rounding, check directly whether the information handed over is accurate’) were assigned to ‘systematic assessment’; five items (‘Comprehensively analyze handover data and assessed data,’ ‘Interpret the meaning of the assessed data,’ ‘Compare past and present data to identify patterns in the patient’s condition,’ ‘Evaluate the effectiveness of treatments and interventions and predict problems that may arise’,’ ‘Predict problems by comprehensively judging patient data before the patient’s condition worsens’) were assigned to ‘pattern recognition and anticipation of problems’; two items (‘Prioritize problems to be solved during the shift,’ ‘Participates in decision-making regarding the patient’s treatment plan’) were assigned to ‘decision-making’’ two items (‘Effectively coordinate communication between patients, guardians, and nurses,’ ‘Identify problems resulting from changes in the patient’s condition and report them to the doctor’) were assigned to ‘communication’; and one item (‘Anticipate and prepare the doctor’s prescription’) was assigned to ‘nursing performance.’ Consequently, 58 preliminary items were developed.

Content validity

In the first round, 12 items with an item content validity index (I-CVI) < 0.80 were removed [25], and two items were combined into a single item, resulting in a total of 45 items. In the second round, one additional item (‘Monitor behavioral patterns’), which had a CVI of < 0.80, was removed to develop a preliminary Korean nursing surveillance scale with 44 items.

Pilot study

The comprehensibility, suitability, and clarity of the scale were verified with no suggested changes. The survey took approximately 10–20 min. The items were scored on a Likert scale ranging from 0 (‘never performed’) to 4 (‘always performed’). The score was calculated as the average of the total score of all items, with higher scores indicating higher performance of nursing surveillance. Therefore, the 44 items for the nursing surveillance scale were finalized.

Psychometric testing phase

Participants’ general characteristics

This study was divided into participants for EFA and CFA. EFA revealed that the average age of the participants was 29.83 ± 5.97 years, 204 (92.7%) were women, 170 (81.3%) had a bachelor’s degree, and 162 (73.6%) were unmarried. Regarding work-related characteristics, the average total career length was 5.98 ± 5.82, the average current department career length was 4.16 ± 3.94 years, 206 (93.6%) participants were staff nurses, and 206 (93.6%) worked in the internal medicine/general surgery department. The nurse staffing was 1.71 ± 1.71, and 191 (87.6%) participants engaged in team nursing as the nursing care delivery method.

CFA revealed that the average age of the participants was 30.45 ± 6.39 years, 205 participants were women (93.6%), 161 graduated college (75.6%), and 153 were unmarried (70.2%). Regarding work-related characteristics, the average total career length was 6.69 ± 6.48 years, the average current department career length was 1.55 ± 1.22 years, 202 participants were staff nurses (92.7%), and 126 participants worked in the internal medicine/general surgery department (58.1%). The nurse staffing was 1.55 ± 1.22, and 145 (66.2%) participants engaged in team nursing as the nursing care delivery method (Table 1).

Table 1 Participant characteristics (N = 439)
Full size table

Validation

Suitability of data and item analysis

Before testing validity, the suitability of the data was examined based on the mean, standard deviation, skewness, and kurtosis of the 44 nursing surveillance items. The mean scores ranged between 2.52 and 3.90 and the standard deviations ranged between 0.30 and 1.03. Based on the skewness criteria with an absolute value ≥ 3 or kurtosis with an absolute value ≥ 7, ‘Q1. Ascertain the patient’s condition during handover,’ was removed. The remaining 43 items showed skewness ranging between − 0.33 and − 2.69 and kurtosis ranging between − 0.79 and 5.26. The correlations of each item with all 43 items ranged between 0.44 and 0.76, and the reliability of the 43 items was 0.96. The reliability showed little change after the items were deleted, ranging between 0.960 and 0.962. Consequently, the 43 items were used in the EFA.

Construct validity

Exploratory factor analysis

After excluding one item during item analysis, the Kaiser-Meyer-Olkin (KMO) statistic and Bartlett’s sphericity were calculated to determine whether the remaining 43 items were suitable for factor analysis. The KMO was 0.93 and Bartlett’s sphericity was χ2 = 6748.17 (p < .001), indicating that the items were suitable for factor analysis [32]. In the first round of the EFA, based on the criteria of 0.4 for communality and 0.4 for factor loading, 43 items in seven factors had an eigenvalue of ≥ 1.0, and the cumulative variance was 66.0%. In the second round of the EFA, there were 29 items in six factors, and the cumulative variance was 66.1%. In the third round of the EFA, there were 25 items in five factors, and the cumulative variance was 64.1%. In the fourth round of the EFA, there were 24 items in five factors, and the cumulative variance was 65.1%. In the fifth round of the EFA, there were 21 items in four factors, and the cumulative variance was 63.4%. In the sixth round of the EFA, there were 18 items in four factors, and the cumulative variance was 65.8%. In the seventh round of the EFA, there were 16 items in four factors, and the cumulative variance was 70.1%. The scale was finalized at 16 items, with six items in ‘Anticipation of problems and decision-making,’ five items in ‘Systematic assessment,’ three items in ‘Recognition of patterns,’ and two items in ‘Identification of patient’s self-care and coping strategies’ (Table 2).

Table 2 Factor loadings by exploratory factor analysis (n = 220)
Full size table

Confirmatory factor analysis

CFA was performed to determine whether the 4-factor structure from EFA was suitable after a normality test. All factor loadings in the CFA were ≥ 0.40 (range: 0.58–0.93). To examine model fit, χ2, CMIN, SRMR (≤ 0.05), RMSEA (≤ 0.08), GFI (≥ 0.90), TLI (≥ 0.90), and CFI (≥ 0.90) were tested [33]. The nursing surveillance model exceeded the criteria for all fit indices (χ2 = 155.62 [df = 94, p < .001], CMIN = 1.65, SRMR = 0.048, RMSEA = 0.055, GFI = 0.921, NFI = 0.916, TLI = 0.955, CFI = 0.964). Based on these results, the model was acceptable.

Convergent validity

The criteria used included item-total correlation ≥ 0.40, AVE ≥ 0.50, and CR ≥ 0.70 [34]. The item-total correlations ranged between 0.48 and 0.80, the AVE ranged between 0.52 and 0.60, and the CR ranged between 0.74 and 0.88, indicating that the 16-item nursing surveillance scale satisfied the convergent validity criteria (Table 3).

Table 3 Convergent validity of nursing surveillance scale (n = 219)
Full size table

Discriminant validity

To ensure discriminant validity, the AVE of each factor has to be larger than the square of the correlation coefficient between factors (Φ2) and the 95% confidence interval [Φ ± 2 × SE] of the correlation coefficient (Φ) should not include 1.00 [34]. Given that the AVE of each factor was larger than the square of the correlation coefficient between factors (Φ2) and the 95% confidence interval [Φ ± 2 × SE] of the correlation coefficient (Φ) did not include 1.00, the scale’s discriminant validity was confirmed (Table 4).

Table 4 Discriminant validity (n = 219)
Full size table

Reliability

Regarding the nursing surveillance scale’s internal consistency, Cronbach’s α was 0.90 and the reliability per factor was high, at 0.87 for Factor 1 (6 items), 0.84 for Factor 2 (5 items), 0.82 for Factor 3 (3 items), and 0.71 for Factor 4 (2 items), demonstrating the scale’s internal consistency (Table 4).

Criterion validity

The nursing surveillance scale exhibited strong positive correlations (r = .90, p < .001) with the 21-item NIC list of surveillance activities. Therefore, the nursing surveillance scale developed in this study measures concepts related to NIC surveillance activities.

Discussion

The NIC definition of surveillance is not actively used in clinical practice in South Korea. However, in interviews on concept analysis of nursing surveillance, nurses stated that they performed surveillance activities in their practice, including systematic assessment, pattern recognition, anticipation of problems, decision-making, communication, and providing interventions. In practice, nurses identified the overall situation by observation and assessment of the patient’s condition, made decisions such as continual monitoring or responding to risks based on the patient’s condition, communicated with doctors concerning the patient’s health and safety, and provided appropriate nursing [19]. Therefore, this study aimed to develop and validate a nursing surveillance scale, which is essential for understanding the multi-dimensional concept and importance of nursing surveillance, and promoting the active performance of surveillance activities by nurses in South Korea. The nursing surveillance scale in this study consisted of 16 items in total, which was shorter than both the 46-item and 21-item NIC lists of surveillance activities. This scale had four factors, which reflected all six attributes of nursing surveillance proposed by Kim and Cho [19]. Specifically, the ‘anticipation of problems and decision-making’ factor in this scale consisted of items corresponding to the attributes of ‘anticipation of problems,’ ‘decision-making,’ and ‘communication’ [8, 19], the ‘systematic assessment’ factor consisted of items corresponding to the attributes of ‘systematic assessment’ and ‘nursing performance’ [19], the ‘recognition of patterns’ factor consisted of items corresponding to the attribute of ‘pattern recognition’ [8, 19], and the ‘identification of patient’s self-care and coping strategies’ factor consisted of items corresponding to the attribute of ‘systematic assessment’ [19].

The nursing surveillance scale developed in this study reflects the empirical characteristics of surveillance activities perceived and performed by nurses in South Korea. During the development process, nurse-led or nurse-determined items such as ‘discuss treatment plans with doctors,’ ‘seek consultation with experts,’ ‘start treatment according to agreed-upon protocols,’ and ‘recommend interdisciplinary services’ among the NIC surveillance activities were removed. These results appeared to reflect international differences in the division of social and technical roles in nurses’ work [23]. In South Korea, the role of nurses was emphasized as performing the instructions of doctors and administering prescriptions. However, given that nursing surveillance is a core intervention for the early detection of risks and prevention of errors [8], to promote surveillance by nurses, who are close to patients throughout the day, it is essential to provide support at the organizational level, such as constructing a culture that supports surveillance, interdisciplinary communication [13], and developing and implementing interdisciplinary protocols [5].

The ‘anticipation of problems and decision-making’ factor consists of six items in ‘overall judgment of patient data,’ ‘anticipation of potential problems through evaluation of treatment and intervention effects,’ ‘selection of required monitoring indices based on the patient’s condition,’ ‘deciding the frequency of data collection,’ ‘communicating with doctors to solve problems depending on changes in the patient’s condition,’ and ‘participating in decision-making about the patient’s treatment plans.’ This factor emphasizes cognitive factors in surveillance, such as nurses anticipating problems by collecting and interpreting data to ascertain patterns in the patient’s condition [8] and making clinical decisions about patient care [21, 22]. Additionally, this factor includes professional and independent characteristics of surveillance, such as nurses judging which data to collect and decision-making behaviors within the scope of their work [35], reflecting cognitive and behavioral factors that measure ‘anticipation of problems,’ ‘decision-making,’ and ‘communication’ among the surveillance attributes suggested by Kim and Cho [4, 19]. Specifically, four items were derived from field interviews as follows: ‘to overall judgment of patient data,’ ‘anticipation of potential problems based on evaluation of treatment or intervention effects,’ ‘communication with doctors to solve problems based on changes in the patient’s condition,’ and ‘participation in decision-making concerning patients’ treatment plans’ [19]. These results can demonstrate that clinical nurses in South Korea participate in decision-making concerning patients’ treatment plans by anticipating patients’ problems, making decisions about monitoring, and communicating with other healthcare providers. Therefore, this factor measures cognitive and behavioral factors in nursing surveillance. Given that nursing surveillance is performed based on experience and professionalism, education and support, particularly for new nurses, is essential for ensuring patients’ health and safety, and simulation-based learning will be a crucial educational strategy [2, 36].

The ‘systematic assessment’ factor consists of five items from the NIC surveillance activities were monitoring the signs of infection and bleeding, monitoring the condition of excretion and skin, and managing data-gathering devices. These items correspond to specific physical symptom monitoring activities included in the 46-item NIC list of surveillance activities. Systematic assessment includes ongoing and repeated tasks performed by nurses, such as measuring vital signs, patient observation, checking test results, asking patients and caregivers questions, checking health history, and evaluating the effects of treatments and procedures. Several previous studies emphasized the need for developing electronic nursing records systems as a strategy to aid nurses, who have to care for many patients, to effectively perform surveillance [11, 13].

The ‘recognition of patterns’ factor consists of three items related to verifying data from rounds and handover, monitoring whether vital signs are appropriate or unstable, and closely, continually monitoring critical patients. During handover, nurses gather fragments of data and identify cues through a bidirectional communication process [21]. Nurses check the patient’s condition in advance by inspecting electronic nursing records before handover, clarifying patient data by closely listening and asking questions during handover, and directly observing, assessing, and asking questions about the patients during rounds after handover. Nurses perceive the verification of handover content as the start of surveillance [19]. This reflects the surveillance activities performed by hospital nurses in the acute phase to protect patients’ health and safety, ascertaining and monitoring changes in patients’ vital signs and symptoms. In particular, pattern recognition is represented in how expert nurses ascertain a patient’s condition through systematic, complete assessment, and detect changes or abnormalities in the patient’s condition based on their experience [37].

The ‘identification of patient’s self-care and coping strategies’ factor consists of two items from the 46-item NIC surveillance activities list: ‘monitoring patients self-care abilities’ and ‘monitoring the coping strategies of patients and their families.’ Thus, while performing surveillance, Korean nurses assess the patient’s physical and cognitive condition and obtain information about self-care and coping from the patient and their family. These items are included in the ‘systematic assessment’ attributes of nursing surveillance. These findings suggest that Korean nurses prioritize the patient’s self-care ability and the family’s coping strategies important during surveillance. Furthermore, these results were consistent with another study in which clinical nursing experts in the US reported that the characteristics of patients and their families affect surveillance [37]. This may be due to family or caregivers often helping and caring for patients while sleeping by their bedside in Korean acute hospitals. In Korea, the ratio of nurse to patient is high; hence, the patient’s self-care ability and the family’s coping strategies can be important conditions in nursing surveillance. Therefore, it is necessary for future research to verify the validity of this factor and determine how it affects nursing surveillance.

Limitation and proposals

Given that this scale was validated using a convenience sample of nurses at general hospitals, it is crucial to perform refinement studies on nurses from various clinical fields. The scale was developed based on the NIC surveillance activities and attributes of nursing surveillance among Korean nurses. Therefore, this scale represents the characteristics of nursing surveillance in South Korea; however, limitations may be present when applying this scale to nursing practice in other countries without modification. Future studies should be performed using the nursing surveillance scale to compare surveillance performance in samples of nurses from diverse fields, including intensive care units, pediatric departments, and geriatric hospitals, and to identify antecedents and consequences of nursing surveillance. Additionally, it is important to perform studies to develop systems to support nursing surveillance and clinical decision support systems.

Conclusions

This study developed a nursing surveillance scale to improve nursing surveillance in acute hospitals. This study can provide precedent to motivate the development of nursing surveillance scales in other countries, and ultimately stimulate studies on nursing surveillance, which is essential for patient safety.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

NIC:

Nursing Intervention Classification

CVI:

Content validity index

EFA:

Exploratory factor analysis

CFA:

Confirmatory factor analysis

CMIN/DF:

Chi-square minimum/degree of freedom

GFI:

Goodness-of-fit-index

NFI:

Normed fit index

TLI:

Tucker-Lewis index

CFI:

Comparative fit index

SRMR:

Standardized root-mean-square residual

RMSEA:

Root mean square error of approximation

References

  1. Institute of Medicine. Keeping patients safe: transforming the work environment of nurses. Washington, D.C.: National Academies; 2004.

    Google Scholar 

  2. Kim SY, Cho M-K. Factors affecting nursing surveillance activity among clinical nurses. Healthcare. 2023;11(9):1273. https://doi.org/10.3390/healthcare11091273.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Kelly LA. Nursing surveillance in the acute care setting: latent variable development and analysis. The University of Arizona; 2009.

  4. Dresser S. The role of nursing surveillance in keeping patients safe. J Nurs Adm. 2012;42(7–8):361–8. https://doi.org/10.1097/NNA.0b013e3182619377.

    Article  PubMed  Google Scholar 

  5. Stotts JR, Lyndon A, Chan GK, Bekmezian A, Rehm RS. Nursing surveillance for deterioration in pediatric patients: an integrative review. J Pediatr Nurs. 2020;50:59–74. https://doi.org/10.1016/j.pedn.2019.10.008.

    Article  PubMed  Google Scholar 

  6. Giuliano KK. Improving patient safety through the use of nursing surveillance. Biomed Instrum Technol. 2017;51(s2):34–43. https://doi.org/10.2345/0899-8205-51.s2.34.

    Article  PubMed  Google Scholar 

  7. Henneman EA, Gawlinski A, Giuliano KK. Surveillance: a strategy for improving patient safety in acute and critical care units. Crit Care Nurse. 2012;32(2):e9–18. https://doi.org/10.4037/ccn2012166.

    Article  PubMed  Google Scholar 

  8. Halverson CC, Scott Tilley D. Nursing surveillance: a concept analysis. Nurs Forum. 2022;57(3):454–60. https://doi.org/10.1111/nuf.12702.

    Article  PubMed  Google Scholar 

  9. Butcher HK, Bulechek GM, McCloskey Dochterman JM, Wagner CM. Nursing interventions classification (NIC). 7th ed. London, England: Mosby; 2018.

    Google Scholar 

  10. Peet J, Theobald KA, Douglas C. Building safety cultures at the frontline: an emancipatory practice development approach for strengthening nursing surveillance on an acute care ward. J Clin Nurs. 2022;31(5–6):642–56. https://doi.org/10.1111/jocn.15923.

    Article  PubMed  Google Scholar 

  11. Christabel S, Sudhaker C, Varma M. Nurses’ outlook on healthcare-associated infection surveillance: findings from focus group discussions. Afr J Nurs Midwif. 2022;24(1):1–11. https://doi.org/10.25159/2520-5293/9628.

    Article  Google Scholar 

  12. Copanitsanou P, Santy-Tomlinson J. The nurses’ role in the diagnosis and surveillance of orthopaedic surgical site infections. Int J Orthop Trauma Nurs. 2021;41:100818. https://doi.org/10.1016/j.ijotn.2020.100818.

    Article  PubMed  Google Scholar 

  13. Halverson CC, Scott Tilley D. Creating a culture of support for nursing surveillance. Nurs Forum. 2022;57(6):1204–12. https://doi.org/10.1111/nuf.12823.

    Article  PubMed  Google Scholar 

  14. Perry-Woodford ZL, Marinova P. Nurse-led surveillance of ileoanal pouch patients post-stoma reversal: outcomes at 12-month review. Br J Nurs. 2022;31(6):S14–21. https://doi.org/10.12968/bjon.2022.31.6.S14.

    Article  PubMed  Google Scholar 

  15. Shever LL, Titler MG, Kerr P, Qin R, Kim T, Picone DM. The effect of high nursing surveillance on hospital cost. J Nurs Scholarsh. 2008;40(2):161–9. https://doi.org/10.1111/j.1547-5069.2008.00221.x.

    Article  PubMed  Google Scholar 

  16. Juvé-Udina M-E, Fabrellas-Padrés N, Adamuz-Tomás J, Cadenas-González S, Gonzalez-Samartino M, de la Cueva Ariza L, et al. Surveillance nursing diagnoses, ongoing assessment and outcomes on in-patients who suffered a cardiorespiratory arrest. Rev Esc Enferm USP. 2017;51:e03286. https://doi.org/10.1111/jonm.12885.

    Article  PubMed  Google Scholar 

  17. Kelly L, Vincent D. The dimensions of nursing surveillance: a concept analysis: dimensions of nursing surveillance. J Adv Nurs. 2011;67(3):652–61. https://doi.org/10.1111/j.1365-2648.2010.05525.x.

    Article  PubMed  Google Scholar 

  18. Jahrsdoerfer M. Clinical surveillance, a concept analysis: leveraging real-time data and advanced analytics to anticipate patient deterioration. Bringing theory in practice. Online J Nurs Inf. 2019;23.

  19. Kim SY, Cho M-K. Concept analysis of nursing surveillance using a hybrid model. Healthcare. 2023;11(11):1613. https://doi.org/10.3390/healthcare11111613.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Schwart-Barcott D, Kim H. An expansion and elaboration of the hybrid model of concept development. In: Rodgers BL, Knafl KA, editors. Concept development in nursing: foundations, techniques, and applications. 2nd ed. Philadelphia, PA: Saunders; 2000. pp. 129–59.

    Google Scholar 

  21. Pfrimmer DM, Johnson MR, Guthmiller ML, Lehman JL, Ernste VK, Rhudy LM. Surveillance: a nursing intervention for improving patient safety in critical care environment. Dimens Crit Care Nurs. 2017;36(1):45–52. https://doi.org/10.1097/DCC.0000000000000217.

    Article  PubMed  Google Scholar 

  22. Milhomme D, Gagnon J, Lechasseur K. The clinical surveillance process as carried out by expert nurses in a critical care context: a theoretical explanation. Intensive Crit Care Nurs. 2018;44:24–30. https://doi.org/10.1016/j.iccn.2017.07.010.

    Article  PubMed  Google Scholar 

  23. Leal JAL, de Melo CMM. The nurses’ work process in different countries: an integrative review. Rev Bras Enferm. 2018;71(2):413–23. https://doi.org/10.1590/0034-7167-2016-0468.

    Article  PubMed  Google Scholar 

  24. DeVellis RF. Scale development: theory and applications. 4th ed. Thousand Oaks, CA: SAGE; 2016.

    Google Scholar 

  25. Polit-O’Hara D, Yang FM. Measurement and the measurement of change: a primer for the health professions. Philadelphia: Wolters Kluwer; 2016.

    Google Scholar 

  26. Wagner CM, Butcher HK, Clarke MF. Nursing interventions classification (NIC). 8th ed. Philadelphia, PA: Elsevier - Health Sciences Division; 2023.

    Google Scholar 

  27. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10(7):1–9.

    Google Scholar 

  28. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis: a global perspective. 7th ed. Upper Saddle River, NJ: Pearson Education; 2010.

    Google Scholar 

  29. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 7th ed. London: Pearson; 2014. pp. 600–38.

    Google Scholar 

  30. Hu L-T, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  31. Song JJ. SPSS/AMOS statistical analysis method (Revised Edition). Paju: 21st Century; 2015. pp. 146–482.

  32. Williams B, Onsman A, Brown T. Exploratory factor analysis: a five-step guide for novices. Australas J Paramed. 2010;8:1–13. https://doi.org/10.33151/ajp.8.3.

    Article  Google Scholar 

  33. Tabachnick BG, Fidell LS. Using multivariate statistics: United States edition. 5th ed. Boston, MA: Pearson; 2006. pp. 607–75.

    Google Scholar 

  34. Noh KS. The proper methods of statistical analysis for dissertation: SPSS & AMOS 21. Seoul: Hanbit Academy Inc.; 2019. pp. 14–193.

  35. Bulechek GM, Butcher HK, Dochterman JM. Nursing interventions classification (NIC). 5th ed. Maryland Heights, MO: Mosby; 2008.

    Google Scholar 

  36. Hong M-H, Park J-Y. Nurses’ perception of accreditation, awareness and performance of infection control in an accredited healthcare system. J Korean Acad Nurs Adm. 2016;22(2):167. https://doi.org/10.11111/jkana.2016.22.2.167.

    Article  Google Scholar 

  37. Brier J, Carolyn M, Haverly M, Januario ME, Padula C, Tal A, et al. Knowing something is not right is beyond intuition: development of a clinical algorithm to enhance surveillance and assist nurses to organise and communicate clinical findings. J Clin Nurs. 2015;24(5–6):832–43. https://doi.org/10.1111/jocn.12670.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors (s) sincerely appreciate the participation of those who participated in this study.

Funding

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government MSIT (Ministry of Science and ICT (No.RS-2023-00273954).

Author information

Authors and Affiliations

  1. Department of Nursing, Changwon National University, 20 Changwondaehak-ro, Uichang-gu, Changwon, Republic of Korea

    Se Young Kim

  2. Department of Nursing Science, College of Medicine, Research Institute of Nursing Science, Chungbuk National University, 1 Chungdae-ro, Seowon-gu, Cheongju, 28644, Republic of Korea

    Mi-Kyoung Cho

Authors
  1. Se Young Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mi-Kyoung Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, S.Y.K. and M.-K.C.; methodology, M.-K.C.; software, M.-K.C.; validation, S.Y.K. and M.-K.C.; formal analysis, M.-K.C.; investigation, S.Y.K.; resources, S.Y.K. and M.-K.C.; data curation, S.Y.K. and M.-K.C.; writing—original draft preparation, S.Y.K. and M.-K.C.; writing—review and editing, S.Y.K. and M.-K.C.; visualization, M.-K.C. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Mi-Kyoung Cho.

Ethics declarations

Ethics approval and consent to participate

This study was reviewed and approved by the Institutional Review Board of Changwon National University (IRB No. 7001066-202304-HR-016). This study obtained informed consent to participate from all of the participants. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, S.Y., Cho, MK. Testing the validity and reliability of the Korean nursing surveillance scale: a methodological study. BMC Nurs 23, 709 (2024). https://doi.org/10.1186/s12912-024-02378-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12912-024-02378-x

Keywords

BMC Nursing

ISSN: 1472-6955

Contact us