Managing patient deterioration: a protocol for enhancing undergraduate nursing students’ competence through web-based simulation and feedback techniques
© Cooper et al.; licensee BioMed Central Ltd. 2012
Received: 7 August 2012
Accepted: 12 September 2012
Published: 28 September 2012
To describe a funded proposal for the development of an on-line evidence based educational program for the management of deteriorating patients.
There are international concerns regarding the management of deteriorating patients with issues around the ‘failure to rescue’. The primary response to these issues has been the development of medical emergency teams with little focus on the education of primary first responders.
A mixed methods triangulated convergent design.
In this four phase proposal we plan to 1. examine nursing student team ability to manage deteriorating patients and based upon these findings 2. develop web based educational material, including interactive scenarios. This educational material will be tested and refined in the third Phase 3, prior to evaluation and dissemination in the final phase.
This project aims to enhance knowledge development for the management of deteriorating patients through rigorous assessment of team performance and to produce a contemporary evidence-based online training program.
KeywordsEducation Nursing Patient deterioration Simulation
In this paper we describe an Australian based funded program of translational research that aims to produce an evidence based education package that will improve nursing student management of deteriorating patients. In Australia the Commonwealth Government has invested in the development of learning and teaching in the higher education sector through the Australian Learning and Teaching Council (ALTC) and more recently the Office for Learning and Teaching (OLT). Established in 2011 the OLT aims to promote excellence in teaching and learning with nationally competitive funding of A$50 million over the next four years. This approach is similar to that in the United Kingdom where the Higher Education Academy was established in 2004 aiming to support research and evaluation in order to enhance students learning experience. These programs aim to improve the quality of teaching and to professionalise the workforce with a strong focus on the transfer of research outcomes into practical and applied educational programs.
In 2011 the MODE-UQGT collaboration received ALTC funding of A$220,000 for three Australian Universities (Monash & Deakin Universities, University of Queensland) and a Further Education partner (GippsTAFE) to produce a sustainable program of education on the management of deteriorating patients. We know that the management of deteriorating patients is poor and when left untreated leads to expensive and often unsuccessful resuscitation procedures[2–5]. In addition, it is acknowledged that non-metropolitan settings have fewer resources for managing deteriorating patients[6, 7], communication and referral processes are less well established[4, 8], and demands on enrolled and registered nurses have increased. However not enough is known about the educational needs, decision-making strategies and clinical practices of students in order to enhance their future practice. This study aims to address these issues by examining students’ performance in simulated settings, and based on these findings produce on-line evidence based educational materials, assessments, and video recorded interactive scenarios using patient actors. Participants will be able to ‘click on’ actions e.g. taking vital signs and recording electrocardiograms (ECG), whilst records of their performance are retained for feedback.
Management of deteriorating patients can be influenced by education and past experience[9, 10]. In health care contemporary education approaches emphasise the need for active learning and increased use of simulated environments[12–15] to reduce medical errors in settings with high technological, environmental and psychological fidelity. Since 2008 in pilot work with individual nursing students[16, 17], midwifery students and Registered nurses working in a hospital setting we have examined patient management in simulated settings using an interventional program known as FIRST2ACT (Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends). It was found that participants had good theoretical knowledge but that they often failed to respond appropriately. However, findings also indicated that the educational experience had a significant impact on participants’ reported learning, and significant improvements in patient management skills. A recommendation from these studies was to extend the work to understand how small teams (3 nursing students) respond to clinical emergencies and from these findings produce a web based educational package (FIRST2ACT Web). The project will address these objectives in four sequential phases described below. In line with funding criteria the project has received ethical approval and has been independently evaluated by a leading academic experienced in pedagogy of teaching/learning and health care practice. This External Assessor is operating in conjunction with an Internal Assessor with a track record of successful multi-site and multi-national project management. An evaluation plan was developed by the assessors including a risk assessment and staged plan for budget expenditure.
The approach incorporates five components: 1) developing core knowledge; 2) assessment (learning stimulus); 3) simulation; 4) reflective review; and 5) performance feedback. The theoretical underpinning for these stages is the belief that the critical thinking skills necessary for practice are best acquired through experience, an approach supported by Experiential Learning Theory[22, 23], and the notion of concrete experience and abstract conceptualisation, achieved through activity or reflective observation. In addition, modalities of learning such as visual, visual/verbal, physical (or kinaesthetic), and auditory have been described demanding a variety of teaching approaches to meet students’ learning needs.
Assessment as a stimulus for learning (for and as learning) contributes to the development of independent learning skills and ongoing professional development, encouraging students to “think, decide and act” ( p305). Simulation is defined as “an education technique in which elements of the real world are appropriately integrated to achieve specific goals related to learning or evaluation” ( p75). Clinical simulation, which should always include feedback techniques, can be enacted through mannequins, actors and role play, consolidating theory into practice in a safe environment whilst reducing the pressure on hospital learning environments[28–31]. Simulation-based training increases self-reported knowledge and/or confidence in nursing and medical students[31, 32], has benefits over didactic teaching techniques, improves outcomes in Objective Structured Clinical Examinations (OSCE), and most importantly positively impacts patient care. Authenticity or fidelity is essential to replicate the clinical environment with ‘true’ representations of patient states.
Reflection and self-assessment are essential for building skills with benefits of video self-review over oral feedback alone, including realistic and explicit perspectives of performance. However, constructive feedback (debriefing) is also essential for the development of knowledge and skills and enables subsequent reflection. These processes of teaching and learning should be linked in a contiguous process to enhance knowledge and improve skill retention.
These findings underpin our pilot work and the approach in this project leading to the following research questions.
- 1.In emergency situations how do undergraduate nursing students’ perform and what are their decision strategies in primary response teams (Phase 1)? Specifically we aim to:
Examine participants’ ability to recognise patient deterioration in a simulated environment and establish which clinical cues are most commonly identified and/or missed as signs of deterioration;
Identify the relationship between knowledge and skills in the recognition of physiological changes in a simulated environment;
Develop an understanding of decision-making processes through student reflection.
What impact does the web-based learning program (Phases 2 & 3) (FIRST 2 ACT Web) have on student learning and on their clinical activity (Phase 4)?
These questions will be addressed through four Phases (over two years). The project is designed as an interventional analysis incorporating mixed methods. The mixed methods approach will use a triangulated convergent design aiming to draw together quantitative and qualitative data to inform the development of each phase and the final outcomes of the study.
Recruitment and ethical approval
Applicable University and Institutional Ethics approval have been obtained for all phases of the study [Lead institution: Monash University Human Research Ethics Committee CF11/3414 -2011001825]. All students were fully informed of the procedures and voluntarily consented to participate in the project.
Phase 1: understanding team work and decision making(months 1–8)
A multiple choice structured knowledge questionnaire (MCQ).
Three video recorded team based simulation exercises (OSCEs) to identify performance skills.
Video review of the simulation exercises to facilitate reflective review of performance.
Using a triangulated convergent design we will perform separate quantitative and qualitative analysis and then compare the data with matrices of quantitative findings and qualitative themes.
All final-year nursing students studying on a Bachelor of Nursing or combined degree at Monash University, Gippsland Campus (n = 120); Deakin University, Melbourne Burwood Campus (n = 350); and the University of Queensland, Ipswich Campus (n = 100) will be eligible. Total population n = 570. All participants will have completed a standard educational program on emergency care prior to recruitment.
A minimum of 100 students will be selected from across the campuses. In our previous studies we have had the following response rates: nursing students (46%), midwifery students (78%), Registered nurses (82%). We therefore anticipate a response rate of greater than 50%.
Data collection tools
Multiple choice questionnaire (MCQ)
Additional file 1 Completed by participants prior to the OSCE and adapted from previously validated instruments[40, 41] and verified by a panel of clinical experts. This measure will enable identification of knowledge in relation to skill performance (at OSCEs) and prompts development of learning (i.e. learning through assessment).
Simulation exercises (OSCEs)
Additional file 2 All participants will complete three contrasting simulation exercises in their university clinical skill laboratories. Each scenario will be based on common presenting conditions including an acute myocardial infarction, hypovolaemia and chronic obstructive pulmonary disease. Standardised patient actors will be employed to simulate these presenting conditions to enhance the reality of the simulation. These team based scenarios (for groups of three learners, covering 5 groups a day) have been developed from previous work[16–18] and include nominal yes/no ratings of performance that will be rated and verified by two clinicians.
Team emergency assessment measure (TEAM)
Additional file 3 In addition to clinical skill OSCE rating, non-technical teamwork skills will be assessed by two clinicians using the valid and reliable teamwork assessment tool TEAM. Using a rating scale of 0–4 this tool includes ratings of leadership, teamwork and task management and a final overall global rating from 1–10. Detailed guidance on how to use TEAM is available from the authors including the training requirements for assessors.
Situation awareness global assessment technique (SAGAT)
On the completion of each scenario team leaders will be asked a series of questions to ascertain their understanding of the situation including their awareness, understanding and prediction of future events in relation to each simulation. A rapid questioning technique is required to encourage respondents to give their first ‘gut reaction’ response to each query. Questions were developed by an expert panel using Goal Directed Task Analysis in order to identify core and peripheral situation awareness requirements.Additional file 4 illustrates this process for the acute myocardial infarction scenario. Questions are designed to identify awareness of the patient’s physiological state and to ascertain respondents’ awareness of the wider situation, for example, what the patient is wearing on their wrist and whether suction is available?
Video recordings of participants’ interaction during the simulation exercise will be used as a memory recall service to stimulate the team to provide a audio-recorded reflective account of their decision making, a process known as ‘photo elicitation’[44, 45]. This review will enhance the educative processes and will enable the teaching team to understand decision making processes.
Performance feedback and training
Individual constructive feedback from a trained instructor will follow the video review (and the first three scenarios). Written participant feedback/evaluation will be sought after this first phase of training.
Participant demographics, questionnaire, OSCE and TEAM performance will be described with the use of descriptive and inferential statistics. Qualitative: Interview and video data will be content analysed to identify common themes.
Phase 2: development of a web-based electronic educational package (months 9–13)
Based on the findings from previous studies and from Phase 1 we will develop an interactive web based educational package which will include a course handbook; assessment tests - Multiple Choice Questionnaires (MCQ) - performed at the start and end of course; scenarios (OSCEs); and feedback techniques. A series of three professionally recorded videoed OSCEs (based on the three Phase 1 scenarios) will be developed depicting professional actors as patients who are deteriorating. Each of the three OSCE videos will run for eight minutes (the approximate response time for a resuscitation team) with patients significantly deteriorating at the midpoint. A range of ‘pop up’ videos will also be recorded to simulate the taking of vital signs, procedural recordings (e.g. 12 lead ECG & intravenous cannulation), and the patients history (spoken by the actor). Using a ‘mouse over’ function participants will be able to click on an action for example, lay the patient flat, give oxygen, or take an ECG. On completion results will be provided with explanations on performance outcomes and, where the pass mark has been reached, a certificate will be provided.
Phase 3: program implementation (months 14–17)
The final educational package will be made available to applicable nursing lecturers within the MODE-UQGT collaboration. Web-based and hard-copy instructor material will be produced to guide lecturers through the course material. Lecturers un-associated with the project will be recruited to test the transferability of the material. The sample size of 614 students is a whole population sample allowing for an attrition rate of up to 30%. Based on prior data from nursing students who had completed the FIRST2ACT multiple choice questionnaire this sample size significantly exceeds the power required to detect a change in knowledge before and after the FIRST2ACT Web program [n = 33 participants; based upon MCQ Time 1 = 66% v Time 2 = 85% with a power of 80% and a one sided significance level of 0.05.]
The program will be implemented across cohorts of final-year nursing students at Monash University, Gippsland Campus (approximately 120 students); Deakin University, Melbourne Burwood Campus (approximately 350 students); the University of Queensland, Ipswich Campus (approximately 100 students) and one campus of nursing diploma students as GippsTAFE – Morwell (approximately 44 students). [Total students n = 614]. These sites have been selected as they represent students from two levels of training (Bachelor and Diploma) studying in rural (Monash Gippsland, UQ Ipswich) and urban locations (Deakin Burwood). All sites run generic units/modules in acute care – the FIRST 2 ACT Web program will replace specific teaching approaches to patient deterioration without the need to alter unit/module objectives and assessments. In this trial stage of the program assessment outcomes from FIRST 2 ACT Web will not influence final unit/module assessment outcomes.
Phase 4: evaluation of learning and dissemination strategy (months 17–24)
At the commencement of the study produce a picture of how the project will work using an ‘intended project logic’ diagram. This exercise will be performed with stakeholders (project team, reference group, lecturers at trial sites, clinical placement co-ordinators and practice educators) in order to focus on the context and objectives of the evaluation e.g. the impact of FIRST 2 ACT Web on learning and teaching. Outcomes from these discussions will guide the evaluation methods which may include the following approaches.
- 2.Analysis of outcome data including:
MCQ scores pre-post course with applicable inferential analysis (e.g. paired t-tests).
Paper-based, in-class student evaluations of:
Course satisfaction, e.g. relevance, applicability, interest, delivery mode, and descriptive outcomes.
Reflective self reports of specific knowledge levels pre-post course.
Ten stakeholder focus groups, based on a stratified sample (seven in Victoria and three in Queensland) to identify perceived impact of the program. Core themes and outcomes will be identified using Miles and Huberman’s stages of data analysis .
Based on outcomes from the evaluation, update and adapt FIRST 2 ACT Web to meet learning and teaching requirements.
Outcomes from this project will include a web-based, evidence-based teaching and learning package incorporating innovative simulation techniques. Using a multi-faceted strategy that targets local, national and international interdisciplinary audiences findings and outcomes will be disseminated across clinical and tertiary education sectors and through policy decision makers in patient safety. This will occur through library repositories and peer review publications, conference presentations and workshops, and a web based instructor forum.
This study is part of a national Australian initiative to enhance the quality of learning and teaching in the higher education sector. Guidelines for funding differ somewhat from research funding schemes in that there is a forthright focus on translation of outcomes, dissemination and implementation strategies. This protocol therefore emphasises these aspects through the development of evidence, which informs program development, which in itself is rigorously evaluated and adapted. There are a number of issues associated with a large project of this sort. For example, leadership and management, recruitment strategies and ethical considerations, departmental and university collaboration, technical development and web access issues, and team member succession planning. The project teams early assessment of the risks have raised many of these issues and enabled them to be proactively addressed.
In summary, this project aims to enhance knowledge development through rigorous assessment of team performance and the educative processes of learning through assessment, personal reflective learning, and professional feedback. This will lead to an in-depth understanding of decision making and a contemporary interactive evidence-based online training program. Results and outcomes from the study will be available from 2013.
Australian learning and teaching council
Capillary refill time
Feedback incorporating review and simulation techniques to act on clinical trends
Multiple choice questionnaire
Morphine oxygen, nitrates, asprin
Objective structured clinical examinations
Office for learning and teaching
Situation awareness global assessment technique
- SpO2 :
Team emergency assessment measure.
- Commonwealth of Australia: Australian Government directory. http://australia.gov.au/directories/australia/oft
- Harrison GA, Jacques TC, Kilborn G, McLaws M: The prevalence of recordings of the signs of critical conditions and emergency response in hospital wards – the SOCCER study. Resuscitation. 2005, 65: 149-157. 10.1016/j.resuscitation.2004.11.017.View ArticlePubMedGoogle Scholar
- Intensive Care Society: Guidelines to the Introduction of Outreach Services. 2002, London: Intensive Care Society, 9-Google Scholar
- Endacott R, Westley M: Managing patients at risk of deterioration in rural hospitals: A qualitative study. Aust J Rural Health. 2006, 14: 275-279. 10.1111/j.1440-1584.2006.00829.x.View ArticlePubMedGoogle Scholar
- Baudouin S, Evans T: Improving outcomes for severely ill medical patients. Clin Med. 2002, 2: 92-94.View ArticleGoogle Scholar
- Hore C, Lancashire W, Roberts J, Fassett R: Integrated critical care: an approach to specialist cover for critical care in a rural setting. Med J Aust. 2003, 179: 95-97.PubMedGoogle Scholar
- Peake S, Judd N: Supporting rural community-based critical care. Curr Opin Crit Care. 2007, 13: 720-724. 10.1097/MCC.0b013e3282f1bb21.View ArticlePubMedGoogle Scholar
- Endacott R, Kidd T, Chaboyer W, Edington J: Recognition and communication of patient deterioration in a regional hospital: a multi-methods study. Aust Crit Care. 2007, 20: 100-105. 10.1016/j.aucc.2007.05.002.View ArticlePubMedGoogle Scholar
- Cioffi J: A study of the use of past experiences in clinical decision making in emergency situations. Int J Nurs Stud. 2001, 38: 591-599. 10.1016/S0020-7489(00)00096-1.View ArticlePubMedGoogle Scholar
- United Kingdom Central Council for Nursing: Fitness for Practice: The Report of the Commission for Nursing and Midwifery Education. 1999, London: United Kingdom Central CouncilGoogle Scholar
- Endacott R, Scholes J, Freeman M, Cooper S: The reality of clinical learning in critical care settings: a practitioner: student gap?. J Clin Nurs. 2003, 12: 778-785. 10.1046/j.1365-2702.2003.00785.x.View ArticlePubMedGoogle Scholar
- Childs JC, Sepples S: Clinical teaching by simulation: lessons learned from a complex patient care scenario. Nurs Educ Perspect. 2006, 27: 154-158.Google Scholar
- Flin RM, Maran N: Identifying and training non-technical skills for teams in acute medicine. Qual Saf Health Care. 2004, 13: 80-84. 10.1136/qshc.2002.003871.View ArticleGoogle Scholar
- Bucknall TK: Medical error and decision making: Learning from the past and present in intensive care. Aust Crit Care. 2010, 23: 150-156. 10.1016/j.aucc.2010.06.001.View ArticlePubMedGoogle Scholar
- Fritz P, Flanagan B: Review of mannequin-based high fidelity simulation in emergency medicine. Emerg Med Aust. 2008, 20: 1-9. 10.1111/j.1742-6723.2007.01022.x.View ArticleGoogle Scholar
- Cooper S, Kinsman L, Buykx P, McConnell-Henry T, Endacott R, Scholes J: Managing the deteriorating patient in a simulated environment: nursing students’ knowledge, skill and situation awareness. J Clin Nurs. 2010, 19: 2309-2318. 10.1111/j.1365-2702.2009.03164.x.View ArticlePubMedGoogle Scholar
- Endacott R, Scholes SJ, Kinsman L, McConnell-Henry T: When do patient signs become cues? Detecting clinical cues of deterioration in a simulated environment. J Adv Nurs. In press
- Cooper S, Bulle B, Biro MA, Jones J, Miles M, Gilmour C, Buykx P, Boland R, Kinsman L, Scholes J, Endacott R: Managing women with acute physiological deterioration: Student midwives performance in a simulated setting. Wom Birth. 2011, e-pubGoogle Scholar
- Buykx P, Kinsman L, Cooper S, McConnell-Henry T, Cant R, Endacott R, Scholes J: FIRST2 ACT: educating nurses to identify patient deterioration – a theory-based model for best practice simulation education. Nurse Educ Today. 2011, 31: 687-693. 10.1016/j.nedt.2011.03.006.View ArticlePubMedGoogle Scholar
- Kinsman L, Buykx P, Champion R, Cant R, Cooper S, Endacott R, McConnell-Henry T, Porter J, Missen K, Scholes J: The First2Act simulation program improves nursing practice in a rural Australian hospital. Aust J Rural Health. 2012, In pressGoogle Scholar
- Lisko S, O'Dell V: Integration of theory and practice: experiential learning theory and nursing education. Nurs Educ Perspect. 2010, 31: 106-PubMedGoogle Scholar
- Kolb DA: Experiential Learning: Experience As The Source Of Learning And Development. 1984, New Jersey: Prentice HallGoogle Scholar
- Kolb AY, Kolb DA: Learning styles and learning spaces: enhancing experiential learning in higher education. Acad Manag Learn Educ. 2005, 4: 193-212. 10.5465/AMLE.2005.17268566.View ArticleGoogle Scholar
- Bransford JD, Johnson MK: Contextual prerequisites for learning: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behaviour. 1972, 11: 717-726. 10.1016/S0022-5371(72)80006-9.View ArticleGoogle Scholar
- Boud D, Falchikov N: Aligning assessment with long-term learning. Assessment & Evaluation in Higher Education. 2006, 31: 399-413. 10.1080/02602930600679050.View ArticleGoogle Scholar
- Tiwari A, Lam D, Yuen KH, Chan R, Fung T, Chan S: Student learning in clinical nursing education: perceptions of the relationship between assessment and learning. Nurse Educ Today. 2005, 25: 299-308. 10.1016/j.nedt.2005.01.013.View ArticlePubMedGoogle Scholar
- Decker S, Sportsman S, Puetz L, Billings L: The evolution of simulation and its contribution to competency. J Contin Educ Nurs. 2008, 39: 74-80. 10.3928/00220124-20080201-06.View ArticlePubMedGoogle Scholar
- Alinier G, Hunt B, Gordon R, Harwood C: Effectiveness of intermediate-fidelity simulation training technology in undergraduate nursing education. J Adv Nurs. 2006, 54: 359-369. 10.1111/j.1365-2648.2006.03810.x.View ArticlePubMedGoogle Scholar
- Bantz D, Dancer MM, Hodson-Carlton K, Van Hove S: A daylong clinical laboratory: from gaming to high-fidelity simulators. Nurse Educ. 2007, 32: 274-277. 10.1097/01.NNE.0000299476.57185.f3.View ArticlePubMedGoogle Scholar
- Scalese RJ, Obeso VT, Issenberg SB: Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med. 2008, 23 (S1): 46-49. 10.1007/s11606-007-0283-4.View ArticlePubMedGoogle Scholar
- Waldner MH, Olson JK: Taking the patient to the classroom: Applying theoretical frameworks to simulation in nursing education. Int J Nurs Educ Schol. 2007, 4: 18-Google Scholar
- Bearnson CS, Wiker KM: Human patient simulators: a new face in baccalaureate nursing education at Brigham Young University. J Nurs Educ. 2005, 44: 421-425.PubMedGoogle Scholar
- Cant R, Cooper S: Simulation-based learning in nurse education: systematic review. J Adv Nurs. 2009, 66: 3-15.View ArticleGoogle Scholar
- Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S: Whitelaw A: does training in obstetric emergencies improve neonatal outcome?. BJOG. 2006, 113: 177-182. 10.1111/j.1471-0528.2006.00800.x.View ArticlePubMedGoogle Scholar
- Fanning R, Gaba M: The role of debriefing in simulation-based learning. Simul Healthc. 2007, 2: 115-125. 10.1097/SIH.0b013e3180315539.View ArticlePubMedGoogle Scholar
- Shute V: Focus on Formative Feedback. 2007, Princetone, NJ: Educational Testing ServiceGoogle Scholar
- Bienstock JL, Katz NT, Cox SM, Hueppchen N, Erickson S, Puscheck EE: To the point: medical education reviews-providing feedback. Am J Obstet Gynecol. 2007, 196: 508-513. 10.1016/j.ajog.2006.08.021.View ArticlePubMedGoogle Scholar
- Cooper S, Porter J, Endacott R: Mixed methods research in emergency care. Emerg Med J. 2010, Pre publication at http://emj.bmj.com/content/early/2010/10/30/emj.2010.096321.shortGoogle Scholar
- Wiseman J, Snell L: The deteriorating patient: a realistic but 'low tech' simulation of emergency decision making. Clin Teach. 2008, 5: 93-97. 10.1111/j.1743-498X.2008.00213.x.View ArticleGoogle Scholar
- Smith G, Poplett N: Knowledge of aspects of acute care in trainee doctors. Postgrad Med J. 2002, 78: 335-338. 10.1136/pmj.78.920.335.View ArticlePubMedPubMed CentralGoogle Scholar
- Endacott R, Jevon P, Cooper S: Clinical Nursing Skills: Core And Advanced. 2009, Oxford: Oxford University PressGoogle Scholar
- Cooper S, Cant R, Sellick K, Porter J, Somers G, Kinsman L, Nestel D: Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM). Resuscitation. 2010, 81: 446-452. 410.1016/j.resuscitation.2009.1011.1027.View ArticlePubMedGoogle Scholar
- Technologies SA: Super Sagat: Situational awareness global assessment technique. 2009, http://www.satechnologies.comGoogle Scholar
- Harper D: On the authority of the image: visual methods at the crossroads. Handbook of Qualitative Research. Edited by: Denzin N, Lincoln Y. 1994, Thousand Oaks: Sage, 403-412.Google Scholar
- Pink S: Viewing Ethnographers Photographs: Interviewing with Images. 2007, London: SageGoogle Scholar
- Schatzman L: Dimensional analysis: Notes on an alternative approach to the rounding of theory in qualitative research. Social Organisation and Social Processes: Essays in Honour of Anslem Strauss. Edited by: Maines D. 1991, New York: Aldine, 303-314.Google Scholar
- Kirkpatrick DI: Evaluation of training. Training Development Handbook. Edited by: Bittel CR. 1967, London: McGraw HillGoogle Scholar
- Clarke D: Instructional System Development – Evaluation Phase. 1995, http://www.nwlink.com/~donclrk/hrd/sat6.html. Chapter VIGoogle Scholar
- Miles M, Huberman AM: Qualitative Data Analysis: An Expanded Sourcebook. 1994, California: Sage, Thousand Oaks, 2Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6955/11/18/prepub