The role of simulation-based training to improve team performance in implementing one-hour sepsis bundle: a randomized trial

Abstract
Background: Excellent team performance is one of the keys in managing patients with sepsis successfully. Simulation-based training with high fidelity manikin (HFM) is one of the many ways to enhance team performance. This study aims to compare the role of simulation-based training with HFM with conventional method to improve team performance in conducting one-hour sepsis bundle.

Methodology: This randomized single-blind study was conducted on 16 doctors and 24 nurses in intensive care unit. Subjects were divided randomly into two groups, the simulation group and the discussion group. Simulation group received simulation training with HFM in conducting one-hour sepsis bundle in sepsis patients, while conventional group received case-based oral discussion with their tutor. The training was rounded off by a case simulation exam using HFM for both groups. The team performance consisted of clinical skills and communication skills evaluated through a validated assessment tool. Skills being assessed included initial assessment, diagnosing sepsis with SOFA and conducting one-hour sepsis bundle.

Results: The simulation group with high fidelity manikin completed the one-hour sepsis bundle better than the conventional group (p = 0.022). Particularly in collecting venous blood samples (p = 0.027) and blood culture samples (p = 0.011), along with giving the correct intravenous fluid replacement, tailored for each scenario (p = 0.027). Communication aspect was not significantly different in both groups.

Conclusion. Team performance in implementing one-hour sepsis bundle is better in the simulation group trained with high fidelity manikin as compared to conventional training group, who received case-based oral discussion.

Abbreviations: HFM - high fidelity manikin; ICU - Intensive Care Unit; SOFA - Sequential Organ Failure Assessment

Key words: Simulation-based training; One-hour sepsis bundle; Intensive care unit; Medical education; Interprofessional training

Citation: Sugiarto A, Tantri AR, Manggala SK, Peddyandhari FS, Auerkari AN, Fabiola T, Swannjo JP, Anakotta V, Theresia S. The role of simulation-based training to improve team performance in implementing one-hour sepsis bundle: a randomized trial. Anaesth. pain intensive care 2022;26(3):463-468; DOI: 10.35975/apic.v26i4.1954
Received: November 08, 2021; Reviewed: May 03, 2022; Accepted: 31 May 2022

 
Introduction
A well-coordinated and highly professional performance by any critical care team in managing sepsis patients is essential for the patients’ good outcome. An excellent team performance can prevent mistakes or any delay at any step in the management of the patients. 1, 2 Studies showed that strenuous training, particularly simulation-based training, was effective for teamwork training, and could improve both technical and soft skills, such as intra-team communication and coordination. 3, 4 Trainings in the medical field can be accomplished by various methods; from conventional methods such as lectures, training in the classroom, group discussion, to more sophisticated methods such as simulation-based training using high fidelity manikin (HFM) which resembles real-life cases. Repeated practice in an artificially created clinical situation has been proven advantageous in exploring teamwork behaviors. 5 The artificial clinical situation produced using an HFM could resemble a sick patient with his clinical signs and symptoms.

Simulation-based training can be carried out using different level of manikin features and modalities. The manikin will be chosen in accordance with the training purpose. The more sophisticated the manikin, the more excellent the outcomes of the studies are. 6 - 9 Resemblance to the actual condition during the training is expected to improve training effectiveness. One-hour sepsis bundle is one of the guidelines in managing sepsis. It was a result from updated version of the 2016 surviving sepsis campaign. Based on facts mentioned above regarding sepsis and simulation-based training, we examined the role of simulation-based training to improve team performance in conducting one-hour sepsis bundle.

Methodology
This was an experimental randomized single-blind study to examine the role of simulation-based training in improving team performance in conducting sepsis management. Participants were randomized using www.randomizer.org to an intervention (simulation) and a control group (discussion group). Ethical approval was obtained from Health Research Ethics Committee, Universitas Indonesia and Cipto Mangunkusumo National General Hospital (KET-693/UN2.F1/ETIK/PPM.00.02/2020). The trial was registered in the international trial registery (ClinicalTrials.gov: Identifier NCT05076721)

The selection of medical workers in the intensive care unit as research subjects was based on the fact most sepsis cases were handled in the Intensive Care Unit (ICU).3 Participants recruited included doctors and nurses working at Cipto Mangunkusumo National General Hospital ICU. The recruitement was announced during academic conferences, flyers, and also by personal invitation from the head of the ICU. All participants who were not fit during the study were excluded. We obtained written informed consent from all participants. There were 16 doctors and 24 nurses. The participants were divided randomly into two treatment groups: discussion group and simulation group. through www.randomizer.org. Each treatment group was further divided into four subgroups consisting of two doctors and three nurses. There were 8 subgroups in total, 4 subgroups in the simulation group and 4 subgroups in the discussion group. The definition of simulation and discussion groups were explained below.

Both of the groups received same lectures from the same speakers. The simulation groups were trained with HFM. The discussion groups only discussed the scenarios provided by the investigator. Each group had one anesthesiologist as a tutor. The final assessment used HFM to evaluate the group performance.

The study was conducted in the course of 3 days in October 2021. The first-day was an online lecture conducted with Zoom, on the second- and the third days were the training sessions and the final assessment. The training was conducted in the ‘Simulation Based Medical Education & Research Centre (SIMUBEAR)’, in the ‘Indonesian Medical Education & Research Institute’ (IMERI) building, while the final assessment was conducted on the same day, right after the training, in the ICU of Cipto Mangunkusumo National General Hospital. The location of the training and the final assessment was different as the investigator wanted to replicate the condition and situation as close as possible, this however created a time limit. Therefore, both groups were divided into second and third day. The discussion groups were in the second day and the simulation groups were in the third day

On the first day of the study, the lecture sessions were conducted for all participants for approximately 2 h. There were three sessions; the first and second session for 45 min to 1 h and the third session for 15 min. First lecture was delivered by an anesthesiologist-intensivist about the management of sepsis. The second lecture was delivered by an intensive care qualified nurse about the role of nursing in managing sepsis. Emphasis was laid on conducting relevant initial assessment, diagnosing sepsis using the Sequential Organ Failure Assessment (SOFA), and implementing one-hour bundle sepsis, as well as its differences with the previous bundle. The third session discussed about the HFM which would be used later on the final assessment. This last session discussed about the features of the manikin. On the second and third day, the participants joined their assigned groups and received simulation or case-based discussion training according to their groups. After each day’s final assessment, every participant was evaluated with post-test.

The scenario for the second-day’s training consisted of a male, 39 y old, who had undergone laparotomy and fistula resection due to necrotizing fasciitis of abdominal wall and enterocutaneous fistula with signs of sepsis. The scenario for the final assessment on the second day were a woman, 58 y old with decrease of consciousness, plantar abcess, and uncontrolled diabetes melitus with signs of sepsis. The third day training’s scenario consisted of a male, 65 y old, with signs of sepsis and respiratory distress. The third-day final assessment’s scenario consisted of a woman, 38 y old, who underwent tumor removal and craniotomy due to brainstem meningioma with signs of sepsis.

The simulation scenarios for the training session for the simulation group as well as the final assessment for both groups, were displayed on the monitor inside the simulation area. An HFM was used only for the simulation group’s training session for this study. We used an HFM in the final assessment of this study in order to replicate the emergency situation as realistic as possible. Laerdal SimMan® 3G manikin (Laerdal Medical Corp., Wappingers Falls, NY) was used in this study, connected to a computer with LLEAP application. The manikin was programmed to perform certain physical findings, such as weak pulses and poor respiratory findings, and then respond according to the interventions. Therefore, the participants were challenged to provide accurate treatment, such as conducting intubation, as the case progressed. The scenarios were required to be input beforehand in order for the manikin to display desired responses.

The development and validation of the assessment tool to evaluate the team performance in terms of the team’s skills and communication were designed before the study was conducted and was developed by the investigators. Validation was conducted by four anesthesiologist-intensivists. Validity of the assessment tool content was accomplished by Index of Aiken. The experts conducted a qualitative test review and reported the propriety between the assessment tool’s item with the indicator via Likert scale with five answer options. The content validity Index of Aiken on the overall test item were between 0.88 – 0.94.

The assessment tool used scoring method, with 0 as non-conducted, 1 as conducted but not perfect, and 5 as perfectly conducted. The evaluation consisted of 4 categories; initial assessment, diagnosed sepsis using SOFA score, implemented one-hour bundle sepsis, and communication aspects. The maximum score of each category was as follow; initial assessment 20 points, diagnosed sepsis using SOFA score 40 points, implemented one-hour bundle sepsis 65 points, and communication aspect 20 points. The maximum total points for this assessment tool were 145.

Primary outcome measured the differences in the scores of skills in implementing one-hour bundle sepsis between the groups. Secondary outcome measured the differences in intra-team communication’s score in implementing one-hour bundle sepsis.

Statistical analysis
Recorded data for team performance in conducting one-hour sepsis bundle was collected and analyzed statistically by Statistical Package for Social Scientist (SPSS). The parametric data were analyzed with independent T-test and the non-parametric data were analyzed with Mann-whitney test.

Results
Subjects’ characteristics are displayed in Table 1. Subjects’ skills between two treatment groups in general were compared. Table 2 shows that simulation group significantly excelled in skills (p = 0.019) compared to the conventional group. However, the communication component was not significantly different between the two groups (p = 0.287).

Simulation group had better skills in conducting the one-hour sepsis bundle compared to the conventional group (p = 0.022). Furthermore, as per analysis of one-hour bundle components, the simulation group was significantly proficient at collecting blood samples to measure lactate levels (P = 0.027), collecting blood cultures (P = 0.011), and giving initial intravenous fluid replacement (P = 0.027) suitable to each case.
Table 1: Baseline characteristics of subjects
Variable Simulation Group
(n = 20)		 Discussion Group
(n = 20)
Gender
 Male 10 9
 Female 10 11
Age (y) 31 (26 – 42) @ 32.00 (24 –55) @
Working span (y) 5.60 (1 – 24) @ 7.26 (1 – 32) @
@ Median (min. – max)
Discussion
The training carried out in this study was done using two methods, namely the simulation-based training method and the case-based discussions method. Our study showed that the simulation group had better skills in implementing one-hour bundle sepsis than the discussion group. Simulation training using communication score was not significantly different between the two groups. In a study of simulation-based training, Sorensen et al. stated that differences in modality and fidelity of training tools affect the outcome
Table 2: Comparation of skills in implementing one-hour sepsis bundle and communication in simulation and conventional group.
Variables Simulation Group
(n = 4)		 Discussion Group
(n = 4)		 P value@
Clinical skills 89.50 ± 8.73 76.75 ± 10.36 0.019
Communication skills 78.13 ± 17.36 65.63 ± 24.77 0.287
@ Unpaired T-test, mean ± SD
The value in this table is in percentage. The value is determined by our checklist with our validated assessment tool with 125 (100%) as the maximum score for skills variable and 15 (100%) for communication variable
 
Table 3. Comparation of One-hour Sepsis Bundle Performance between Simulation Group and Conventional Group
Variables Simulation Group
(n = 4)		 Discussion Group
(n = 4)		 P value
Diagnosing sepsis aided by SOFA scoring system 86.25 (32.50 – 100) 65 (35 – 100) 0.124#
One-Hour Bundle 91.53 ± 7.17 75.19 ± 15.37 0.022@
-Collecting blood sample for lactate 100 85 (20 – 100) 0.027#
-Collecting blood culture 100 60 (20 – 100) 0.011#
-Broad-spectrum antibiotics therapy 70 (50 – 100) 70 (50 – 100) 0.777@
-Intravenous Fluid Replacement 100 87.5 (25 – 100) 0.027#
-Vasoactive drugs administration 86.50 ± 14.96 78.50 ± 13.68 0.283@
@Unpaired T-test, mean ± SD; #Mann-Whitney test, median (min.–max.)
The value in this table was in percentage
The value is determined with our validated assessment tool with the maximum score for diagnosing sepsis is 40 (100%), one-hour bundle implementation is 65 (100%), collecting blood sample for lactate is 10 (100%), collecting blood culture is 10 (100%), broad-spectrum antibiotics therapy is 10 (100%), intravenous fluid replacement is 10 (100%), and vasoactive drugs administration is 25 (100%).
after training. 7 Training with simulation modalities with higher tool fidelity provides better results in training. Simulation-based training is a training which has been widely recommended as a teaching strategy to improve services to patients. 9 In several education centres, simulation-based training has been integrated into their education programs and has been proven to improve performance, particularly skills of both individuals and teams involved in the training. 10 Simulations allow for repetitive learning in medical cases which are life threatening in everyday care. Similarly, safety is of prime importance in the anesthesiology. Anesthesiology pioneered the use of simulation in training. Human factors play a big part in critical incidents. Understanding and identifying key cognitive errors specific to the practice of intensive care and anesthesiology is the first step in metacognition training and strategies to prevent these errors and improve patient safety. Manikins can enhace the ability of the clinicians regarding quick recognition and early response to life threatening scenios. 11, 12
The skills evaluated in this study were the skill to conduct initial assessment of the patient, to diagnose sepsis using SOFA score and to perform the one-hour bundle afterwards. The results show that the two of the skill components (diagnosing sepsis with SOFA score and conducting the one-hour sepsis bundle) tested in this training were better in the simulation group than in the discussion group. There was a significant difference in the skills of performing one-hour bundle points as a whole (p = 0.022). Of the five components of the one-hour bundle that need to be done, there were significant differences between the two groups in the subjects’ skills in drawing blood samples and cultures, as well as skills in fluid therapy management.

Better team skills in the simulation group might be because the simulation group has a better picture of performing the expected skills during practice. In the simulation practice, they also used HFM, which has been proven to improve effectivity in training. 6 Also, the team developed a common perception of the team in handling sepsis cases due to the simulation practice. This is important because the information obtained and understood by each individual in the group may vary, even though the training is held in groups. Therefore, the same perception is needed in order for a medical team to work optimally and in harmony. 6
The implementation of one-hour sepsis bundle in the clinical practice requires a good mutual understanding by each clinician. Previous research on sepsis in 2017 stated that one of the obstacles in practicing sepsis management with new clinical guidelines is the lack of understanding and confidence in the effectiveness of the sepsis bundle. 7 It is hoped that clinicians will have more confidence in managing patients and produce better service outcomes for patients with simulation training on the one-hour sepsis bundle. We also recommend that the simulation training includes HFM as not only it is available for wide variety of specialized, 9 it also allows the trainee to be involved in uncommon and critical case scenarios. 10
Limitations
Our study was only conducted with relatively small sample size, as it was conducted in one hospital. We also conducted lectures before the training started which could potentially increase the participants’ knowledge and performance. However, it is more likely that these increments are due to the combination of simulation and lectures as the simulation groups had higher scores compared to the discussion groups.

Conclusion
Simulation-based training is superior than case-based discussion training to improve team performance in implementing one-hour sepsis bundle.
Data availability
The numerical data generated during this research is available with the authors.

Acknowledgement
We gratefully thank Faculty of Medicine, Universitas Indonesia for the PUTI grant (NKB-2247/UN2.RST/HKP.05.00/2020) which helped fund this project.

Conflict of interest
The study utilized a grant by Faculty of Medicine, Universitas Indonesia, and no external or industry funding was involved.

Authors’ contribution
AS, AT: Concept and design, analysis and interpretation data, critical revision of the manuscript

SK, FS, AN, TF: Concept and design, drafting the manuscript, critical revision of the manuscript

JP, VA, ST: Concept and design, drafting the manuscript, statistical analysis

Reference
  1. Costar DM, Hall KK. Improving team performance and patient safety on the job through team training and performance support tools: a systematic review. J Patient Saf. 2020;16(3S Suppl 1):S48-S56. [PubMed] DOI: 1097/PTS.0000000000000746
  2. Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, et al. Saving lives: A meta-analysis of team training in healthcare. J Appl Psychol. 2016;101(9):1266-1304. [PubMed] DOI: 1037/apl0000120
  3. Gillon S, Radford S, Chalwin R, Devita M, Endacott R, Jones D. Crisis resource management, simulation training and the medical emergency team. Crit Care Resusc. 2012;14(3):227-35. [PubMed]
  4. McEwan D, Ruissen GR, Eys MA, Zumbo BD, Beauchamp MR. The effectiveness of teamwork training on teamwork behaviors and team performance: A systematic review and meta-analysis of controlled interventions. PLoS One. 2017;12(1):e0169604. [PubMed] DOI: 1371/journal.pone.0169604
  5. Trovato G. SEPSIS. educational and best practice frontiers. beyond the boundaries of fatality, enhancing clinical skills and precision medicine. Ther Clin Risk Manag. 2020;16:87-93. [PubMed] DOI: 2147/TCRM.S232530
  6. Watters C, Reedy G, Ross A, Morgan NJ, Handslip R, Jaye P. Does interprofessional simulation increase self-efficacy: a comparative study. BMJ Open. 2015 Jan 13;5(1):e005472. [PubMed] DOI: 1136/bmjopen-2014-005472
  7. Sørensen JL, Østergaard D, LeBlanc V, Ottesen B, Konge L, Dieckmann P, et al. Design of simulation-based medical education and advantages and disadvantages of in situ simulation versus off-site simulation. BMC Med Educ. 2017 Jan 21;17(1):20. [PubMed] DOI: 1186/s12909-016-0838-3
  8. Aldosary FF. The use of high-fidelity mannequin training to improve the quality of healthcare providers’ performance of CP. J Transl Sci. 2018;4(2). DOI: 15761/JTS.1000213
  9. Armenia S, Thangamathesvaran L, Caine AD, King N, Kunac A, Merchant AM. The role of high-fidelity team-based simulation in acute care: A systematic review. Surg J. 2018;4(3):e136-51. [PubMed] DOI: 1055/s-0038-1667315
  10. Geis GL, Wheeler DS, Bunger A, Militello LG, Taylor RG, Bauer JP, et al. A validation argument for a simulation-based training course centered on assessment, recognition, and early management of pediatric sepsis. Simul Healthc. 2018 Feb;13(1):16-26. [PubMed] DOI: 1097/SIH.0000000000000271
  11. Husson N, Carreira C, Babo N. One shock after another; simulation can prevent fixation errors: A case report. Anaesth Pain Intensive Care. 2019;23(4):401-403. [PubMed] DOI: 35975/apic.v23i4.1179
  12. Khan TH, Khalid A. CPR training models improve technical skills of the resuscitators. Anaesth Pain Intensive Care 2001;5(1):41-44. [FreeFullText]