Distributed practice. The more the merrier? A randomised bronchoscopy simulation study

RESEARCH ARTICLE

Distributed practice. The more the merrier? A randomised bronchoscopy simulation study

Anne Sofie Bjerrum1*, Berit Eika2, Peder Charles3 and Ole Hilberg1

1Department of Respiratory Diseases, Aarhus University Hospital, Aarhus, Denmark; 2Rector’s Office, Aarhus University, Aarhus, Denmark; 3Centre for Health Sciences Education, Aarhus University, Aarhus, Denmark

Abstract

Introduction: The distribution of practice affects the acquisition of skills. Distributed practice has shown to be more effective for skills acquisition than massed training. However, it remains unknown as to which is the most effective distributed practice schedule for learning bronchoscopy skills through simulation training. This study compares two distributed practice schedules: One-day distributed practice and weekly distributed practice.

Method: Twenty physicians in training were randomly assigned to one-day distributed or weekly distributed bronchoscopy simulation practice. Performance was assessed with a pre-test, a post-test after each practice session, and a 4-week retention test using previously validated simulator measures. Data were analysed with repeated measures ANOVA.

Results: No interaction was found between group and test (F(4,72) <1.68, p>0.16), except for the measure ‘percent-segments-entered’, and no main effect of group was found for any of the measures (F(1,72)< 0.87, p>0.36), which indicates that there was no difference between the learning curves of the one-day distributed practice schedule and the weekly distributed practice schedule.

Discussion: We found no difference in effectiveness of bronchoscopy skills acquisition between the one-day distributed practice and the weekly distributed practice. This finding suggests that the choice of bronchoscopy training practice may be guided by what best suits the clinical practice.

Keywords: medical education; simulation; training methods; curriculum design; bronchoscopy

Citation: Med Educ Online 2016, 21: 30517 - http://dx.doi.org/10.3402/meo.v21.30517

Responsible Editor: Colleen Colbert, Cleveland Clinic Lerner College of Medicine, USA.

Copyright: © 2016 Anne Sofie Bjerrum et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License, allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.

Received: 2 December 2015; Revised: 19 March 2016; Accepted: 22 March 2016; Published: 10 May 2016

Competing interests and funding: The authors report no declarations of interest.

*Correspondence to: Anne Sofie Bjerrum, Department of Respiratory Diseases, Aarhus University Hospital, Norrebrogade 44, Bygn 2B, DK-8000 Aarhus, Denmark, Email: annbjerr@rm.dk

 

Simulation-based training has become an integrated part of the training of many procedural and surgical skills because it offers students and physicians in training an opportunity to practise their skills in a safe and stress-reduced environment. To optimise the skills training laboratory experience, flexible and feasible training methods are needed. The scheduling of such practice is shown to influence skills acquisition in psychomotor as well as in cognitive domains; studies within these domains have shown that distributed practice is superior to massed practice for tasks like typing, ball toss, and second-language learning (1, 2). Massed practice refers to a practice schedule that allows no periods of rest in between practice, whereas distributed practice refers to a schedule where periods of practice are interspersed with periods of rest (3). The underlying mechanisms driving the beneficial effects of distributed practice are not well understood. Theoretical explanations for the effect of distributed practice relate these effects to the larger number of retrieval cues in distributed practice than in massed practice, which may facilitate recollection (4).

Within the context of medical clinical skills courses, only distributed practice schedules are relevant to examine as even short courses are variations of distributed practice. Several distributed practice schedules have been examined within medical procedural skills learning studies with diverse results. Verdaasdonk et al. (3) compared a one-day distributed practice schedule with a practice schedule distributed within three consecutive days for learning basic laparoscopic skills on a virtual reality simulator (3). They found that the latter practice was more effective than the former, but this superiority was noted only for the ‘time score’. Weekly and monthly practice schedules have also been studied. Moulton et al. (5) found a strong distributed practice effect of weekly distributed practice compared with one-day distributed practice (practice distributed within the same day) for learning microvascular anastomosis. Mitchell et al. (6) found a weekly distributed practice schedule and a monthly distributed practice schedule to be equally effective for learning microvascular anastomosis. Initial work on distributed practice schedules within medical simulation training has shown various beneficial effects of several distributed practice schedules ranging from one-day distributed practice to monthly distributed practice. Two reviews on the distributed practice effect within the psychomotor domain and the cognitive domain showed that the magnitude of the distributed practice effect found in previous studies was influenced by task type, task complexity, and the interval between practice sessions (1, 2). Previous medical procedural skills learning studies have focused on suturing tasks and basic hand–eye coordination tasks which differ much in terms of tasks and their complexity from those involved in bronchoscopy skills acquisition wherefore the optimal practice schedule for learning bronchoscopy remains unknown.

In the present study, we therefore compare the effectiveness of a weekly distributed practice schedule (a practice schedule with promising effects on skills acquisition within medical simulation training) with that of a one-day distributed practice schedule (the practice schedule of most clinical skills training courses) for learning bronchoscopy skills on a high-fidelity simulator.

Method

Study design

We conducted a randomised bronchoscopy simulation study on the effects of one-day distributed practice compared with weekly distributed practice.

Participants

Participants included 20 PGY 1–PGY 3 residents from the Department of Respiratory Diseases at Aarhus University Hospital in Denmark. They were enrolled in the study from 2010 to 2012. Approximately 25 PGY 1–PGY 3 residents were employed at the department at that period. Exclusion criteria were previous experience with bronchoscopy simulators and experience with performing real-life bronchoscopy. Two residents declined to participate in the study. Each participant provided informed consent and was randomly assigned to one-day distributed practice or weekly distributed practice using a randomisation procedure with closed envelopes. The number of participants in each group was determined with a power calculation. Ten participants were enrolled in each group.

The procedure

When performing a bronchoscopy, the airways are inspected with a flexible bronchoscope to the sub-segmental bronchus level. The scope is to be centred in the lumen to avoid wall collision and to avoid red-out (which occurs when the tip of the scope is pressed against the mucosa). Orientation is difficult and systematic inspection of the bronchial system is essential to secure a complete inspection and to reduce repeated sub-segmental bronchi inspection, thereby reducing procedure time and the risk of wall collisions and red-out.

The simulator

The simulator is an Accutouch high-fidelity virtual reality simulator from CAE Healthcare®, Quebec, Canada. It consists of a proxy flexible bronchoscope, a robotic interface device, a computer with simulation software, and a monitor. The virtual patient responds to the trainee’s instrumentation of the equipment with coughing, and the simulator provides force-feedback which further enhances the realism.

Training

Before training, participants were introduced to the simulator equipment by an instructor. An atlas of the airways was handed out and studied by the participants, and they watched a 15-min instruction video available on the simulator.

The training was divided into three sessions. Participants in the one-day distributed practice group received the three sessions within one day, and participants in the weekly distributed practice group attended the same three sessions within a period of 3 weeks, one session per week. Two major breaks (a coffee break and a lunch break) were scheduled within the programme for the one-day distributed practice group (see Fig. 1).

During each practice session, participants first watched the 15-min video available on the simulator; the video showed a consultant explaining and demonstrating how to navigate the bronchoscope. Thereafter, participants practised unsupervised on the same bronchoscopy simulator case three times. An instructor was present in the room, but no performance feedback was given to participants to avoid potential bias from an instructor. An atlas of the airways and a utility in the simulator software (road signs) naming the different segmental bronchi on the monitor were available for all participants throughout the training. A time limit of 20 min was allowed for each case. After completing each case, a case summary appeared on the monitor, presenting procedure time, wall collisions, time in red-out, and percent-segments-entered for the particular case performed.

Assessment

To establish the initial skill level, participants completed a pre-test consisting of a simulator case that was not part of the training. Learning outcomes were assessed after each of the three training sessions with post-test 1, post-test 2, and post-test 3 using the same test-case as the pre-test. To assess how well skills were retained, participants were assessed 4 weeks after the last post-test with a retention test using the same test-case used for pre- and post-tests.

During testing, participants were instructed to perform the simulated bronchoscopy as quickly and thoroughly as possible, and no simulator utility and no atlas of the airways were available during testing.

Participants were tested with bronchoscopy simulator metrics. Validity evidence has been accumulated previously for these simulator measures in the form of construct validity evidence (79). These included ‘procedure time’, ‘percent-segments-entered’, ‘wall collisions’, and ‘red-out’. The combined measure ‘percent-segments-entered-per-minute’ was chosen as the primary outcome measure as we believe that it reflects bronchoscopy skills learning to a higher extent than ‘percent-segments-entered’ and ‘procedure time’ in isolation.

Ethics

Participants provided informed consent and anonymity was guaranteed. As this study was conducted in a non-clinical setting, an exemption letter was issued from the Danish local ethical committee.

Statistics

Data were analysed using STATA version 11.0 (Stata-Corp LP, College Station, TX, USA).

Data were log transformed in order to fulfil the assumptions necessary for using univariate repeated measures analyses of variance (ANOVA). Separate univariate repeated measures ANOVAs were applied on each variable with test (pre-test, post-test 1, post-test 2, post-test 3, and retention test) as within-subjects factor and group as between-subjects factor. Post-hoc analyses of group differences were Bonferroni corrected. A significance level of 0.01 was used as five pairwise comparisons were made on each variable. For all other analyses, a significance level of 0.05 was used.

Results

Twenty physicians in training participated in the study. Pre-test, post-tests (1, 2, and 3), and 4-week retention tests were collected for all physicians in training.

Fig 1

Fig. 1.  Study flow chart.

Test data for each variable (mean±standard error) are depicted in Fig. 2. The results of the repeated measures ANOVAs are outlined in Table 1.

Fig 2

Fig. 2.  Simulator metrics test scores (mean+standard error) as a function of pre-test, post-test 1, post-test 2, post-test 3, and retention test for the daily distributed practice group and the weekly distributed practice group.


Table 1. Statistical analyses from the repeated measures analyses of variance (ANOVA) on log-transformed measures from the pre-test, post-tests, and retention tests
  Test×group interaction Group main effect Test main effect
Segments/minute F(4,72)=0.43, p=0.79 F(1,18)=0.03, p=0.87 F(4,72)=52.89, p<0.0001
Collisions F(4,72)=1.68, p=0.16 F(1,18)=0.07, p=0.80 F(4,72)=0.76, p=0.55
Red-out F(4,72)=0.37, p=0.83 F(1,18)=0.87, p=0.36 F(4,72)=13.95, p<0.0001
Time F(4,72)=0.99, p=0.42 F(1,18)=0.00, p=0.95 F(4,72)=110.11, p<0.0001
Segments F(4,72)=2.89, p=0.03 F(1,18)=0.11, p=0.75 F(4,72)=58.23, p<0.0001

A significant main effect of the test was found for all measures (F(2,72) > 11.11, p<0.0001) except for the variable ‘wall collisions’, which indicates a significant performance improvement from pre-test to post-test and retention test. No main effect of group was found for any of the variables (F(1,72)< 0.87, p>0.36), which indicates no difference in test scores between the two distributed practice conditions. Except for the variable ‘percent-segments-entered’, no interaction between group and test was found for any of the variables, that is, the learning curves were parallel.

Post-hoc analyses of the significant interaction between test and group for the measure ‘percent-segments-entered’ showed no significant differences between the two practice groups at any time point.

Post-hoc analyses of the significant main effect of test on the variables ‘percent-segments-entered-per-minute’, ‘percent-segments-entered’, ‘procedure time’, and ‘red-out’ showed a significant increase in test score from pre-test to the final post-test (post-test 3) (p < 0.001 for all variables) and retention test (p < 0.001 for all variables). Deterioration in performance from the final post-test (post-test 3) to retention test was seen for all variables, as shown in Fig. 2; however, the deterioration was not statistically significant for any of the variables.

Discussion

In this study, we compared the effect of two different distributed practice schedules (one-day distributed practice and weekly distributed practice) on the acquisition of bronchoscopy skills. We found no difference in effectiveness of skills acquisition between one-day distributed practice and weekly distributed practice, that is, we found no additional effect by distributing the practice beyond a single day.

This result differs from the results reported in most previous studies on distributed practice within medical procedural and surgical skills training (3, 5, 10), which show that daily and weekly distributed practices are more effective for skills acquisition than one-day distributed practice. However, important differences exist between our study and previous studies, and these differences may have influenced the distributed practice effect. Most importantly, there are differences between the task types and the complexity of the tasks examined in previous studies and our study. The skills examined in previous surgical skills training studies were basic hand–eye coordination tasks in laparoscopy skills learning (3, 10) and suturing tasks in microvascular anastomosis training (5, 6). The tasks involved in basic laparoscopy skills learning and suturing contain a higher degree of psychomotor skills components than the tasks involved in performing and learning bronchoscopy skills. Furthermore, the orientation tasks and pattern recognition tasks involved in bronchoscopy skills acquisition are cognitively more complex than the basic hand–eye coordination tasks and suturing tasks. The influence of task type and task complexity on the distributed practice effect is discussed in a review by Donovan et al. (2) They argue that the strong distributed practice effect is limited to tasks that are low in overall complexity and mental requirements but high in physical requirements, such as ball tossing and typing (2). Thus, it is likely that the training of a task like microvascular suturing, which involves a high degree of psychomotor involvement but possibly a lower degree of overall complexity, benefits more from a weekly distributed practice schedule than does the training of bronchoscopy. Consequently, the results of our study support a previously proposed association: that the size of the distributed practice effect appears to depend on task type and task complexity as well as the length of time in between practice sessions (2, 11).

Our results were obtained in an unsupervised practice setting. Participants from both practice groups (both the one-day distributed practice group and the weekly distributed practice group) showed a relatively slow performance improvement from pre-test to post-test compared with other bronchoscopy simulation studies that included feedback (12, 13). The most important difference in the training set-up between these studies is the inclusion/exclusion of feedback and supervision. Thus, our results reveal the effects of training in an unsupervised training setting. Supervision and feedback is important in complex skills learning, and studies have consistently shown larger learning gains for practice curricula that include feedback than for those that do not (1416). However, in our study the unsupervised practice setting was chosen to enhance the feasibility of a weekly distributed practice schedule. Furthermore, the unsupervised practice setting secured an experimental study set-up without the potential bias that may emanate from an instructor providing feedback.

Feasibility is an important feature to account for in the practice curriculum. In an unsupervised practice setting, the scheduling of practice involves only the physician in training. In such a setting, a weekly distributed practice schedule represents a flexible and feasible practice schedule. In a supervised practice setting, however, both physicians in training and instructors have to coordinate their schedules to be able to attend the practice sessions at the same time. The coordination of two or even more individuals’ practice schedules may be challenged by the demanding and busy clinical life at most hospitals. As the results of our study challenge the superiority of a weekly distributed practice schedule (at least for bronchoscopy skills acquisition), a less distributed practice schedule like the one-day distributed practice schedule may be favoured in a practice setting that includes feedback from an instructor, and a weekly distributed practice schedule may be favoured in an unsupervised practice setting.

Our study has several limitations. In this study, bronchoscopy skills were assessed in an experimental setting. Although it has been shown that bronchoscopy skills acquired through simulation training transfer readily to the operating room (17), investigators are yet to show how specific practice interventions transfer to the operating room. In addition, we only assessed performance with simulator-measured metrics. The objective, structured assessments (OSATS) developed for bronchoscopy simulation were deselected due to the lack of opportunity for blinding of the assessors (18) and due to lack of feasibility (19). Another potential limitation is generalisation of results from this study. As the effects of various distributed practice schedules are likely to be task dependent and dependent on the complexity of the task at hand, the results of our study may not be readily generalised to the practice of medical procedural skills that are less complex than bronchoscopy skills learning. Finally, the number of participants was small, although similar in size to previous distributed practice studies, and this may also affect generalisability.

Conclusions

We found no difference in effectiveness of basic bronchoscopy skills acquisition between one-day distributed practice and weekly distributed practice. As opposed to previous studies, we found no additional effect of distributing the practice beyond a single day. This discrepancy may be explained by the differences in task type and task complexity between bronchoscopy skills learning and learning to suture and learning basic hand–eye coordination tasks associated with laparoscopy. Our findings suggest that the choice of bronchoscopy training practice may be guided by what best suits the clinical practice.

Authors’ contributions

ASB, BE, PC, and OH were responsible for the study conception and design. ASB drafted the paper and performed data collection and data analyses. All authors contributed to the critical revision of the paper and approved the final manuscript for publication.

Ethical approval

The study was carried out in accordance with the Declaration of Helsinki. All participants gave written informed consent prior to the practice sessions and the anonymity of each participant was guaranteed.As this study was conducted in a non-clinical setting, an exemption letter was issued from the Danish local ethical committee.

References

  1. Cepeda NJ, Pashler H, Vul E, Wixted JT, Rohrer D. Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol Bull 2006; 132: 354–80. PubMed Abstract | Publisher Full Text
  2. Donovan JJ, Radosevich DJ. A meta-analytic review of the distribution of practice effect: now you see it, now you don’t. J Appl Psychol 1999; 84: 795–805. Publisher Full Text
  3. Verdaasdonk EG, Stassen LP, van Wijk RP, Dankelman J. The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc 2007; 21: 214–19. PubMed Abstract | Publisher Full Text
  4. Benjamin AS, Tullis J. What makes distributed practice effective? Cogn Psychol 2010; 61: 228–47. PubMed Abstract | PubMed Central Full Text | Publisher Full Text
  5. Moulton CA, Dubrowski A, MacRae H, Graham B, Grober E, Reznick R. Teaching surgical skills: what kind of practice makes perfect?: a randomized, controlled trial. Ann Surg 2006; 244: 400–9. PubMed Abstract | PubMed Central Full Text
  6. Mitchell EL, Lee DY, Sevdalis N, Partsafas AW, Landry GJ, Liem TK, et al. Evaluation of distributed practice schedules on retention of a newly acquired surgical skill: a randomized trial. Am J Surg 2011; 201: 31–9. PubMed Abstract | Publisher Full Text
  7. Moorthy K, Smith S, Brown T, Bann S, Darzi A. Evaluation of virtual reality bronchoscopy as a learning and assessment tool. Respiration 2003; 70: 195–9. PubMed Abstract | Publisher Full Text
  8. Ost D, DeRosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC. Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med 2001; 164: 2248–55. PubMed Abstract | Publisher Full Text
  9. Colt HG, Crawford SW, Galbraith O III. Virtual reality bronchoscopy simulation: a revolution in procedural training. Chest 2001; 120: 1333–9. PubMed Abstract | Publisher Full Text
  10. Mackay S, Morgan P, Datta V, Chang A, Darzi A. Practice distribution in procedural skills training: a randomized controlled trial. Surg Endosc 2002; 16: 957–61. PubMed Abstract | Publisher Full Text
  11. Cepeda NJ, Coburn N, Rohrer D, Wixted JT, Mozer MC, Pashler H. Optimizing distributed practice: theoretical analysis and practical implications. Exp Psychol 2009; 56: 236–46. PubMed Abstract | Publisher Full Text
  12. Bjerrum AS, Hilberg O, van GT, Charles P, Eika B. Effects of modelling examples in complex procedural skills training: a randomised study. Med Educ 2013; 47: 888–98. PubMed Abstract | Publisher Full Text
  13. Bjerrum AS, Eika B, Charles P, Hilberg O. Dyad practice is efficient practice: a randomised bronchoscopy simulation study. Med Educ 2014; 48: 705–12. PubMed Abstract | Publisher Full Text
  14. Boyle E, O’Keeffe DA, Naughton PA, Hill AD, McDonnell CO, Moneley D. The importance of expert feedback during endovascular simulator training. J Vasc Surg 2011; 54: 240–8. PubMed Abstract | Publisher Full Text
  15. Kruglikova I, Grantcharov TP, Drewes AM, Funch-Jensen P. The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity Virtual-Reality simulation: a randomised controlled trial. Gut 2010; 59: 181–5. PubMed Abstract | Publisher Full Text
  16. Mahmood T, Darzi A. The learning curve for a colonoscopy simulator in the absence of any feedback: no feedback, no learning. Surg Endosc 2004; 18: 1224–30. PubMed Abstract | Publisher Full Text
  17. Wahidi MM, Silvestri GA, Coakley RD, Ferguson JS, Shepherd RW, Moses L, et al. A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows. Chest 2010; 137: 1040–9. PubMed Abstract | Publisher Full Text
  18. Davoudi M, Osann K, Colt HG. Validation of two instruments to assess technical bronchoscopic skill using virtual reality simulation. Respiration 2008; 76: 92–101. PubMed Abstract | Publisher Full Text
  19. Konge L, Arendrup H, von BC, Ringsted C. Using performance in multiple simulated scenarios to assess bronchoscopy skills. Respiration 2011; 81: 483–90. PubMed Abstract | Publisher Full Text
About The Authors

Anne Sofie Bjerrum

Denmark

Berit Eika

Denmark

Peder Charles

Denmark

Ole Hilberg

Denmark

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Related Content