Students’ Expectations and Experiences of Meaningful Simulation-Based Medical Education

Tuulikki Keskitalo & Heli Ruokamo

University of Lapland, Faculty of Education
Centre for Media Pedagogy (CMP)
Rovaniemi, Finland
Email: firstname.lastname@ulapland.fi

Abstract

This study aims to investigate students’ expectations and experiences of meaningful learning in simulation-based learning environments. We set the following research question: How do students’ experiences of meaningful simulation-based learning correspond to their expectations? The students’ (n = 87; male 51, female 36) pre- and post-questionnaires were analyzed using statistical methods. The results indicated that students’ expectations and experiences of meaningful learning were positive, and for most statements, there were statistically significant differences between the mean pre-questionnaire rating and the mean post-questionnaire rating, thereby indicating that students’ actual experiences of simulation-based learning were more positive than their expectations. Thus, students’ experiences exceeded their expectations.

Keywords: simulation-based medical education, meaningful learning, students, expectations, experiences, quantitative study

Introduction

Simulation and virtual realities have gained significant attention within the health-care and medical education industries around the world over the past few years (Helle & Säljö, 2012). Interest in the pedagogical use of these environments—that is, when, how and what to use these technologies for in education to support effective learning—has also increased. According to Rall and Dieckmann (2005), “Simulation, in short, means to do something in the ‘as if’, to resemble ‘reality’ (always not perfectly, because then it would be reality again), e.g., to train or learn something without the risks or costs of doing it in reality” (p. 2). However, in this study, we use the concept of a simulation-based learning environment (SBLE) because we want to emphasize the learning purpose of these technologically rich yet safe learning environments. According to Keskitalo (2015), these should also be considered complex cultural, social, physical, and pedagogical environments. Importantly, simulations can involve technology, but not all of them do.

The overall aim of this study was to develop a pedagogical model for health-care simulation educators (see Keskitalo, 2015; Keskitalo & Ruokamo, 2015; Keskitalo, Ruokamo, & Gaba, 2014). The pedagogical model—a theory-based model—can be used to design curriculums, plan instruction, and create instructional materials (Joyce & Weil, 1980). Generally, pedagogical models provide ideas for teaching and learning, and they can appear in various types of pedagogical solutions. The principles of this pedagogical model were derived from the sociocultural theory of learning (Lave & Wenger, 1991; Vygotsky, 1978), meaningful learning (e.g., Ausubel, 1968; Jonassen, 1995; Ruokamo & Pohjolainen, 2000), and previous pedagogical models of simulation-based health-care education (Dieckmann 2009; Joyce, Calhoun, & Hopkins, 2002). Within the pedagogical model, sociocultural theory helps us to understand the complexity of learning and the development of expertise from a wider perspective. As the theory suggests, there is constant interplay between individual and social factors (Palincsar, 1998; Säljö, 2009). The characteristics of meaningful learning were chosen as a general framework and can be considered ideal goals when creating an effective learning environment. Moreover, meaningful learning characteristics can help the facilitators to shed light on the things that are known to enhance learning (Keskitalo, 2015). The main contribution of previous pedagogical models has been in helping to structure the simulation-based learning process.

This particular study aimed to investigate students’ expectations and experiences of meaningful learning in an SBLE in order to gain a deep understanding of the conditions and processes that lead to effective learning in that environment and to further develop both the pedagogical model and educational practice. Moreover, there are only a limited number of studies related to the expectations of teaching and learning in SBLEs (e.g., Keskitalo, 2012). For this study, we set the following research question: How do students’ experiences of meaningful simulation-based learning correspond to their expectations? This article first focuses on the characteristics of meaningful learning and the previous research related to the topic. Then, the research question and methods are introduced. Finally, the research results are summarized, followed by a discussion.

Theoretical Background

Meaningful learning is a concept invented by Ausubel (1968). It resembles the constructivist idea of learning, whereby new information is assimilated into what the learner already knows (Ausubel, Novak, & Hanesian, 1978). According to this view, both the learning materials and the task must be meaningful, and the learners must engage with the learning process (Ausubel et al., 1978). Later, Jonassen (1995) developed Ausubel’s ideas in a more social constructivist direction and described seven characteristics of meaningful learning (see also Nevgi & Löfström, 2005). In previous studies (e.g., Keskitalo, 2015; Keskitalo et al., 2014), we developed those characteristics in a more practice-oriented direction and thus included experiential, experimental, emotional, socio-constructive, collaborative, active, responsible, reflective, critical, competence-based, contextual, goal-oriented, self-directed, and individual characteristics. We argue that these selected characteristics describe optimal student training in this context and can foster students’ effective and meaningful learning. As Keskitalo (2015) points out, with these characteristics in mind, facilitators may consider teaching and learning processes from a wider perspective and develop pedagogical practices that are effective, innovative, and meaningful.

Regarding the characteristics of meaningful learning, the terms “experiential and experimental learning” indicate that students can use their own experiences as a starting point for learning (Kolb, 1984) and gain valuable experience before entering health-care practice (Cleave-Hogg & Morgan, 2002; Gaba, 2004). The emotional characteristics of learning reflect the fact that emotions are always intertwined with learning (Damasio, 2001; DeMaria et al., 2010; Immordino-Yang & Faeth, 2010). A simulation setting is expected to arouse strong feelings and motivation to learn, but it can also lead to disbelief because of its artificial nature (Dieckmann, Gaba, & Rall, 2007). Simulation-based education is usually based on students’ collaboration and interaction with other students, the environment, the simulators, and other technical devices. Thus, learning can be viewed as socio-constructive and collaborative in nature (Lave & Wenger, 1991; Vygotsky, 1978). The active and responsible aspects of learning refer to the fact that when training, students are active and are expected to be responsible for their own learning (Fanning & Gaba, 2007; Jonassen, 1995). In the simulation setting, debriefing has an important role because it allows students an opportunity to reflect on their learning (Issenberg, McGaghie, Petrusa, Gordon, & Scalese, 2005). Additionally, students should critically evaluate their own learning, their acquired information, and their learning environment. Learning is also contextual (Vygostky, 1978)—that is, knowledge is best learned when it is taught and practiced in a context that resembles real life (Bransford, Brown, & Cocking, 1999). Therefore, in order to promote learning transfer, facilitators should structure the training with specific learning objectives in mind based on the competencies that students will need to handle real-life situations (Fanning & Gaba, 2007). Students are also encouraged to set their own goals for learning in relation to the course goals and their own levels of expertise. During the learning process, it is also important to reevaluate these goals (Brockett & Hiemstra, 1991). Learning also differs among individuals (De Corte, 1995); therefore, facilitators should provide individual guidance and feedback to each student. Although many students enjoy simulation-based education, there are those who do not.

Previous Studies on Expectations and Experiences of Simulation-Based Healthcare Education

In this study, the term “student expectations” refers to students’ expectations about the learning process in SBLE (cf. Keskitalo, 2012). Previous studies in the service delivery sector have divided expectations into expected, or predictive, normative, and experience-based (Higgh, Polonsky, & Hollick, 2005; Shewchuk et al., 2007). Expected, or predictive, expectations refer to the expectations that students have about learning in these environments. Normative expectations are, in students’ opinions, what should happen in these learning environments. Experience-based expectations are based on students’ previous knowledge and experience (Edberg & Andersson, 2015).

Despite the increased interest in this research area, there are only a limited number of studies related to the expectations of teaching and learning in SBLEs (e.g., Keskitalo, 2012). Prior research has concentrated on discovering students’ expectations about studying medicine or nursing. For example, Miles and Leinster (2007) compared first-year medical students’ expectations and experiences. The results revealed that students’ expectations were more positive than their actual experiences regarding learning and teachers, their academic self-perceptions, and their social self-perceptions. The study also revealed that facilitators were poor at providing feedback and constructive criticism, the learning goals were poorly articulated, and the timetable for courses was not very well planned. Also, the support system was poorer than expected for stressed students. In a recent study, Edberg and Andersson (2015) discovered that students’ expectations about the curriculum had changed from a biomedical orientation to a nursing orientation, which meant that formerly, students had expected that biomedicine would constitute the main content of the curriculum. Edberg and Andersson (2015) concluded that this could be a sign of a paradigm shift from a medical orientation toward nursing science.

However, Keskitalo (2012) discovered that students had high expectations of activities involving simulations within health-care education. Adult students, in particular, seemed to expect a great deal. Students expected a great deal of simulation facilitators in that they expected the facilitators to be competent and well prepared for teaching (Keskitalo, 2012). To sum up, from a pedagogical viewpoint, the use of simulations in education is primarily expected to provide students with active and experiential learning opportunities in order for them to better integrate theory into practice (Cleave-Hogg & Morgan, 2002; Gaba, 2004; Keskitalo, 2012; Rall & Dieckmann, 2005). In contrast, from a health-care practice viewpoint, the use of simulations is expected to eventually improve patient care and safety by providing professionals with opportunities to practice on rare cases and to use teamwork, among other goals. Therefore, expectations for simulations in health-care education tend to be high.

There is also notable evidence that students enjoy simulation-based education (e.g., Brewer, 2011; Keskitalo et al., 2014; Konia & Yao, 2013). Many describe it as fun, experiential, and safe, but in an SBLE, students also have the opportunity to learn at their own pace (Brewer, 2011). In addition to being experiential and enjoyable, the use of simulations enhances learning, such as via its moderate effects on clinical practice (Chakravarthy, Haar, McCoy, Denmark, & Loftipour, 2011; Cook et al., 2011; Eaves & Flagg, 2001; Hayden, Smiley, Alexander, Kardong-Edgren & Jeffries, 2014; Konia & Yao, 2013; Paige, Arora, Fernandez, & Seymour, 2015). In an SBLE, students have the opportunity to learn clinical skills and basic science concepts that are otherwise difficult to understand (Beauchesne & Douglas, 2011; Chakravarthy et al., 2011; Hope, Garside & Prescott, 2011). SBLEs help in combining the various forms of knowledge into a bigger picture (Bland, Topping, & Wood, 2011). The use of simulations also seems to enhance students’ self-confidence (Figueroa, Sepanski, & Goldberg, 2013; Gough, Hellaby, Jones, & MacKinnon, 2012; Hope et al., 2011; Morgan & Cleave-Hogg, 2002; Paskins & Peile, 2010). For example, Hope et al. (2011) discovered many benefits of using simulations in nursing education. The students felt that they were more ready for practice and that the simulation-based education enhanced their humanistic abilities and their technical and problem-solving skills, which improved their confidence. Similar results have been found within the medical education environment (e.g., Konia & Yao, 2013).

Research Question and Methods

With this as a background, this study aimed to answer the following research question: How do students’ experiences of meaningful simulation-based learning correspond to their expectations?

Data Collection and Analysis

Empirical data were collected in two simulation centers at Stanford University in the years 2010–2014. The data were collected from facilitators and students using group interviews, observations, video recordings, and pre- and post-questionnaires. For this article, we analyzed students’ (n = 87; male 51, female 36) pre- and post-questionnaires, which were distributed at the beginning and at the end of the course, respectively. The students were mainly anesthesia residents and medical students. They were studying anesthesia crisis resource management, emergency medicine, and anesthesia clerkship. The mean age of the respondents was 30 years old; the youngest respondent was 25 years old, and the oldest was 40 years old. Ten percent of the students had no prior experience in simulation-based education, but most of the students had undergone simulation-based education before (one session, 14%; two sessions, 7%; three sessions, 9%; three or more, 48%). Altogether, the data were collected from 19 courses, which lasted from three to nine hours. All the students who took part in the courses participated voluntarily in the research as well. The chosen courses were those that were running during the research period at Stanford and to which the researchers had access. During the courses, all the activities were completed in a group format created by the facilitators. During the scenarios, there was usually one student who had a leading role (the “hot seat” person) and would call on the others to help. In some of the courses—for example, the anesthesia crisis resource management course—all the students could be leaders one by one; however, there were also courses in which all the students did not have the chance to experience being a leader. Those students not taking part in the scenario watched from a separate room via a television. Before the study, research permission was applied for and approved by the institutional review board, and thereafter, consent was obtained from the participants.

The pre-questionnaire consisted of Likert-type questions related to the students’ expectations of the teaching and learning processes in an SBLE. Each of the 52 statements was scored on a continuum (1 = does not describe my expectations at all; 5 = describes my expectations very well). In addition, 29 Likert-type questions (0 = not at all, 5 = to a great extent) focused on the emotions that students experienced during the course. The students were asked to evaluate the degree to which they felt a given emotion (e.g., enjoyment of studying, boredom, sense of community, etc.) before and after the course. Five questions were also aimed at collecting students’ background information, and one open question gave the students space to write any other comments they had. The post-questionnaire questions were similar to those in the pre-questionnaire but dealt with students’ experiences. The detailed description of the development and usage of these questionnaires can be found in a previous article (Keskitalo, 2012). For further analysis, we selected statements that reflected the characteristics of meaningful learning (27 items) (see also Nevgi & Löfström, 2005; Ruokamo, Hakkarainen, & Eriksson, 2012). The data were analyzed using the frequencies, means, and standard deviations of the individual statements. The sum variables of the items measuring meaningful learning were computed using factor analysis (principal component analysis, see Appendices 1 and 2) and Cronbach’s alpha. The paired-samples t-test was used to compare the differences between the students’ expectations and their experiences (see Miles & Leinster, 2007).

Findings

The students estimated their expectations and experiences regarding the meaningfulness of simulation-based learning based on the 27 items from the pre- and post-questionnaires. Table 1 displays the reliabilities for each of the subscales that are used to measure meaningful learning in this study. In addition, the subscales were renamed based on the statistical analysis (factor analysis, principal component analysis) and theoretical reasoning (Ruokamo et al., 2012; Poikela, Ruokamo, & Teräs, 2015).

Table 1: Cronbach’s α Values for the Subscales

The reliabilities of the various subscales are acceptable. Only in the first subscale (concrete) is the alpha value below 0.7, which is considered to be the threshold for acceptable internal consistency in most social sciences research (Nunnally, 1978).

Table 2 reveals the results of the questionnaires, which indicate that students’ expectations and experiences were positive, and for most statements, there were statistically significant differences between the mean pre-questionnaire ratings and the mean post-questionnaire ratings. This indicates that students’ experiences were rated more highly than their expectations.

Table 2: Mean (Standard Deviation) Ratings of the Sum Variable (Bold), Examples of the Individual Statements Before and After the Course as well as Dissonances of Means

The results revealed that the students had the highest expectations of the socio-constructive (M = 4.23; SD = 0.71) characteristics of meaningful learning. When exploring the individual statements of the pre-questionnaire, students particularly expected to feel safe during the lessons (M = 4.34, SD = 0.77) and that the lessons will be applicable to the future work (M = 4.34; SD = 0.83). The lowest expectations that students had were related to the individual (M = 3.74; SD = 0.77) characteristics of meaningful learning. Students did not greatly expect that the course will take the students’ individuality into account (M = 3.72; SD = 1.01) or that the study skills that they have adopted will also work for them in this course (M = 3.58; SD = 0.98).

The analysis of the post-questionnaires revealed that students experienced the courses favourably in terms of the socio-constructive (M = 4.48; SD = 0.59) and emotional (M = 4.48; SD =0.55) characteristics of meaningful learning.However, whenstudying individual items, students valued the courses’ competence-based characteristics as well because the individual statements training in simulation settings developed my competence (M = 4.56; SD = 0.63) and the lessons were applicable to my future work (M = 4.63; SD = 0.58) received the highest ratings in the post-questionnaire.

When analyzing the difference between expectations and experiences, students ranked their experiences significantly more highly than their expectations in the statements that measured the emotional (0.39***, p = 0.001) characteristics of meaningful learning. Regarding the emotional characteristics of meaningful learning, the results indicated that students felt safer and more motivated than they had expected. Individuality was not rated very highly in the pre-questionnaire (M = 3.74; SD = 0.77), although the experiences were significantly better than the students’ expectations in this regard (M = 4.04, SD = 0.77; 0.30*, p = o.o5). The individual characteristics of learning also received the lowest ratings in the post-questionnaire.

Discussion and Concluding Remarks

Even though the study’s results are somewhat descriptive, this research provided us with useful information about the expectations and experiences of student learning in an SBLE. This study complements the existing literature on students’ experiences with information about their expectations because these can be important determinants of the learning event (Dieckmann & Yliniemi, 2012).

The results suggest that although students have quite high expectations of the training in SBLEs (cf. Keskitalo, 2012), their experiences were even higher. In this study, students rated their experiences more highly than their expectations, which stood in contrast with the results of previous research in the medical domain (Miles & Leinster, 2007).As the results suggest, students’ experiences have exceeded their expectations. Notably, most of the students had some prior experience with simulation-based training, which certainly affected their expectations. Therefore, students could have expressed their experience-based expectations in these questionnaires (Parasuraman, Zeithaml, & Berry, 1988). By examining students’ expectations and experiences, we are able to develop the areas of education in which meaningful learning characteristics should be more emphasized.

In this study, students experienced the emotional characteristics of learning significantly more highly than they expected to, which could be explained by the fact that during simulation-based education, special emphasis is usually placed on the emotional aspect of learning (e.g., DeMaria et al., 2010). Although the individual characteristic of meaningful learning was the least highly rated, their ratings were still positive. However, based on previous studies, this characteristic of meaningful learning is the one that requires most of the work in these collaborative settings because there may be students who expect more individualized guidance and feedback (Keskitalo et al., 2014). Therefore, future research should concentrate on how we can realize the individual characteristics of learning in these collaborative settings, for example, through individualized counselling sessions or clinical hours in SBLEs.

In summary, this study shed light on the students’ expectations, which are often under-recognized but still affect their learning experience in many ways. Furthermore, by comparing students’ expectations and experiences, we were also able to identify and address the areas with which students are least satisfied. In addition, we must continue this research to ensure that this questionnaire can be considered a valid measure of students’ expectations and experiences. In this study, Cronbach’s alpha values were calculated for each of the subscales; these values indicate that this questionnaire can be considered a valid measurement (Nunnally, 1978), except for the value that measured the expectations for the concreteness of learning in the pre-questionnaire (α=0.69). According to Tavakol and Dennick (2011), a low alpha value can depend on a number of things, including a number of questions, poor interrelatedness between items, and heterogeneous constructs. In future, we should conduct the study with a wider population and in other contexts in order to gain more reliable results and to develop a reliable and valid test for measuring the meaningful learning of students within a simulation-based learning environment. The principal component analysis also works better with a wider population. For example, the metacognitive variable includes only two items, and more research is needed to evaluate which items form a logical aggregate that can be used to describe and measure the metacognitive characteristics of meaningful learning. For now, these results should be interpreted with careful consideration. In addition, the research period was quite long, which is why the conditions for learning might have changed, and there were also different facilitators for some of the courses, which affected the condition for learning as well. However, the purpose of this paper was not to study the longitudinal effects of teaching and learning conditions, but rather to inquire into students’ expectations and experiences of meaningful simulation-based learning. This is, however, an interesting topic to cover in future research but with more participants.

However, the results of this study will enable us to continue designing a more user-friendly pedagogical model and ensure its improved integration into simulation-based education practices. This research offers useful insights regarding teaching and learning—especially for health-care teachers, teacher educators, instructor trainers, designers, and researchers—regarding how to plan more meaningful and effective simulation-based education.

References

Ausubel, D. P. (1968).Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston.

Ausubel, D. P., Novak, J. D., & Hanesian, H. (1978).Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston.

Beauchesne, M. A., & Douglas, B. (2011). Simulation: Enhancing Pediatric, Advanced, Practice Nursing Education. Newborn & Infant Nursing Reviews, 11(1), 28–34.

Bland, A., Topping, A., & Wood, B. (2011).A concept analysis of simulation as a learning strategy in the education of undergraduate nursing students. Nurse Education Today, 31(7), 664–670.

Bransford, J. D., Brown, A. L. & Cocking, R. R. (1999).How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Brewer, E. P. (2011). Successful techniques for using human patient simulation in nursing education. Journal of Nursing Scholarship, 43(3), 311–317.

Brockett, R. G., & Hiemstra, R. (1991). Self-direction in adult learning: Perspectives on theory, research, and practice. London: Routledge.

Cleave-Hogg, D., & Morgan, P. J. (2002). Experiential learning in an anaesthesia simulation centre: analysis of students’ comments. Medical Teacher, 24(1), 23–26.

Chakravarthy, B., Ter Haar, E., Bhat, S. S., McCoy, C. E., Denmark, T. K., & Lotfipour, S. (2011). Simulation in medical school education: Review for emergency medicine. Western Journal of Emergency Medicine 12(4), 461–466.

Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2011). Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. JAMA 306(9), 979–988.

Damasio, A. (2001). Descartesin virhe. Emootio, järki ja ihmisen aivot. [Descarte’s mistake. Emotions, intelligence and human brains.] Helsinki: Terra Cognita.

De Corte, E. (1995). Fostering cognitive growth: A perspective from research on mathematics learning and instruction. Educational Psychology, 30, 37–46.

DeMaria, S., Bryson, E. O., Mooney, T. J., Silverstein, J. H., Reich, D. L., Bodian, C., & Levine, A. I. (2010). Adding emotional stressors to training in simulated cardiopulmonary arrest enhances participant performance. Medical Education, 44(10), 1006–1015.

Dieckmann, P. (2009).Simulation setting for learning in acute medical care.In P. Dieckmann (Eds.), Using Simulations for Education, Training and Research (pp. 40–138). Germany: Pabst Science Publishers.

Dieckmann, P., Gaba, D., & Rall, M. (2007). Deepening the theoretical foundations of patient simulation as social practice. Simulation in Healthcare, 2(3), 183–193.

Dieckmann, P., & Yliniemi, P. (2012).Sociodrama and psychodrama and their relation to simulation in health care. In E. Poikela & P. Poikela (Eds.), Towards simulation pedagogy. Developing nursing simulation in a European network (pp. 40–49).Rovaniemi: Rovaniemi University of Applied Sciences.

Eaves, R. H., & Flagg, A. J. (2001). The U.S. Air Force pilot simulated medical unit: A teaching strategy with multiple applications. Journal of Nursing Education, 40(3),110–116.

Edberg, A.-K., & Andersson, P. L. (2015). The shift from a medical to a nursing orientation: A comparison of Swedish nursing students' expectations when entering the nursing degree programme in 2003 and 2013. Nurse Education Today, 35(9),78–83.

Fanning, R. M., & Gaba, D. M. (2007). The role of debriefing in simulation-based Learning. Simulation in Healthcare, 2(2), 115–125.

Figueroa, M. I., Sepanski, R., Goldberg, S. P., & Shah, S. (2013).Improving teamwork, confidence, and collaboration among members of a pediatric cardiovascular intensive care unit multidisciplinary team using simulation-based team training. Pediatric Cardiology, 34(3), 612–619.

Gaba, D. M. (2004). The future vision of simulation in health care. Quality and Safety in Health Care, 13(Suppl. 1), 2–10.

Gough, S., Hellaby, M., Jones, N., & MacKinnon, R. (2012). A review of undergraduate interprofessional simulation-based education (IPSE). Collegian, 19(3),153–170.

Hayden, J. K., Smiley, R. A., Alexander, M., Kardong-Edgren, S., & Jeffries, P. R. (2014). The NCSBN National Simulation Study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. Journal of Nursing Regulation, 5(2),3–40.

Helle, L., & Säljö, R. (2012). Collaborating with digital tools and peers in medical education: Cases and simulations as interventions in learning. Instructional Science, 40(5),737–744.

Hope, A., Garside, J., & Prescott, S. (2011).Rethinking theory and practice: Pre-registration student nurses experiences of simulation teaching and learning in the acquisition of clinical skills in preparation for practice. Nurse Education Today, 31(7), 711–715.

Immordino-Yang, M.H., & Faeth, M. (2010).The role of emotion and skilled intuition in learning. In D. Sousa (Eds.), Mind, Brain and Education: Neuroscience Implications for the Classroom (pp. 69–83). Bloomington, IN, USA: Solution Tree Press.

Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher, 27(1), 10–28.

Jonassen, D. H. (1995). Supporting communities of learners with technology: A vision for integrating technology with learning in schools. Educational Technology, 35(4), 60–63.  

Joyce, B., Calhoun, E., & Hopkins, D. (2002). Models of learning – Tools for teaching. (Second Edition.) Buckingham: Open University Press.

Joyce, B., & Weil, M. (1980). Models of teaching. (Second Edition.) New Jersey: Prentice/Hall International.

Keskitalo, T. (2012). Students’ expectations of the learning process in virtual reality and simulation-based learning environments. Australasian Journal of Educational Technology, 28(5), 841–856.

Keskitalo, T. (2015). Designing a pedagogical model simulation-based healthcare education. Acta Universitatis Lapponiensis 299. Rovaniemi, Lapland University Press.

Keskitalo, T., & Ruokamo, H. (2015). A pedagogical model for simulation-based learning in healthcare. Seminar.net: Media, Technology & Life-long Learning, 11(2).

Keskitalo, T., Ruokamo, H., & Gaba, D. (2014). Towards meaningful simulation-based learning with medical students and junior physicians. Medical Teacher, 36(3), 230–239.

Kolb, D. A. (1984). Experiential learning. Experiences as a source of learning and development. Englewood Cliffs. N. J.: Prentice Hall.

Konia, M., & Yao, A. (2013). Simulation a new educational paradigm? Journal of Biomedical Research, 27(2), 75–80.

Lave, J., & Wenger, E. (1991).Situated learning; Legitimate peripheral participation. Cambridge: Cambridge University Press.

Miles, S., & Leinster, S. J. (2007).Medical students’ perceptions of their educational environment: Expected versus actual perceptions. Medical Education, 41(3), 265–272.

Morgan, P. J., & Cleave-Hogg, D. (2002). Comparison between medical students’ experience, confidence and competence. Medical Education, 36(6), 534–539.

Nevgi, A., & Löfström, E. (2005).The quality of online learning: Teachers’ and students’ evaluation of meaningful learning experiences in web-based course. In S. Kiefer, J. Michalak, A. Sabanci & K. Winter (Eds.), Analysis of educational policies in comparative educational perspective (pp. 187–203). Linz: Trauner.

Nunnally, J. C. (1978).Psychometric theory. (2^nd ed.). New York: McGraw-Hill.

Paige, J. T., Arora, S., Fernandez, G., & Seymour, N. (2015). Debriefing 101: training faculty to promote learning in simulation-based training. American Journal of Surgery, 209(1), 126–131.

Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49(1), 345–375.

Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988).SERVQUAL: A multiple-item scale for measuring customer perceptions of service quality. Journal of Retail, 64(1), 12–40.

Paskins, Z., & Peile, E. (2010). Final year medical students’ views on simulation-based teaching: A comparison with the Best Evidence Medical Education Systematic Review. Medical Teacher, 32(7), 569–577.

Poikela, P., Ruokamo, H., & Teräs, M. (2015). Comparison of meaningful learning characteristics in simulated nursing practice after traditional versus computer-based simulation method: A qualitative videography study. Nurse Education Today, 35(2), 373–382.

Rall, M., & Dieckmann, P. (2005). Simulation and Patient Safety: The use of simulation to enhance patient safety on a systems level. Current Anaesthesia and Critical Care, 16(5),273–281.

Ruokamo, H., & Pohjolainen, S. (2000). Distance learning in multimedia networks project: Main results. British Journal of Educational Technology, 31(2), 117–125.

Ruokamo, H., Hakkarainen, P., & Eriksson, M. 2011. Designing a Model for Enhanced Teaching and Meaningful E-Learning. In A. D. Olofsson & J. O. Lindberg (Eds.), Informed Design of Educational Technologies in Higher Education: Enhanced Learning and Teaching (pp. 375–392). IGI Global.

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55.

Säljö, R. (2009). Learning, theories of learning, and units of analysis in research. Educational Psychologist, 44(3), 202–208.

Vygotski, L. (1978). Mind in society: the development of higher psychological processes. Cambridge: Harvard University Press.

Acknowledgements

Tekes (The Finnish Funding Agency for Innovations) and ERDF (European Regional Development Fund), as well as a number of public and private financiers, have provided the financing for this research as part of the Medi Peda III, MediPro, and CRICS projects. The authors would like to thank the CISL (Center for Immersive and Simulation-Based Learning) group for their helpful comments and guidance, as well as the teachers and students of Stanford University for their participation in this research. This study was also part of the research activities of the CICERO Learning Network, Finland (www.cicero.fi).

Appendix 1. Results of principal component analysis (PCA) for pre-questionnaire

Appendix 2.Results of principal component analysis (PCA) for post-questionnaire