Teachers as users of ICT from the student perspective in higher education flipped classroom classes

Teachers as users of ICT from the student perspective in higher education flipped classroom classes

The role of information and communication technology (ICT) is highly emphasized in today’s higher education. ICT skills are seen as a central element in the 21st century skill set that students are expected to master (Voog & Roblin, 2012). The development of different technologies is fast. Annual Horizon reports (Horizon reports, 2017) (see Table 1) show several emerging technologies with different pedagogical approaches (e.g. Adams Becker et al., 2017). Current trends focus on areas such as bring your own device (BYOD), flipped classroom (FC), robotics and artificial intelligence.

The rapid development of technologies offers new possibilities and challenges for teaching and learning in institutions of higher education. One of these challenges is teachers’ competence in using developing technologies. The aim of this paper is to investigate how university students perceive their teachers as users of technology and how they perceive a technologically supported learning environment such as the flipped classroom.

Previous studies have identified several factors that affect the integration of ICT in teaching and learning. Brinkerhoff (2006) outlines factors such as administrative and institutional support, the availability of resources, training and experience and attitudes to change. Ertmer (1999; 2005) defines so-called first- and second-order barriers that affect ICT integration. First-order barriers are factors such as support, time resources, training and technical skills. Second-order barriers are more personal factors such as pedagogical beliefs, i.e. how teachers perceive the nature of learning. Harris (2005) notes a demand for a shared vision for ICT integration in teaching. According to the widely adopted Technological Pedagogical Content Knowledge (TPACK) model developed by Mishra and Koehler (2006), the skilled use of ICT in education requires that teachers integrate their Pedagogical Knowledge (PK), Content Knowledge (CK) and Technological Knowledge (TK) in a way that these three areas support each other. According to Koehler et al., (2013, p. 16) TPACK is: “an understanding that emerges from interactions among content, pedagogy, and technology knowledge […] knowledge underlying truly meaningful and deeply skilled teaching with technology.”

Based on the Theory of Planned Behavior by Ajzen (1991), behavioral intentions, in this case the intention to use ICT in education, are affected by attitudes to the behavior, subjective norms (i.e. expectations of important others) and perceived behavioral control (i.e. factors such as the availability of necessary resources and skills). Teo and Tan (2012) have found that attitudes are the most important factor affecting the use of ICT in education. These studies indicate that there are many factors affecting the use of ICT in education, factors that depend on the university and factors that depend on teachers. One important aspect is the extent of the change, i.e. how large change is pursued to integrate ICT to teaching. According to Zhao, Pugh, Sheldon and Bryers (2002), instead of taking a revolutionary approach (very large) in change with ICT, teachers should take evolutionary approach (small) in change with ICT. More precisely, teachers are more likely to experience less frustration and more success with progressive steps that are smaller (Zhao et al., 2002)

One possible way to foster change is to implement the flipped classroom (FC) teaching method. This study focuses on the results of a large-scale project in higher education, focusing on changing pedagogical practices and integrating ICT through implementing FC. FC has emerged as an innovative solution for emphasizing student-centered learning (Baepler et al., 2014; Chen et al., 2017; Chen et al., 2014; Lage et al., 2000). Compared to lecture-based teaching, emphasizing frontal lectures with students taking notes and studying post-class materials alone, FC inverts this design. Lectures are replaced with materials, typically online videos, that are distributed and studied before class meetings through digital platforms. The face-to-face time in the class meeting is used to practice the content together and to concentrate on complex issues through student-centered learning approaches (Chen et al., 2014; Lage et al., 2000).

The FC model is well justified considering factors that affect ICT integration and change in general. When considering traditional lectures from the perspective of the TPACK framework, a strong emphasis is on content knowledge. The teacher disseminates his or her expertise to the students, and only a limited amount of interaction takes place. In the FC model, lectures are replaced with videos that the students watch prior to class. This replacement of the lecture challenges the pedagogical knowledge of the teacher. There is no reason to lecture anymore, and the teacher needs to rethink how to scaffold the lesson and support students working with challenging topics.

From the perspective of ICT in education, we value this emphasis on pedagogy. This is in line with Watson’s (2001) “Pedagogy Before Technology”, which suggests the importance of starting with the design of pedagogical practices that can then be supported with technology. Similarly, Ertmer (2005) refers to the importance of pedagogical beliefs in ICT integration. In accordance with Watson (2001) and Ertmer (2005), we assume that the greatest added value that technology can provide to teaching and learning is in practices that emphasize student collaboration and taking an active role. For this reason, we consider FC to be a way to develop teaching and learning practices toward more student-centered ones and a suitable way to support ICT integration. When considering the extent of the change (Zhao et al., 2002) we find the FC model optimal. The FC model incorporates the role of the teacher as a content expert, i.e. the teacher defines and creates the pre-materials, in this case video clips for students to be used before class meetings. From this perspective, the change is small, however, the direction is toward a more student-centered teaching approach

As to the other factors listed above, concerning the success of ICT integration and change in general, this project is based on a shared vision (see Harris, 2005) among the teachers and trainers of the model. More precisely, FC model was agreed upon as a way to develop teaching and learning and to support ICT integration. Moreover, in line with factors suggested by Brinkerhoff (2006), the process has strong administrative support within our institution. The participating teachers were provided with resources and time for developing their courses as well as training and collaborative support. Since fall 2016, over 100 teachers have participated voluntarily in re-designing their courses. This high number is indicative of a generally positive attitude to the development. This group of teachers was supported by researchers and experts in ICT and pedagogy, who provide the teachers with help and support when needed.

The technological support aspects of the FC training are presented in Figure 1+(Sointu &Valtonen, 2015). First, videos are recorded with the screen capture option on teachers’ own devices. Screen capturing allows teachers to use a variety of programs and software. Second, the videos are transferred to a media server that allowed for simple video editing. Third, the media server allows easy uploading, embedding and sharing of the videos for the students as part of the digital learning environments or web pages.

The teachers participated in a one-year program to adapt their teaching and to flip one of their courses. Teachers were supported in the use of ICT as suggested by (Sointu & Valtonen, 2015). The support was in the form of tutorial videos on how to use ICT in pedagogically meaningful ways, which allowed teachers to learn in their own time. Practical hands-on support for the use of ICT was provided as needed. However, the technical perspective was not the only support for teachers. We recognized the importance to integrate pedagogy, content and technology as suggested in the TPACK framework. Therefore, four other kinds of support were put in place. First, practical hands-on support for flipping the course was provided individually and in large groups. One expert met with all teachers to start the process of change. Second, workshops were arranged, and the teachers had the opportunity to learn about the pedagogical aspects of FC and discuss and share ideas collaboratively. Thus, support was also provided for planning pedagogical practices for face-to-face periods. Third, self-study materials were provided for the teachers. Finally, teachers were allowed to take time to flip their course, and this development was supported institutionally.

According to Shih and Chuang (2013), it is important to understand student perceptions of their teachers’ TPACK when developing teaching and learning practices. In line with this, the purpose of this study was to investigate the effects of the FC integration from two perspectives. (1) How the FC courses affected the ways the students assessed teachers’ content knowledge (CK), combined content and pedagogical knowledge (PCK), combined content, pedagogical and technological knowledge (TPACK) (Koehler, Mishra & Cain, 2013), attitudes to use ICT in teaching (Teo see Tan, 2012) and (2) how students perceived FC and the type of future preferences they had regarding FC.

Methods

Participants and Procedures

The sample included 317 higher education students (N _female >= 227, 71.6%; N _male = 90, 28.6%) from 16 different courses from University of Eastern Finland (UEF) during the academic year 2016-2017. Students participated in courses taught by teachers who had participated in the UEF Flipped Classroom (FC) development work. The courses took place during one period of the academic year (i.e. they lasted for around eight weeks maximum). From the student convenience sample, 43.8% (N = 139) were first-year students. Participation in the study was voluntary for both teachers and students. Informed consent was obtained from each individual student. The response rate in the different courses varied from 25 % to 75 % of the enrolled students. The research design was a quasi-experimental pre-post-test design without a control group (Sadish, Cook, Campbell 2002). Students responded to an electronic survey at the start of the course (pre-test, T1) and at the end of the course (post-test, T2).

Measures

Two surveys were used: (1) Technological Pedagogical Content Knowledge for 21^st century skills (TPACK-21; Valtonen et al, 2015a, 2017) and (2) Theory of Planned Behavior for Information and Communication Technology for education (TPB-ICT; Sointu et al., 2017; Valtonen et al., 2015b). Both measures originally measure teachers’ perceptions of their TPACK-21 and TPB-ICT in education and have demonstrated adequate psychometric properties. However, the measures were adapted to reflect student perceptions of their teachers’ skills and attitudes. To this end we used only parts of the above-mentioned instruments.

From the TPACK-21, we used the Content Knowledge (CK) scale, the Pedagogical Content Knowledge (PCK) scale and the Technological Pedagogical Content Knowledge (TPACK) scale. CK reflects student perceptions of teachers’ content expertise. PCK reflects student perceptions of teachers’ skills in teaching their content area. TPACK reflects student perceptions of teachers’ skills in integrating technology, pedagogy and content in teaching. From the TPB-ICT measure, attitudes for using technology in teaching (ATT) scale was used. ATT reflects student perceptions of teachers’ attitudes to using technology in teaching. Students responded with six-point Likert-type scale (1 = Totally disagree – 6 = Totally agree) to statements such as My teachers are experts in their field (CK), My teacher uses versatile teaching methods in order make study content easily comprehensible (PCK), My teachers are able to use teaching methods and technology for dealing with the content area (TPACK) and, My teachers demonstrate interest in using technology in their teaching (ATT).

We also investigated student perceptions of the flipped classroom using three direct statements. Students responded with four categories (1 = very well, 2 = well, 3 = not so well, 4 = not at all) to the statement The method with pre-materials and contact meetings suited me. Students responded with four response categories (1 = definitely yes, 2 = yes, quite surely, 3 = not so surely, 4 = no way) to the second statement In the future, I would like to study in similarly arranged courses. Students responded with “yes” or “no” to the third statement I would prefer to study in traditional lectures rather than in a course using this type of approach (flipped classroom).

Data Analysis

The items of TPACK-21 and TPB-ICT measures were combined into composite scores. Each composite score’s internal consistency was assessed with Cronbach’s alpha (α) for both pre-test (T1) and post-test (T2) measurement points. We considered an alpha level of 0.7 or higher to indicate the adequate reliability of a composite scale (Nunnally 1978). Descriptive statistics (Mean [M] and Standard deviation [SD]) were calculated for each of the composite scores. Additionally, percentages of the perceptions of flipped classroom teaching were calculated for the three individual items. Paired sample t-tests were used to compare the differences between the two measurement points to investigate the differences of the studied area between T1-T2 measurement points. We also calculated Pearson correlations between the T1-T2 measurement points for composite scores. In addition, we investigated the differences between two groups: (1) first-year students and (2) second-year students and beyond. Cohen’s d was calculated estimating the effect size between the two measurement points (Morris & DeShon, 2008). Based on Hattie’s (2009) interpretation of Cohen’s d in educational settings, d < 0.1 indicates developmental effects, d = 0.2–0.3 indicates teacher effects and d > 0.4 indicates a zone of desired effects (see also Lenhard & Lenhard, 2016).

Results

Table 1 presents means (M), standard deviations (SD), internal consistency (α), Pearson correlations (r), paired t-test results (t) and Cohen’s-d (d) for student perceptions of teachers’ Content Knowledge (CK), Pedagogical Content Knowledge (PCK), Technological Pedagogical Content Knowledge (TPACK), and attitudes to using technology in teaching (ATT). All composite scores met the criteria for adequate internal consistency.

Significant differences between pre-test (T1) and post-test (T2) measurement points were found in the paired sample t-tests for PCK: t(316) = 3.28, p = .00, TPACK: t(316) = 2.77, p = .01, and ATT: t(316) = 3.01, p = .00. Students assessed their teachers PCK, TPACK and ATT higher at the end of the course than at the beginning of the course. The effect size between the two measurements points (i.e. taking the correlation into account) was developmental in TPACK and ATT (d = 0.18), and in the teacher’s effect zone in PCK (d = 0.21) according to Hattie’s (2007) Cohen’s d criteria. There were non-significant differences in the CK scale. However, the CK scale scores were higher (M _T1 = 4.89 [SD _T1 = 0.71]; M _T2 = 4.93 [SD _T2 = 0.84])than the other areas (PCK, TPACK, ATT), which indicates that students perceive their teachers as content specialists.

Table 2 presents the same indicators as Table 1, however separately for first-year students and second-year and higher students. The first-year student perceptions of their teachers’ CK, PCK, TPACK or ATT indicate that they did not perceive differences in their teachers’ knowledge in the measured areas. However, second-year and higher students showed statistically significant higher scores in all measures at T2 compared to T1. The paired sample t-tests showed significant differences between T1 and T2 measurement points in CK: t(176) = 3.01 p = .01, PCK: t(176) = 4.15, p = .00, TPACK: t(316) = 3.23, p = .01, and ATT: t(176) = 4.70, p = .00. According to Hattie’s (2007) Cohen’s d criteria, the effect sizes between two measurements points were in the teachers’ effect zone for CK, PCK, TPACK and ATT.

Figure 2 presents how the students perceived the FC method. 88.1% of the students considered the FC method with pre-materials and contact meetings as well-suited to them. 78.5% of the students indicated that they would like to participate in similarly arranged courses in the future. Almost two-thirds of the students (70.1%) stated that they would prefer to study using the FC method rather than in more traditional lectures.

Discussion

The purpose of this study was to investigate how students in higher education flipped classroom (FC) courses assess teacher content knowledge (CK), pedagogical content knowledge (PCK) and technological pedagogical content knowledge (TPACK) and attitudes to ICT in teaching (ATT), before and after the course. This comparison was done for the whole data set and for two groups: (1) first-year students and (2) second-year and higher students. We also investigated how students perceived FC method and what preferences for the future they had regarding the FC method. A comparison of the data as one group showed that the students assessed their teachers’ PCK, TPACK and ATT higher after the FC course. No difference was found in CK. After splitting the data into two groups, the results for the first-year students did not show statistically significant differences for CK, PCK, TPACK or ATT at T1 and T2. However, second-year and higher students assessed their teachers’ skill higher in T2 for all measured areas. The majority of the students perceived FC in a positive way and were willing to take FC-type courses in the future. The effect sizes were at a moderate level in general, however, they should be compared in the context in which the courses lasted for approximately one period of academic year. Results can be considered reasonable from this perspective.

These results indicate that the FC approach can change student perceptions of teachers’ TPACK and teachers’ attitudes to using technology. The three steps for ICT integration to teaching in the project and the individual support that was offered seem to improve the teachers’ TPACK-related skills. We assume that the FC model provided opportunities for the meaningful use of ICT to support learning. As suggested by Watson (2001) and Ertmer (2005), pedagogy and pedagogical beliefs are important factors for the successful integration of ICT in teaching. The FC model emphasizes the teacher as a content expert, as an expert in pedagogy and as an expert in the use of ICT in education. Results of this study suggest that, from a student perspective, teacher skills in integrating content, pedagogy and technology, are increasing.

In order to make changes successful, the change must be reasonable (see. Zhao et al., 2002). One of the reasons the results of this study indicate such successful integration of pedagogy and technology is the nature of the FC model, which is optimal for initiating change. In other words, FC is an easy-to-grasp model that allows teachers to consider pedagogy and ICT in their teaching more thoroughly. We assume that emphasizing collaborative and student-centered methods in teaching makes the expertise in pedagogy and in use of ICT more visible for students, thereby approaching the definition of skilled teaching with technology by Koehler and colleagues (2013). The role of the content specialist remains in pre-materials and contact meetings. Through more student-centered approaches, teachers can use time more effectively. Results show that students recognize the benefits of the FC model and the change in teaching skills, which indicates that the teachers developed professionally during the process.

The results indicate that the second-year and higher students perceived significant change in all CK, PCK, TPACK and ATT areas, whereas the first-year students did not. Higher year students may have a better understanding of teaching practices, technology use, teacher skill levels and teacher attitudes to the use of ICT in education. First-year students may not have this experience. This result can be considered logical and, thus, increases the reliability of this study.

These findings align well with the aspects presented in earlier studies for the successful integration of ICT (Brikerhoff, 2006; Ertmer 1999; 2005) and with the vision of how teacher training should be organized. Sufficient support for pedagogy, the use of ICT, a peer-support culture and self-study materials combined with time for developing teaching practices are important in developing the use of ICT in teaching. There must be a shared vision for ICT integration (Harris, 2005), and FC creates a framework for teachers participating the training and a framework for support personnel who train teachers. FC seems to influence teacher attitudes to using ICT in teaching, which is according to Teo and Tan (2012) one of the most important factor influencing the use of ICT in education. The FC project with support and training allows teachers to develop their skills systematically. Thus, it seems that support for teachers with the flipped classroom framework is effective for professional development and to improve their content related pedagogical-technological skills in higher education from the student perspective.

Limitations and future research

The present study has limitations. First, a convenience sample was used. Future researchers should consider stronger sampling methods in order to verify the results of the study. Second, only two aspects of ICT integration were addressed, namely the TPACK model and the Theory of Planned Behavior model for ICT in education. Future research should consider using other theoretical frameworks and measurement instruments based on these theories. Third, study did not include an in-depth investigation of the psychometric properties of the adapted instruments. Future research should be extended to a more in-depth investigation of reliability and validity. Fourth, typical analysis methods were used, but future research would benefit from using more sophisticated methods, such as structural equation modeling. Finally, the pedagogical investigations were based on the whole FC model. Future research should more specifically consider pedagogical approaches within FC.

Acknowledgements

All teachers and students who participated this study are gratefully acknowledged. This study was supported by the grant from Finnish Ministry of Education and Culture and Finnish [OKM/199/523/2016], and Suomen Akatemia/Academy of Finland [Grant 296799 and 273970].