Improving Learning Design Using Learning Analytics in Relation to Study Experience
DOI:
https://doi.org/10.7577/seminar.5722Keywords:
Learning analytics, learning design, study experience, higher education, student-generated dataAbstract
In higher education, the learner-centered approach faces challenges due to the growing number and diversity of students and the increasing complexity of course delivery. Study experiences, which correlate with academic achievement, may be enhanced through learning analytics. In particular, analytics can offer valuable, timely insights by collecting and analyzing data on the study experiences and student-related metrics. This allows teachers to gain understanding of students' psychological qualities, which are not typically inferred from standard learning management systems.
Our case study aims to demonstrate how the practical application of learning analytics (LA)-generated data on students’ psychological qualities can guide teachers in enhancing their instructional delivery and, consequently, enhance student experiences. Initially, we assess the reliability of data concerning students’ psychological traits and study experiences. Subsequently, we explore whether these data can provide insights for teachers that can lead to improved student experiences. Student experiences across two consecutive course implementations are compared to illustrate the potential of LA in informing teachers.
The results show that data can be collected reliably on students' daily academic activities and emotional states during the teamwork week. Preliminary findings from the spring term are shared with teachers, which indicate that the use of LA data can positively influence student experiences without requiring structural changes to instructional materials or course implementation. Although the study is not experimental, it provides valuable insights into specific methods of applying LA to inform teachers and enhance student experiences. Further research is needed to deepen the understanding of these applications.
References
Ahmad, A., Schneider, J., Griffiths, D., Biedermann, D., Schiffner, D., Greller, W., & Drachsler, H. (2022). Connecting the dots – A literature review on learning analytics indicators from a learning design perspective. Journal of Computer Assisted Learning, n/a(n/a). https://doi.org/10.1111/jcal.12716
Aldowah, H., Al-Samarraie, H., Alzahrani, A. I., & Alalwan, N. (2020). Factors affecting student dropout in MOOCs: A cause and effect decision‐making model. Journal of Computing in Higher Education, 32(2), 429–454. https://doi.org/10.1007/s12528-019-09241-y
Axelsen, M., Redmond, P., Heinrich, E., & Henderson, M. (2020). The evolving field of learning analytics research in higher education: From data analysis to theory generation, an agenda for future research. Australasian Journal of Educational Technology, 36(2), 1–7. https://doi.org/10.14742/ajet.6266
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman and Company.
Blumenstein, M. (2020). Synergies of Learning Analytics and Learning Design: A Systematic Review of Student Outcomes. Journal of Learning Analytics, 7(3), 13–32. https://doi.org/10.18608/jla.2020.73.3
Camacho-Morles, J., Slemp, G. R., Pekrun, R., Loderer, K., Hou, H., & Oades, L. G. (2021). Activity achievement emotions and academic performance: A meta-analysis. Educational Psychology Review, 33(3), 1051–1095. https://doi.org/10.1007/s10648-020-09585-3
Castro, R. (2019). Blended learning in higher education: Trends and capabilities. Education and Information Technologies, 24(4), 2523–2546. https://doi.org/10.1007/s10639-019-09886-3
Conole, G., & Fill, K. (2005). A learning design toolkit to create pedagogically effective learning activities. Journal of Interactive Media in Education, 1, Article 1. http://www-jime.open.ac.uk/2005/01/
Coutinho, S. A., & Neuman, G. (2008). A model of metacognition, achievement goal orientation, learning style and self-efficacy. Learning Environments Research, 11(2), 131–151. https://doi.org/10.1007/s10984-008-9042-7
De Backer, L., Van Keer, H., & Valcke, M. (2012). Exploring the potential impact of reciprocal peer tutoring on higher education students’ metacognitive knowledge and regulation. Instructional Science, 40(3), 559–588. https://doi.org/10.1007/s11251-011-9190-5
Dweck, C. S., & Master, A. (2009). Self-theories and motivation: Students’ beliefs about intelligence. In Handbook of motivation at school (pp. 123–140). Routledge/Taylor & Francis Group. https://doi.org/10.4324/9780203879498
Elliott, K. M., & Shin, D. (2002). Student Satisfaction: An alternative approach to assessing this important concept. Journal of Higher Education Policy and Management, 24(2), 197–209. https://doi.org/10.1080/1360080022000013518
Francis, P., Broughan, C., Foster, C., & Wilson, C. (2019). Thinking critically about learning analytics, student outcomes, and equity of attainment. Assessment & Evaluation in Higher Education, 45(6), 811–821. https://doi.org/10.1080/02602938.2019.1691975
Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. 56–65. https://doi.org/10.1145/3027385.3027396
Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x
Goh, C., Leong, C., Kasmin, K., Hii, P., & Tan, O. (2017). Students’ Experiences, Learning Outcomes and Satisfaction in e-Learning. Journal of E-Learning and Knowledge Society, 13(2). https://www.learntechlib.org/p/188116
Ifenthaler, D. (2017). Designing effective digital learning environments: Toward learning analytics design. Technology, Knowledge and Learning, 22, 401-404. https://doi.org/10.1007/s10758-017-9333-0
Haya, P. A., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015). Analysing content and patterns of interaction for improving the learning design of networked learning environments. British Journal of Educational Technology, 46(2), 300–316. https://doi.org/10.1111/bjet.12264
Heilala, V., Jääskelä, P., Kärkkäinen, T., & Saarela, M. (2020a) Understanding the Study Experiences of Students in Low Agency Profile: Towards a Smart Education Approach. In A. El Moussati, K. Kpalma, M. Ghaouth Belkasmi, M. Saber, & S. Guégan (Eds.), Advances in Smart Technologies Applications and Case Studies (Vol. 684, pp. 498–508). Springer International Publishing. https://doi.org/10.1007/978-3-030-53187-4_54
Heilala, V., Kelly, R., Saarela, M., Jääskelä, P., & Kärkkäinen, T. (2023). The Finnish Version of the Affinity for Technology Interaction (ATI) Scale: Psychometric Properties and an Examination of Gender Differences. International Journal of Human–Computer Interaction, 39(4), 874–892. https://doi.org/10.1080/10447318.2022.2049142
Heilala, V., Saarela, M., Jääskelä, P., & Karkkainen, T. (2020b) Course Satisfaction in Engineering Education Through the Lens of Student Agency Analytics. 2020 IEEE Frontiers in Education Conference (FIE), 1–9. https://doi.org/10.1109/FIE44824.2020.9274141
Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Pozzi, F., Chacón, J., Prieto, L. P., & Persico, D. (2018). An Integrated Environment for Learning Design. Frontiers in ICT, 5, 9. https://doi.org/10.3389/fict.2018.00009
Hirvonen, R., Tolvanen, A., Aunola, K., & Nurmi, J.-E. (2012). The developmental dynamics of task-avoidant behavior and math performance in kindergarten and elementary school. Learning and Individual Differences, 22(6), 715–723. https://doi.org/10.1016/j.lindif.2012.05.014
Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2). https://doi.org/10.14742/ajet.3767
Ifenthaler, D., & Yau, J. Y.-K. (2020). Utilising learning analytics to support study success in higher education: A systematic review. Educational Technology Research and Development, 68(4), 1961–1990. https://doi.org/10.1007/s11423-020-09788-z
Jayashanka, R., Hewagamage, K. P., & Hettiarachchi, E. (2019). An Intelligent Interactive Visualizer to Improve Blended Learning in Higher Education. 2019 Twelfth International Conference on Ubi-Media Computing (Ubi-Media), 69–73. https://doi.org/10.1109/Ubi-Media.2019.00022
Jääskelä, P., Heilala, V., Kärkkäinen, T., & Häkkinen, P. (2021). Student agency analytics: Learning analytics as a tool for analysing student agency in higher education. Behaviour & Information Technology, 40(8), 790–808. https://doi.org/10.1080/0144929X.2020.1725130
Karaoglan Yilmaz, F. G. (2022). The effect of learning analytics assisted recommendations and guidance feedback on students’ metacognitive awareness and academic achievements. Journal of Computing in Higher Education, 34(2), 396–415. https://doi.org/10.1007/s12528-021-09304-z
Kew, S. N. (2022). Learning Analytics in Online Learning Environment: A Systematic Review on the Focuses and the Types of Student-Related Analytics Data. Technology, Knowledge and Learning, 27(2), 405–427. https://doi.org/10.1007/s10758-021-09541-2
Kew, S. N., & Tasir, Z. (2022). Learning Analytics in Online Learning Environment: A Systematic Review on the Focuses and the Types of Student-Related Analytics Data. Technology, Knowledge and Learning, 27(2), Article 2. https://doi.org/10.1007/s10758-021-09541-2
Koenka, A. C., & Anderman, E. M. (2019). Personalized feedback as a strategy for improving motivation and performance among middle school students. Middle School Journal, 50(5), 15-22. https://doi.org/10.1080/00940771.2019.1674768
Kiuru, N., Aunola, K., Vuori, J., & Nurmi, J.-E. (2007). The Role of Peer Groups in Adolescents’ Educational Expectations and Adjustment. Journal of Youth and Adolescence, 36(8), 995–1009. https://doi.org/10.1007/s10964-006-9118-6
Kryshko, O., Fleischer, J., Grunschel, C., & Leutner, D. (2022). Self-efficacy for motivational regulation and satisfaction with academic studies in STEM undergraduates: The mediating role of study motivation. Learning and Individual Differences, 93, 102096. https://doi.org/10.1016/j.lindif.2021.102096
Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge.
Lim, L.-A., Gentili, S., Pardo, A., Kovanović, V., Whitelock-Wainwright, A., Gašević, D., & Dawson, S. (2021). What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction, 72(Journal Article), 101202. https://doi.org/10.1016/j.learninstruc.2019.04.003
Lyytinen, A., Liljeroos, J., Pekkola, E., Kosonen, J., Mykkänen, M., & Kivistö, J. (2019). Ammatillinen opettajankoulutus Suomessa: reunaehdot, rakenteet ja profiilit.
Mangaroska, K., & Giannakos, M. (2019). Learning Analytics for Learning Design: A Systematic Literature Review of Analytics-Driven Design to Enhance Learning. IEEE Transactions on Learning Technologies, 12(4), Article 4. https://doi.org/10.1109/TLT.2018.2868673
Moore, P. J. (2019). Academic achievement. Educational Psychology, 39(8), 981–983. https://doi.org/10.1080/01443410.2019.1643971
Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229. https://doi.org/10.1111/bjet.12273
Neelen, M., & Kirschner, P. A. (2020). Evidence-informed Learning Design: Creating Training to Improve Performance. KoganPage. https://books.google.fi/books?id=fyutxAEACAAJ
Nunn, S., Avella, J. T., Kanai, T., & Kebritchi, M. (2016). Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. Online Learning, 20(2). https://doi.org/10.24059/olj.v20i2.790
Nurmi, J.-E., Haavisto, T., & Salmela-Aro, K. (1997). CAST-käsikirja.
Oliva-Cordova, L. M. (2021). Learning Analytics to Support Teaching Skills: A Systematic Literature Review. IEEE Access, 9, 58351–58363. https://doi.org/10.1109/ACCESS.2021.3070294
Pan, Z., Biegley, L., Taylor, A., & Zheng, H. (2024). A Systematic Review of Learning Analytics: Incorporated Instructional Interventions on Learning Management Systems. Journal of Learning Analytics, 11(2), Article 2. https://doi.org/10.18608/jla.2023.8093
Papinczak, T., Young, L., Groves, M., & Haynes, M. (2008). Effects of a Metacognitive Intervention on Students’ Approaches to Learning and Self-Efficacy in a First Year Medical Course. Advances in Health Sciences Education, 13(2), 213–232. https://doi.org/10.1007/s10459-006-9036-0
Parpala, A., & Lindblom-Ylänne, S. (2012). Using a research instrument for developing quality at the university. Quality in Higher Education, 18(3), 313–328. https://doi.org/10.1080/13538322.2012.733493
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic Emotions in Students’ Self-Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research. Educational Psychologist, 37(2), 91–105. https://doi.org/10.1207/S15326985EP3702_4
Pintrich, P. R. (2003). A Motivational Science Perspective on the Role of Student Motivation in Learning and Teaching Contexts. Journal of Educational Psychology, 95(4), 667–686. https://doi.org/10.1037/0022-0663.95.4.667
Poon, J. (2012). Use of blended learning to enhance the student learning experience and engagement in property education. Property Management, 30(2), 129–156. https://doi.org/10.1108/02637471211213398
Prat‐Sala, M., & Redford, Paul. (2010). The interplay between motivation, self‐efficacy, and approaches to studying. British Journal of Educational Psychology, 80(2), 283–305. https://doi.org/10.1348/000709909X480563
Räisänen, M., Postareff, L., Mattsson, M., & Lindblom-Ylänne, S. (2020). Study-related exhaustion: First-year students’ use of self-regulation of learning and peer learning and perceived value of peer support. Active Learning in Higher Education, 21(3), 173–188. https://doi.org/10.1177/1469787418798517
Lyytinen, A., Liljeroos, J., Pekkola, E., Kosonen, J., Mykkänen, M., & Kivistö, J. (2019). Ammatillinen opettajankoulutus Suomessa: reunaehdot, rakenteet ja profiilit.Salmela-Aro, K. (with Näätänen, P.). (2005). Nuorten koulu-uupumusmittari BBI-10. Edita.
Saarela, M. (2017). Automatic knowledge discovery from sparse and large-scale educational data: Case Finland. Jyväskylä Studies in Computing, 262. http://urn.fi/URN:ISBN:978-951-39-7084-0
Salmela-Aro, K., Kiuru, N., Leskinen, E., & Nurmi, J.-E. (2009). School Burnout Inventory (SBI): Reliability and Validity. European Journal of Psychological Assessment, 25(1), 48–57. https://doi.org/10.1027/1015-5759.25.1.48
Salmela-Aro, K., & Read, S. (2017). Study engagement and burnout profiles among Finnish higher education students. Burnout Research, 7, 21–28. https://doi.org/10.1016/j.burn.2017.11.001
Salmi, E., Määttä, S., Vehkakoski, T., Aunola, K., Kairaluoma, L., & Pirttimaa, R. (2020). Oppimisvaikeuksien, motivaation ja oppijaminäkäsityksen merkitys ammatillisista opinnoista valmistumisessa – Oppimisen ja oppimisvaikeuksien erityislehti – NMI Bulletin. Oppimisen ja oppimisvaikeuksien erityislehti – NMI bulletin, 30(3), 50–66.
Schraw, G., & Dennison, R. S. (1994). Assessing Metacognitive Awareness. Contemporary Educational Psychology, 19(4), 460–475. https://doi.org/10.1006/ceps.1994.1033
Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 78(Journal Article), 397–407. https://doi.org/10.1016/j.chb.2017.06.030
Siemens, G. (2012). Learning analytics. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 4–8. https://doi.org/10.1145/2330601.2330605
Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE review, 46(5), 30.
Silvola, A., Näykki, P., Kaveri, A., & Muukkonen, H. (2021a). Expectations for Supporting Student Engagement with Learning Analytics: An Academic Path Perspective. Computers & Education, 168, 104192. https://doi.org/10.1016/j.compedu.2021.104192
Silvola, A., Näykki, P., Kaveri, A., & Muukkonen, H. (2021b). Expectations for Supporting Student Engagement with Learning Analytics: An Academic Path Perspective. Computers & Education, 168, 104192. https://doi.org/10.1016/j.compedu.2021.104192
Tempelaar, D. (2019). Supporting the less-adaptive student: The role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45, 1–15. https://doi.org/10.1080/02602938.2019.1677855
Trigwell, K., Ellis, R. A., & Han, F. (2012). Relations between students’ approaches to learning, experienced emotions and outcomes of learning. Studies in Higher Education, 37(7), 811–824. https://doi.org/10.1080/03075079.2010.549220
Tuononen, T., Hyytinen, H., Räisänen, M., Hailikari, T., & Parpala, A. (2023). Metacognitive awareness in relation to university students’ learning profiles. Metacognition and Learning, 18(1), 37–54. https://doi.org/10.1007/s11409-022-09314-x
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/10.1016/j.chb.2018.07.027
Vu, T., Magis-Weinberg, L., Jansen, B. R. J., van Atteveldt, N., Janssen, T. W. P., Lee, N. C., van der Maas, H. L. J., Raijmakers, M. E. J., Sachisthal, M. S. M., & Meeter, M. (2022). Motivation-Achievement Cycles in Learning: A Literature Review and Research Agenda. Educational Psychology Review, 34(1), 39–71. https://doi.org/10.1007/s10648-021-09616-7
Wigfield, A., & Eccles, J. S. (2000). Expectancy–Value Theory of Achievement Motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015
Weimer, M. (2013). Learner-centered teaching: Five key changes to practice. John Wiley & Sons.
Wong, J., Baars, M., de Koning, B. B., van der Zee, T., Davis, D., Khalil, M., Houben, G.-J., & Paas, F. (2019). Educational Theories and Learning Analytics: From Data to Knowledge. In D. Ifenthaler, D.-K. Mah, & J. Y.-K. Yau (Eds.), Utilizing Learning Analytics to Support Study Success (pp. 3–25). Springer International Publishing. https://doi.org/10.1007/978-3-319-64792-0_1
Zhang, J., Zhang, X., Jiang, S., Ordóñez de Pablos, P., & Sun, Y. (2018). Mapping the study of learning analytics in higher education. Behaviour & Information Technology, 37(10–11), 1142–1155. https://doi.org/10.1080/0144929X.2018.1529198
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Satu Aksovaara, Sami Määttä, Tommi Kärkkäinen, Minna Silvennoinen
This work is licensed under a Creative Commons Attribution 4.0 International License.
Seminar.net is a fully open access journal, which means that all articles are available on the internet to all users immediately upon publication. Use and distribution in any medium is permitted, provided the author and the journal are properly credited. The journal allow reuse and remixing of content in accordance with a Creative Commons license CC-BY
- The journal allows the author(s) to hold the copyright without restrictions.
- The journal allows the author(s) to retain publishing rights without restrictions.
- Seminar.net does not charge authors for publishing with us.