A lesson learned: Experiences with the use of various teaching and
learning tools in the course Organizational Behaviour and
Management at the BI Norwegian Business School.
Department of Leadership and Organizational Behaviour
BI Norwegian Business School
Email: cecilie.asting@bi.no
BI LearningLab
BI Norwegian Business School
Email: anne.swanberg@bi.no
Teaching behavioural subjects to business students is a challenge, increasingly so with growing class sizes. In this paper we focus on these special challenges, particularly drawing attention to how feedback can enhance student learning and understanding. One-to-one feedback is not possible in large classes, but students can receive feedback on their progress through well-planned teaching and learning activities. We implemented a range of different feedback activities in our course to support student learning. Measuring learning effects is difficult and, in this case, comparison of grades was not possible. Our experience, however, led to a somewhat better understanding of what can be done and what needs further development to provide valuable feedback for students in their learning process.
Keywords: Teaching, learning, organizational behaviour, feedback.
Teaching large classes presents a variety of challenges. In this paper we focus on how to give and receive feedback when class sizes exceed 100 students. Our main interest is to understand how teachers can work with feedback systematically to improve student learning. Essentially, there are two different perspectives: the teacher giving feedback to the student and the teacher receiving feedback from the student. Both are important to the student and the teacher, in that the student receives guidance in his/her learning process and the teacher understands what problems are most frequent. Together, these feedback exercises can enhance teaching and learning in higher education institutions.
Modern society demands an increasingly competent workforce; as a result, more students are entering higher education. The consequence for many institutions has been larger classes and a body of students more diverse than only a few decades ago. In the past, students seeking higher education were largely motivated by curiosity and knowledge; today, however, a degree may be seen as a necessity for future employment and thus becomes merely an instrument and not a goal in and of itself. It is possible to argue that students who view education as a means and not a goal will take a more strategic approach to learning than students motivated by learning to understand. This diversity in attitude and behaviour impacts on how teachers plan, teach, and give feedback. It is possible to argue that today’s students often need to be motivated to spend enough time on a subject, and the challenge for the teacher is how to stimulate students’ interest. In addition, students may need more extended and differentiated follow up and more assessment to guide them.
The students at BI Norwegian Business School have already made a clear choice when they apply. Their interest is in business and business-related subjects and many are attracted to quantitative courses. The course Organizational Behaviour and Management may therefore surprise many of our students, as initially they may not be interested in human behaviour and relations. The course is taught in the second semester of a bachelor’s degree, and scheduling the course at this early stage in a student’s major field could add to the difficulties in teaching the course. Our experience from teaching these topics for a number of years, reading student evaluations, and discussions with students and other teachers leads us to propose several reasons why this might be the case:
This list is not exhaustive but includes enough information about the challenges facing the teacher in this particular course for our question:
How can we help junior business students understand the importance of learning organizational behaviour and management in preparing them for work?
There is a large demand for higher education, creating a substantial and nuanced challenge for institutions at university level. Our interest is in providing quality teaching and learning for students with a wide variety of interests and motivation. Constructive alignment (Biggs & Tang, 2007) is a concept designed to respond to this challenge and provides a framework for teachers to enhance student learning.
Biggs and Tang (2007) define two types of students who have very different reasons for wanting a university degree: Academic Susan and Non-academic Robert. “Susan is academically committed; she is bright, interested in her studies and wants to do well. She has clear academic or career plans and what she learns is important to her” (Biggs & Tang, 2007: 8, 9). Robert, however, is “at university not out of a driving curiosity about a particular subject or a burning ambition to excel in a particular profession, but to obtain a qualification for a decent job” (Biggs & Tang, 2007: 9). The idea behind the theory of constructive alignment is to plan and implement teaching to make Robert’s learning more like Susan’s. By constructing learning outcomes and planning learning activities to achieve the intended learning, Robert can learn and understand more and better. This type of teaching, according to Biggs and Tang (2007), reduces the teaching gap between different types of students’ learning engagement, as students are required to perform learning activities directly linked to learning outcomes. Learning outcomes are what students are expected to know for the exam at the end of a course. During the course students attend classes and perform activities to receive formative feedback to support their learning. Formative feedback provides the student with information on how s/he is performing according to learning outcomes. The documentation of the student’s learning outcomes is the summative evaluation or the grade the student receives on the final exam. The principle is visualized in the model below (based on Biggs & Tang, 2007).
Taras (2002) claims that higher education is too focused on grades at the expense of the learning process. Formative evaluation is a helpful pedagogical tool to enhance student learning. Ramsden (2003) expresses the same view and argues that formative evaluation is also helpful for the teacher as it allows the teacher to detect which part(s) of the curriculum may create the most misunderstandings and mistakes. Brookhart (2001) points out one important relationship between learning outcomes and formative evaluation when he argues that for formative evaluation to be effective students need to be familiar with learning outcomes. Another condition for formative evaluation to succeed is that students are able to compare their own work with work of the desired quality. Furthermore, they need knowledge about how to work to attain learning outcomes. Feedback is of no use if the recipient is unable to use it to improve his/her own work; that is, students need to be taught how to use feedback for improvement.
Biggs and Tang (2007) argue that assessment in large classes can be solved by using self and peer assessment. The advantages are
The importance of teaching students why they are asked to perform the feedback task, and how, is underlined.
Biggs and Tang (2007) describe and use the SOLO (Structure of the Observed Learning Outcome) taxonomy, a method based on study outcomes in several academic areas. Course objectives given at BI Norwegian Business School are based on Bloom’s taxonomy, which McKeachie (2002) describes as “a popular framework to improve an instructor’s ability to teach thinking, regardless of the discipline. Bloom’s taxonomy became a foundation for teaching across all levels” (McKeachie 2002:285). Biggs and Tang (2007) point out the difference between the two taxonomies by referring to Bloom’s as less hierarchical and less based on student learning than on how educational administrators judge the learning. A list of verbs used for formulating learning outcomes has been developed for both taxonomies, making constructive alignment a practical method in terms of both taxonomies.
One well-known feedback exercise is the One Minute Paper (OMP). OMP “may be defined as a very short, in-class writing activity (taking one minute or less to complete) in response to an instructor-posed question, which prompts students to reflect on the day’s lesson and provides the instructor with useful feedback”[i]. According to Chizmar and Ostrosky (1998), OMP is a simple, low-tech innovation tool to obtain feedback from students. It is widely used, and college teachers see the OMP as a tool that can easily improve their teaching. In large classes feedback is often seen as a challenge, and one reason is that many students feel embarrassed to ask questions when something is unclear or not fully understood. The OMP may ask the students to answer what they think are the most important and the most difficult aspects to understand, giving the teacher important feedback about how to improve or clarify problem areas.
In 2009, BI Norwegian Business School implemented a new structure for its study programme, the so-called “Bachelor Reform.” This new structure was a direct result of the Norwegian Quality Reform in Higher Education (the Norwegian follow-up of the Bologna Process)[ii]. The course Organizational Behaviour and Management was taught for the first time as a second semester course in spring 2010. The course was a reinstitution of an old course but with a revised and extended curriculum, textbooks in newer editions, and a total of 7.5 ECTS for an exam pass. This was also made the only mandatory course in its discipline for the Norwegian Business School’s Bachelor programmes. Interested students could, however, choose from a variety of electives within the organizational behaviour field in their second and third year.
In spring semester 2010, approximately 4000 students signed up for the course; their goal was to pass the exam on May 19. At the Oslo campus, the course was taught in seven parallel classes for almost 2000 students. To inspire students to work throughout the semester and thereby enhance interest and learning, the students were given the potential essay questions for the exam. The course was designed to have 14 lectures, each covering a particular topic. Each week one exam question was published online for the purpose of preparing students. The teacher gave a short briefing in the upcoming lecture. This provided a framework for the student in helping them to determine the focal points and structure of their answers. Only students who attended class received this information.
Organizational Behaviour and Management was a “double” pilot this particular semester. BI Norwegian Business School was introducing a new learning management system (LMS), which includes a range of different options on how to communicate with students. To gain some experience with the system, Organizational Behaviour and Management, as a large-scale course, was chosen as a pilot. This meant we had access to new technology for publishing, communication, and feedback that would help to enhance teaching and learning.
The course Organizational Behaviour and Management, giving 7.5 ECTS, is estimated to require 200 hours work for the students, divided into the following activities and quantities.
|
Activity |
Hours |
|
Time spent in class |
42 |
|
Reading set books |
84 |
|
Task solution |
42 |
|
Study groups and additional readings |
27 |
|
Exam |
5 |
|
Total recommended time |
200 |
One result of the Bachelor Reform is that all courses describe learning outcomes at course level. These outcomes are divided into three areas: knowledge, skills and attitude. For Organizational Behaviour and Management, the course learning outcomes are
Knowledge: Students will acquire a knowledge of basic psychological and organizational theory relevant to employment and further studies in organization and management.
Skills: Students will be able to explain key concepts, processes and theories and how these relate to effectiveness in organisations.
Attitude: Students will develop an understanding of the psychological characteristics and processes and organizational conditions that are important for optimal functioning in the workplace.
In addition, a number of feedback activities to enhance students’ learning processes were implemented. The activities are presented in the following table and explained further below:
|
Activity |
Hours |
|
Time spent in class |
42 |
|
Detailed learning outcomes for all topics |
14 |
|
Published exam questions |
14 |
|
Recommended answer to exam question |
14 |
|
Framework for self-peer assessment exam questions |
14 |
|
Students grading previous exams |
4 |
|
Individual decision style test |
1 |
|
Electronic tests |
8 |
|
One Minute Paper |
2 |
However, the course learning outcomes were not detailed enough to be consistent with the constructive alignment theory. Accordingly, using verbs described in Biggs and Tang (2007), intended learning outcomes for all topics were developed according to Bloom’s taxonomy. It was possible to do this quite accurately since all exam questions were already known.
In the weekly lecture the teacher presented a possible answer to the exam question for the topic under discussion, with the aim of helping students learn how to answer this kind of question. The presentation outlined a “role model” for the students. The answer was not published online.
As mentioned, the reason for publishing exam questions was to motivate students to work with Organizational Behaviour and Management throughout the semester and thereby have a better learning outcome. BI Norwegian Business School has a business model that requires teaching in large classes at bachelor level. The framework for self and/or peer assessment of the exam questions was developed to help the students in their learning process. Teacher assessment was not possible in this case because of the large number of students.
To provide training in assessment skills, students were given four graded exam answers from a previous course. Using the assessment framework the students were asked to consider if this was a good or poor performance. The aim of this activity was to initiate a class discussion about the different parts of the exam, pinpointing where the student had or had not attained intended learning outcomes.
The intended learning outcome from the individual decision style test was that students understand that a problem may be approached in different ways by different people. A class discussion following the test gave the students an option for additional feedback.
How should learning be measured? Course grades given upon examination are, in this respect, our only objective measures. According to the constructive alignment theory this is the documentation. Given that the course (Organizational Behaviour and Management) was being taught for the first time, a comparison of results with grades from earlier exams, such as from spring 2009, does not provide accuracy since the two courses were not identical.
We also surveyed students to get feedback and reactions to the implemented method. At the end of the semester students were asked to report their reactions regarding learning outcomes, teaching methods and their own achievement. Unfortunately, few students completed the survey. These facts represent considerable weaknesses in our measurement; thus, our discussion relates only to the most significant feedback, in this case the answers given to one teacher, representing 1200 students. Results in terms of grades are available for all students taking the exam.
Nevertheless, we believe that it is possible to discuss our experience to try to understand where we succeeded and failed in this pilot. The following discussion is therefore based on our subjective experiences and beliefs related to the available results.
According to the constructive alignment theory, all activities in the learning process should support learning outcomes.
Intended learning outcomes, exam questions and suggested solutions to exam questions formed the main activities in the course. The exam questions and learning outcomes were published on the LMS before every lecture, a load-bearing activity introduced to motivate students to work toward goals throughout the semester.
In the lectures, students, by comparing the teacher’s answers to their own, received important feedback on their work/progress. According to Biggs and Tang (2007), the learning process is enhanced by student activity; in this case, this meant being familiar with, and using, the learning outcomes to solve the exam question assignment. Students’ self-reports show that 68% were familiar or partly familiar with the learning outcomes and 53% claim to have worked with all 14 exam questions. On the basis of these figures, we acknowledge that our intention to motivate the students by using the exam question assignment as preparation failed to a certain degree. Taking the percentages alone, it is possible to argue that these results are satisfactory; however, very few students answered and, as this activity was constantly stressed as being the most important, it should have resulted in the students’ believing in its importance as well. Our experience in class, in discussion, and via email with students indicated that most students only wrote the assignment after the teacher had provided a possible solution. It could be that students are generally used to attending class first and working on a topic later, which might suggest that these types of assignments are difficult for young students for a number of different reasons. It is possible to assume that students who had worked with the exam questions did better in the exam than students who did not work on the assignments. Accordingly, we could also argue that the learning effects for students who came prepared to the lecture were better than for students working after a lecture. The challenge of motivating students to work more assiduously with the exam questions is two-fold: first, motivating a larger number of students to complete the assignment, and second, motivating more students to work with the topic before class. In addition, the learning outcomes are not always as coherently expressed as they should be and some revision would make them a more helpful and motivating tool.
Intended learning outcomes. It was not mandatory for teachers to state learning outcomes either on the LMS or during lectures, with the result that these were published either occasionally or not at all. Some students who had heard about, but not were given, learning outcomes during the semester requested them. In some classes the learning outcomes for all 14 questions had been published by the end of the course.
In conclusion, we argue that developing and publishing learning outcomes positively affects teaching in terms of focusing on the most relevant topics for students wishing to pass the exam. It also seems to have a positive effect for students using them as a tool for understanding. However, it also made students even more strategic in their approach and perhaps less curious about the topic being taught. Several teachers reported that many students questioned why they were required to discuss theories in class that were not mentioned in the syllabus, even if this was done to provide better understanding. Another less fortunate result may be a direct focus on the reproduction of theories with less understanding and application. After grades were published, many students who had earned a B asked for a formal explanation of the assessment. It seemed that they assumed that memorizing theory was adequate to earn them an A, an attitude which devalues the importance of theory application and the complexity of the subject.
Framework for self/peer assessment exam questions. This framework was developed for students to give and receive feedback on their work with the 14 exam questions. This tool was initially meant to be a class activity. Students brought their answers to class then exchanged them with the student sitting beside them; both students then provided feedback using the framework for self/peer assessment. This activity is intended to enhance learning via two different perspectives: giving and receiving feedback. Another advantage of using the framework is that it underlines individual differences, an important lesson in Organizational Behaviour and Management. As mentioned, few students had worked with the different topics before class, and as the exercise asked them to expose their own work, it may have been frightening for some of them. As Brookhart (2001) suggests, students need to learn how to give and use feedback, which is a point we did not give adequate thought to. As we are now more aware of this, we have devoted more time to considering how and why to use the framework during the present semester.
Students grading previous exams. This was a popular activity mainly because many students were unsure and gained confidence by being able to read answers from previous exams. Students who used the assessment framework to assign a grade may have learnt a lot. However, they were probably outnumbered by others who did not perform the task but thought they learnt something simply by getting to know the grade.
Decision Style Test. The individual decision style test is translated from an American textbook. The reason for this test is to pinpoint the individual differences that are stressed in this course but that are not always understood by students. The test was taken in class and the results were discussed. The aim was for students to understand the many different ways of approaching a problem and that these different approaches can give a range of outcomes. The goal was to provoke class discussion and underline differences and to demonstrate how discussion works, but not necessarily to provide students with a right or a wrong answer. This activity worked in terms of showing how dissimilar views result in different outcomes since students received some feedback on their way of thinking. In terms of discussion, the aim was not fulfilled; the barrier was the large class size, which does not promote discussion or the give and take of asking and answering questions. Nevertheless, the test provided a change from the normal lecture and created activity that made the students think about how they make decisions. It also demonstrated that there is no right or wrong answer.
Electronic tests. At the end of the semester a set of eight electronic tests was published online as a help for students in reviewing the syllabus. Unfortunately, these tests were not aligned with the learning outcomes and were therefore less helpful as an activity underlining what the students needed to write a good exam.
We found it difficult to use the LMS for testing according to the learning outcomes in Organizational Behaviour and Management; however, additional experience with our new platform will allow us to revise the tests, making them more useful in terms of constructive alignment.
One Minute Paper.OMP was used twice during the semester to uncover difficulties or areas that were unclear. OMP was surprisingly effective: even in very large classes it was quite easy to uncover areas that needed to be repeated. We feared it would mean a great deal of work to cover all areas, but the students were, in this respect, very united in where they were experiencing or not experiencing problems. Feedback was given primarily in class, but also on the LMS. One advantage of using the LMS was that it was possible to give an immediate response, while class feedback meant waiting until the next class. Our conclusion was that OMP is a powerful tool that can help teachers reveal problem areas and thereby improve student learning.
This article summarizes our subjective experience from working with the course Organizational Behaviour and Management in spring semester 2010. It has been very useful in terms of planning teaching according to constructive alignment, making the teacher focus more on what is most relevant in the syllabus. In a distributed course of this nature, this method can theoretically ensure that students get roughly the same information regardless of teacher, class, or campus. Unfortunately, this was not the case in practice, since using the material was elective and use of the different tools varied among different course instructors. To reduce these discrepancies, compulsory use of the published material may be worth considering.
As described in this paper, we achieved rather varied results. Some activities worked according to intent while others were less useful. However, we still believe that active students learn and that these activities, if further developed, will help students learn more and better.
In the introduction we asked, “How can we make junior business students understand the importance of learning organizational behaviour and management?” This question remains unanswered. What we have obtained, however, is some greater understanding of what activities can be readily implemented in large classes and which need more work to make them an appropriate tool for the teacher.
Biggs, John & Tang, Catherine (2007). Teaching for Quality Learning at University, (3rd ed) Maidenhead, UK: SRHE and Open University Press.
Brookhart, S. (2001). Successful Student's Formative and Summative Uses of Assessment Information. Assessment in Education 8(2): 153-169.
Chizmar John F. and Ostrosky Anthony L. (1998), The one-minute paper: Some empirical findings. Journal of Economic Education; Winter98, Vol. 29 Issue 1, p3-10, 8p, 3 Charts
McKeachie, Wilbert J. (2002). McKeachie’s Teaching Tips, (11th ed) Boston, Houghton Mifflin Company.
Ramsden, P. (2003). Assessing for understanding (pp 176-206). In: P. Ramsden (ed.) Learning to teach in higher education. London, RoutledgeFalmer.
Taras, M. (2002). Using assessment for learning and learning from assessment. Assessment&Evaluation in Higher Education 27(6): 501-510.