Associate Professor
Norwegian University of Science and Technology
Email: Arvid.Staupe@idi.ntnu.no
My experiences described in this article are based on lecturing at the intermediate level at Department of Computer and Information Science (IDI) at NTNU. The primary cause for initiating the research projects, which lasted several years and with a gradual increase in the use of ICT, was the tenfold increase in number of students over only a few years, from barely 20 to more than 250. Two positive results were a significant improvement in grades and a large increase in the proportion of students completing the course compared to earlier years, despite a strong increase in number of students and a decline of resources. Seventy percent of the students registered for the course completed and passed the exam, compared to forty nine percent the year before. For two other similar courses the completion percentages for this pilot year were thirty two percent and forty five percent, respectively. The average final grade improved from 3.2 to 2.5, where 1 is the highest grade in a scale from 1 to 6. The course was structured applying blended learning, net-based learning, and mind tools. The course was comprehensively evaluated with an external examiner and the results were compared to corresponding courses within the program with more traditional formats, and also to a more traditional format within the course itself. A traditional format includes lectures in an auditorium and approved/not approved when it comes to exercises in an obligatory exercise arrangement. The course was organized with a subject teacher, an exercise teacher (assistant teacher), and student assistants. There was one student assistant for each exercise group of 30 students.
Keywords: net based learning, mind tools, hyper system, multimedia, NitVis, theoretical platform, collaboration, learning in groups, learning environment, learning environment in internet, building own learning, internet, MapVier, collaboration tools, workbook, visualization tools, cookbook, correlation, results.
Over only a few years, the number of students registering for the course that I teach, Operating Systems, increased from 20 to more than 250. The teaching method may be appropriate in different fields. Both the students’ exam grades and the proportion of students who took the exam dropped significantly. One easy explanation would be that the increase in number of students naturally caused a bad result; some kind of natural law. Unwilling to accept this, I devised a teaching format centred on online learning and the use of mind tools. I set out to research whether it would be possible to improve the completion percentage and whether the students would achieve better final grades than with traditional class formats. In this experiment, students spent a lot of time on exercises in class. This article presents a two-year experiment that used a strong focus on the use of Information and Communication Technologies (ICT). The following year, after a stop in the experiment, new experiments were carried out, this time with alternative evaluation. The students could choose between 3 different formats for evaluation; 100% traditional exam with optional exercise reports, 50% traditional exam and 50% exercise work or exercise work counted for 100% (Staupe, 2011, forthcoming). These experiments were based on the teaching formats described in this article.
From a learning theory point of view my work was based on the hypothesis that it is the active, hardworking student who learns. This means that passively following the lectures and making an all out effort just before the exam is not enough. To me it was important to come up with a teaching system where the students not only learn bare facts, which they can then reproduce at the exam, but are also capable of producing and creating something themselves by applying principles and methods learned in the course. My view on this topic is strongly influenced by Steen Larsen, professor at Danmark Lærerhøjskole (teacher’s university), who emphasizes that “one cannot “learn” others anything – it is the hardworking student who learns” (my own translations) (Larsen, 1995). This view accepts the principle that the teacher can only communicate information for the students; it is up to the students to learn. Larson interpreted one type of classroom experience as follows: “A chair is a misinterpreted bed, when you sit down your brain switches to standby mode and thinks that you are about to go to sleep” (my own translation). This means that a lecture is too often seen as a form of “entertainment” which one more or less passively follows.
Larsen focuses on the fact that there is a fundamental difference between teaching/education and learning. According to Larsen, recent learning theories equate the terms “learning process” and “work process”, and even though these two terms are not synonyms, they are at least two sides of the same coin.
Unfortunately, he says, everyone involved in teaching and education has supported the students’ lack of responsibility and that it is the teachers who are responsible for the students’ learning. However, teachers cannot “learn” students anything, thus they cannot have that responsibility. Teachers, however, can ensure a good learning environment so that students who want may learn. This means that as a teacher you are responsible to do your very best to create a good learning process so that students can learn if they choose to.
The constructivist learning theory, on which Larsen bases his work, has grown from Piaget’s research on developmental psychology. Piaget (1985) introduced a theory claiming that learning happens when new information interacts with previous knowledge through a process of assimilation and adaptation. One direction in constructivist learning theory, a socio-cultural approach, especially emphasizes that it is through interaction and collaboration with others that new knowledge is developed and how the individual gains new understanding. Nonaka (1994) formulated a law saying that learning always happens at two levels, first at the inter-psychological level and later at the intra-psychological level. The internalization refers to the connection between the social and the internal levels, so that social speech is applied to interact with others, while internal speech is applied to talk to ourselves. Other researchers (Lave and Wenger, 1991; Mantovani, 1996; Hutchins, 1995), have gone even further in emphasizing the importance of the complete environment/context in which communication happens (situated learning). These researchers focus on learning as a group product and do not attempt to analyze learning in relation to single components or individuals.
Thus, learning becomes part of a bigger context rather than just learning facts within a course structure. The learning process is then tied to the factual knowledge that is to be learned and to the environment in which the knowledge is to be used. How one learns, i.e. the learning process has become highly central. A student, who has a well-rehearsed learning process, will later find it easier to stay focused in a shifting knowledge environment; a process that lays the basis for lifelong learning. In the experimental teaching structure, there was a particular focus on:
Collaboration, learning in groups: Through group activity we learn to establish trust and collaboration skills, asking motivating questions and participating in a creative process through dialog. In a good learning environment, we acquire two types of knowledge: The first includes specific/concrete knowledge. The second involves functioning as a learning individual within the environment, developing the ability to adjust to other people and to take joint responsibility. This focus is especially emphasized by researchers of situated learning. Resnick (1998) points out that humans will for the greater part of life learn and work together and not alone and individually, which is in direct contrast to the situation in traditional schooling. Most people will recognize the pleasure of being part of a well-functioning group where the group members trust one another, where they complement each others’ strengths and compensate for each others’ weaknesses, where the group has greater goals than those of the individuals and where group work leads to excellent results. In the end no one may remember who said or did what.
Desktop learning: Learning by the use of interactive media, hypermedia/multimedia. Using such media will support different learning styles and engage the students in their own learning processes. One may seek information covering a bigger scope than what has been possible within traditional lessons, which are based upon available printed material. By using interactive desktop learning, one is allowed to read, write, listen, and speak in a larger world of information.
Based on the learning principles discussed above, we see how available services in a desktop environment can be structured and applied in order to support a learning environment, and where the participants genuinely collaborate in a complete context. The resources/services available in a virtual competence network can be structured from tasks and the activities to be strengthened. I have chosen to structure the services in five arenas, according to which tasks/theories that are to be dealt with: A presentation- /lecture arena to present and receive subject material/theories, a knowledge arena to access library services/online services and the need for answers to questions one might have, a working arena to administrate material/information and to produce new material, a private arena for personal notes and reflections that also can be gathered over time (a private folder), and last a communication arena which, together with the working arena, make up the core of a virtual learning environment based on constructivism and situated learning.
Figure 1. Structuring a virtual desktop learning environment/competence network with sub arenas.
One criticism against behaviourist learning theories has focused on the claim that “someone can learn someone something” and that learning is managed by the teacher. But we know that “the hard worker which learns”, (Larsen, 1995). If this claim is true, then, the possibility of spreading subject material in a simple and flexible way by the use of a computer network must not be confused with learning. However, the subject material can be presented and structured well by the use of a computer and via a computer network by the use of different media, such as text, drawings, images, animations, video, and audio. The computer or the network can in this manner provide and help organize material, but a person must make use of this to learn. A teacher can help a student to organize and apply material, through which process the student may learn. The degree to which this subject material becomes new knowledge for the students depends on the knowledge and the effort of each student.
The activities linked to this arena must both present subject materials in an interesting and comprehensible way, and receive subject material accurately, assuming that the collaborating partners are equal. Based on cognitive principles, this arena will be very important if the subject material is well organized. How much activity there will be in this arena will depend on the value of the subject material to a given user at a given time. There will be a lot activity in this arena if the courses are linked to
extensive use of the subject material (reproductive learning). You cannot
expect so large activity in the arena if the aim for the course is to create
something entirely new where there is not so much to “imitate”. You have to
create your own completely new material in the working/private arena.
This arena is also an information arena. Its purpose is active search for information/answers to topical questions. Databases and libraries will be used to search academic information, and questions will be linked up against “oracle services” to get in touch with resource persons and/or digital information sources, such as Artificial Intelligence (AI) or stored answers in “Frequently Asked Questions” (FAQ). The services both in the presentation arena and in the knowledge arena must mainly be seen as resources in the learning process of the learner.
This arena is for organizing personal material and is the real learning arena. It is based on constructivist learning theories. The learning happens through the learner’s processing of information and production of new material. The tools available in this arena are the same as those available in the presentation-/lecture arena plus artificial intelligence, but here they are used to organize and produce new material. Tools for group ware comprise the personal part of collaboration with others, while communication with others through group ware happens in the communication arena. Here, in the work arena, a work book is available for working with personal material, while storage of the material is placed to the personal arena.
The purpose of the private arena is to store personal material. This material can be in the form of a personal folder for life that you can take with you to develop further to use in solving future tasks. The workbook is stored here. One may also build material linked to artificial intelligence (AI). Together with the work arena, this is the most important arena if the learning environment is based on situated learning.
This arena forms the basis of communication in the virtual learning environment/competence network. Here we find group ware, e-mail, the world wide web (www), and other tools for communication in computer networks.
The learning environment consisted of a web page and access to services, such as the hyper system and an oracle service. Traditional lectures were usually recorded on video and synchronized with corresponding material in the hyper system. An electronic workbook was connected to the course; here the students were expected to present their own and the group’s reflections based on project-based learning where group work was a requirement. We did not apply the most extreme form of project-based learning, i.e. that the students themselves are responsible for planning their studies. From experience, we know that most students would like a framework, an organized structure. However, at the same time we should keep in mind three basic principles for constructivist learning theory/situated learning: Focus on active participants with a will and intention, focus on genuine collaboration among the actors, and focus on a complete environment/context that forms the basis for learning and genuine collaboration.
One of the core parts of the learning environment was the multimedia-/hyper system, through which one can access the subject material and many of the functions listed as needs in a learning environment. The toolbar for this system is shown in Figure 2 (Warholm, 2000).
Figure 2. The toolbar in the multimedia-/hyper system
On the left, we have first two navigation arrows, then MapVier (see below), text search, presentation of document structure, notes, videos, work palette, collaboration tool, paths, configuration, help, exit, history, storing personal work palette, and technical information about the document.
A general problem in a hyper system is control (“lost in hyperspace”). What am I, how did I get here and how do I move further? Through MapVier we get, when we wished, graphical control of information and where we are in the multimedia-/hyper system in the form of a document map.
Figure 3. Example of initial screen, main screen, when choosing MapVier (Tilbake; back, Hjem; home, Fram; forward, Lås; lock, Lenker; links, Gjem; hide, uke; week, uker; weeks, mnd; month, år; year) (Albinussen T, 2003).
There are many options. You may present the document map in different ways:
As icons
As complete text
As a magnified image
Drag a document to the desired spot on the screen
Present the links that exist between the documents
Choose parts of the documents
Get information about whether or not a document has been visited and how long since you visited the document. You can choose between 6 different colours from a palette to indicate it, one colour for one week and another colour for two weeks etc. Documents with no special colour have never been visited.
By using the document map one may navigate in two different modes, either just navigating around in MapVier or synchronizing the navigation with entries in the multimedia/hypermedia system. The documents will then be presented during the navigation. If you navigate around in the system in other ways, for instance with link entries, MapVier will always follow and be visible with the current document as the central document of the document map. You are not “lost in hyperspace”. In figure 3 the cursor is placed on “oversikt (outline)” and is presented with full text close to the document and with individual information windows with more information about the document if desired (right mouse button). The information window (“Info”) tells us whether the document includes an image, time of last visit, external links from the document with URL-addresses, whether it contains animations, document links to other documents with document names and belonging links to the document from other documents. You can freely select a document in the document map as centre document.
It is possible to store personal notes/comments for each document. Documents with notes for the user are shown as an icon (pen and paper) in the upper right corner of the document and the information can be accessed. All personal notes are also displayed in the work palette and can be accessed at all times.
Video is available via an external but integrated service. One may choose to use Windows Media Player, Marratech, etc. with the functions these offer.
In this window the user may view all his/her own notes, add links to interesting documents, and read messages from the subject teacher.
Here, the user gets access to collaboration tools such as Marratech.
Virtual links between documents. There are two types of paths: personal paths and paths that others have added, e.g. the subject teacher. Navigation inside paths is supported. Only the person who initially added the path, the “owner” of the path, may edit it at a later point. One of the modes in the system makes it possible to automatically trace the navigation in the document map, i.e. that documents gone through at a lecture for instance are added as a path. For a student, a possible path could be everything about Unix, etc. When using playback you can choose path and then traverse it back and forth at your own tempo; stopping to go in-depth, watching/listening to videos, etc.
The user decides the layout of the documents. There is a parameter for deciding the size and the layout of the text, the colour of the links, background image, and language. Our course was available both in Norwegian and in English; one could switch at any time between the two languages.
An electronic workbook was created for the course. Here the students were supposed to present both their own and the group’s reflections. In the workbook you had entries for transparency sheets/overheads, lectures, tasks, and miscellaneous. Mode buttons for Edit, Comments, and Help. A student who wished could design his/her own workbooks with corresponding functions.
Keeping an updated workbook was central. To ensure progress in the learning process, each group was responsible for making transparency sheets for the lessons according to the estimated pace for the course. The group members took turns to take responsibility for the lessons. The assignment was to create one series of transparency sheets that covered the entire course. Transparency sheets that individual students had been responsible for were to be included in the workbook. With groups of four, each student had to make a transparency sheet every four weeks. The number of transparency sheets for each lesson was limited to seven. The transparency sheets were collected as a series in the workbook. Each of the students was also assigned to write a summary of each lesson in her/his workbook.
During the semester, a number of tasks were given in order to ensure progress. The tasks made up an important part of the work method. Based on problems/tasks or problem indicators one was to indentify the learning needs in order to better understand/master/solve the problem. Problem indicator means that we need to find the problem to able to solve it. The idea was that the tasks were to be solved as a joint project within the group and the result was to be entered in the workbook for each group member. The obligatory work on the lessons, the individual tasks, the workbook and the project counted for 50 % of the final grade. The deadlines had to be kept. At fixed intervals, copies of the workbooks were made. Tasks handed in after deadlines were not approved; last minute all out efforts were to be avoided.
Modern cultural psychology has strongly emphasized that “people think through tools (artefacts)” (Cole, 1990). Artefacts in a broad sense, that is. Mind tools are ICT-tools and ICT–learning environments that have been developed to function as an intellectual assistant to the learner, to engage and support both critical thinking and learning at higher levels (Jonassen, 2000). We might say that the use of work book to a certain extent belongs to mind tools, but I wished to go further and develop a new mind tool. Jonassen, Peck, and Wilson in Jonassen (2000) describe meaningful learning as:
Active (manipulate/observe) – the learner interacts with the environment and manipulates the objects in the environment.
Constructive (“putting into words”/reflective) – the learners integrate new experiments and interpretations of previous knowledge; constructing their own simple mental model to explain what they observe.
Intentional (reflective/regulating) – the learners can present their learning aims, what they do, the decisions they make, the strategies they make use of, and the answers they find.
Authentic (complex/happens in a context) – learning/teaching happens in a meaningful genuine practice or in a simulated or problem-based learning environment that illustrates reality.
Collaborative (teamwork/co-operative) – the learners work in groups where they share a common understanding of a teaching system and expectations they have of methods in use, and where they as a group discuss and analyze the results.
System-modelling tools are considered to be the most powerful mind tools (Senge, 1990, 1999). Two useful system-modelling tools are Powersim (http://www.powersim.no) and Stella (http://www.iseesystems.com). These include the whole spectrum of knowledge and skills within complex thinking, such as seeing connections (synthesizing) and not least seeing new connections, being intuitive, and being able to express one’s thoughts by the use of models. Visualization tools are also mind tools. However, they are not always independent tools – they often appear in connection with other tools such as spreadsheets, software tools for mathematics, etc.
Visualization tools help the learners to perceive ideas and perform tasks that they otherwise would not be capable of, or that they would have problems understanding (http://en.wikipedia.org/wiki/Software_visualization). It is well known that many people more easily perceive new impressions through visualization, rather than having things presented as purely theoretical or abstract problems, as often is the case in education. One such case is the teaching of operating systems. When performing specific tasks such as programming parts of an operating system, teaching without visualization tools takes too long and does not necessarily provide us with the kind of knowledge we want. Even if the students are capable of programming simple tasks, they often have problems completely understanding what happens. In many tasks the programming problems might become dominant over the other concepts and subject matter.
A new mind tool was developed for the course, NitVis, see figures 4 and 5.
Figure 4. System layers in NitVis
The origin of NitVis was the idea of creating a learning process that involved visualizing various scenarios within the subject arena. Being creative, and developing and working with such scenarios, will provide good learning; include many of the components from complex thinking – not least within creative thinking. Our subject includes many abstract mechanisms, which are not always easy to understand. By studying specific scenarios, one more easily comprehends general concepts and mechanisms. By using a computer one may also visualize scenarios through animation, not just individual stages, but also the transitions between stages.
The solution was to build models that the students could manipulate at a high level, with domain-specific terms. The model consisted of components from the subject arena. In order to visualize a scenario the student had to program a script called a scenario where the elements are declared and later given operations and commands. NitVis interprets the scenario and produces an animation that can be studied in an accompanying Animation Viewer in the form of a Java-applet, so that the animations can be available on the website.
One may also create a scenario for other domains. The teacher and/or the students can define the components, e.g. the students initially are allowed to access a selection of components – and then a library can be created, which may be expanded later.
Figure 5. Example of an interface, scenario reading from disk, student hand-in exercise.
The user had good control of the animations with well-defined states. When an animation reaches a new state it stops until the user wishes to go to the next, or chooses to go back for a closer study of what happened so far. One may also “jump” back and forth between different states. To choose state a railway track is used as a metaphor; one presses down a tie. It is also possible to run the entire animation in one go, from any state (the “Play to end”-button).
NitVis is a combination of simulation and visualization tools, but not being fully equipped with functions and services like we know from commercial dynamic simulation tools, such as Powersim. Thus a system is called scenario writing and not a simulation system. In a pedagogical context we would find many similar characteristics. As with simulation, scenario writing is simulation of activity in a real system. System thinking/simulation/scenario writing could be at different levels, depending on one’s ambitions for the students’ learning, and how complex the system is:
a) The students can run a fully developed model.
b) The students can adjust and manipulate parameters in the current model.
c) The students can change the current model.
d) The students create their own models.
Steed (1992), quoted in Jonassen (2000), argues that building models challenges thinking at a high level, such as evaluation, analysis, and synthesis development. System development represents some of the most complex intellectual activities that students face in an educational context. Jonassen (2000, p. 148) wrote:
I believe that building systems models requires deeper thinking and
understanding of a problem than with any other type of learning activity.
Because it is so complex, learning to build systems models will require more time than learning to use other Mindtools. The intellectual results will justify the effort.
In a learning situation, it is important that the students develop the ability to think at a higher level and develop the ability to communicate their knowledge to their surroundings. Jonassen (2000) lists three levels:
The students demonstrate their models in class. The class discusses and evaluates the models and comments as colleagues on disadvantages, weaknesses, and possibilities in the models.
The students develop their own theories. Using the already-built models as starting points, the students should try to generalize the results in theories that characterize the models, either from entire models or bigger or smaller parts.
The students reflect upon the activity.
Instead of postponing the reflection process until they have completed a project or a task, the students should see this as a continuous process. Do we achieve our goals? Any changes needed? Are we answering the questions or what we emphasize? After finishing the project one should reflect upon one’s learning outcome. What have we learned from developing computer models and developing our own mental models? What have we learned from working together? NitVis was developed for this learning environment.
Many different ICT-tools were relevant for the course, and for those a “cookbook” was developed. Here one could get guidance in the form of short video clips linked up to visualization and animation of the usage. It was easily accessible, one click away – choose tool – and it was there. It was partially developed by the use of NitVis.
The course was organized with a subject teacher, an exercise teacher (assistant teacher) and student assistants. In all, 256 students registered for the course and 235 registered for the exam the first year.
The second year, the course was first announced as an open study, then, due to financial arguments, it became closed, but then was opened again without notifying those that had been refused. There were many rounds back and forth, and therefore table II doesn’t present a real picture of the interest in the subject.
The result was 183 students registered for the exam. Some students followed a traditional practice arrangement with approved/not approved and a requirement of handing in at least 30 % of the exercises, the rest followed the new structure with grades and all exercises submitted.
The lectures were given on the traditional schedule of once a week. These were open to all students. Since so many showed interests in the course and since the lecture halls were too small, each week’s lecture was presented twice. In addition, most lectures were made available on the Internet as video on demand, synchronized with the content of the hyper system. Some students only followed lectures on the Internet, either because their preferred to, or because they were unable to physically attend the traditional lectures. Others switched between auditorium/video or followed both. The idea was that one could search for videos while doing exercises.
The exercise work was structured primarily as group work. The exercise group was first divided into the large groups of 30 students. In earlier semesters, these large groups were only 20 students. Each student assistant was responsible for a large group. Each large group was in turn divided into smaller groups of 4-5 students who collaborated and together produced the group part of the exercise work. One large group was an “online group” that worked and received guidance solely over the Internet. The other groups attended a laboratory and received guidance in the traditional way at regular hours once a week, in addition to online guidance.
In the introductory week of the course the focus was on learning methods; these were presented as an ordinary part of the course, with nothing said about exercise work. Earlier, several student assistants had been trained and had practiced the different roles in project work; group leader, secretary, etc. The student assistants attended the start-up of each group to focus the group on how to succeed with group work.
It was also possible to ask questions through an online oracle service via the Internet. The student assistants, and to some extent the course teacher, answered the questions, often immediately. In the oracle service, the students could see if a student assistant or the course teacher was active, or busy answering other students. Due to shortage of resources, this alternative does not offer the expected opportunities. However, at the end of the semester, we, to a certain extent, opened for this alternative and many questions were both asked and answered. Experience from other courses showed great interest in the oracle service. No less than 2500 written enquiries were received and answered during a single semester for one course. Even though the student assistants, and not the course teacher, answered most enquiries, all enquiries were made available for the course teacher to learn what kind of problems the students encountered and where the shoe pinched. The teacher could see and evaluate the responses the student assistants had sent.
Switching to a new course structure is not a simple operation and I probably wouldn’t have dared do this if the department hadn’t decided to assign more resources, especially to the exercise part, so that they equalled the use of resources in equivalent subjects. This was decided at a faculty meeting and was also confirmed later. Feeling confident about decisions and assurances, we put the new arrangement to life. However, instead of having 20 students per group the student assistants were now expected to serve 30 students. It became hectic. Today (in 2010), we have a traditional arrangement of one student assistant per 10 students.
Meetings were held weekly between the subject teacher and the student assistants. When the subject teacher was busy, the exercise teacher led the meeting and wrote full minutes. Obviously, the student assistant located in Canada could not attend the meetings; however, she kept close contact over the internet – especially with the exercise teacher. The “around the world” placement would prove handy; many students work late at night and could then get quick answers from Canada.
Most groups and students handed in impressive workbooks. They had put a lot of effort into both the individual part and the group part. In the exercise part of the course, each part of the exercise was due on a deadline for entry into the workbook. Then the student assistants read through and commented on the workbooks. Each student handed in a workload of some 50 to 70 pages. Each workbook was evaluated so that the group work and the individual work counted equally. In addition, virtues such as keeping deadlines were evaluated. The evaluation of workbooks and credit for keeping deadlines counted for 50 % of the final grade. In addition, the groups were, to some extent, evaluated on teamwork and how each member contributed to the group. The result from this evaluation was not included in the final course grade, but used as a basis for studying various aspects of group work.
Figure 6. Correlation between grades for workbook and exam showing grades 1 to 6. Grade 1 is best.
Many students achieved better grades on their work in the workbook than on the exam, some even much better. However, some students did better on the exam than in the workbook. It appeared that the latter group had put less effort into the workbook, and the groups to which they had belonged had not been well functioning. Even if these students had been working on the material, perhaps we had a group that was not adept at expressing themselves in writing, even though they had time and available equipment. Possibly, some of these are also incapable of following the written requirements of a master’s program. There seem to be many who quit a master’s study because they have problems completing larger written tasks; this despite the fact that they manage to do well in an individual subject with traditional written tasks and therefore were admitted to a master’s program.
Analyses of the workbook grades compared to exam grades show little correlation (0.2 – 0.3). Chi-square analyses yield similar results. If we make a hypothesis saying that the exam grade is to exist within a certain distribution from the means between the workbook grades and the achieved exam grades, we see that the result is not within a 5 % level. This means that the observations deviate from what we consider as “reasonably normal”. It is obvious that different qualities and skills were measured when evaluating the workbook compared to what we traditionally measure on the exam. However, this does not mean that the qualities measured in the workbook are not as good or important. Rather the opposite if we see this as a comparison between creating and producing material individually and creating and producing material in collaboration with others. As discussed in the section “Learning theoretical platform” above, most learning in the educational system is individual, but much learning in the outside world is in a group setting.
One should notice that even though there was only a weak correlation, the students achieved a significantly better result on the exam than the years before the experiences. An improvement in result of 0.7 grade points for the mean grade and the same for the median should be considered a significant improvement. So, one gets the best of both worlds; one has achieved a good “work result” and a good traditional exam result.
The good results correlate well with the student assistants’ observations. On several occasions throughout the fall, they reported that the students were “much better than previous year”. The external examiner expressed the same experience and we discussed what changes were made in the teaching system. For those who followed traditional teaching we did not get the same good results, and the median was weaker than the mean grade; a tendency towards a weaker result than the mean grade.
It should be mentioned that some students complained about the new teaching system, especially at the beginning of the course. They thought that writing belonged to Norwegian as a subject and that they finished this in the upper secondary school. But they were not familiar with interdisciplinary work, where knowledge from one subject can be applied in another. And they were not familiar with writing and structuring of written tasks as part of a learning process.
When discussing the course structure at the end of the course some students criticized the course for being too demanding compared to the number of study points. Others strongly disagreed with this statement. They felt that the course was not more demanding than it ought to be. During the discussion it was also said that it was when one put off the exercise work and went for the “all out effort at the end” method that the course seemed very demanding. After the course was finished, I received a lot of feedback on the course structure from students who stopped by my office, and most of them were very positive. Some expressed that all courses should have been structured like this.
The exam was a traditional 4-hour exam. In order to pass the course one had to pass the exam. In addition to subject teachers, external examiners were being used. The exam counted for 50 % of the final grade.
For year X+2 I went back to a traditional teaching system, see results in Table I.
Table 1. Structure of the study. First experiment year called X. Grade 1 is highest/best, grade 6 lowest (*open/closed/open study, ** closed study). Exam headings; Attended, Passed, Not passed, Mean grade, Median, Standard deviation, Skew symmetric, Better than grade 1.6 in %. Curriculum text; Traditional, Traditional, Traditional, Traditional, Traditional, New, Traditional, New, Traditional.
As we can see, the exam grade for the new course structure improved from 3.2 to 2.5 the first year (X). In year X+1 the median rose slightly more, from 2.5 to 2.4. The course grade, which includes the workbook, has improved from 3.2 to 2.3 for year X and for year X+1 it rose further to 2.2. For students who chose a traditional arrangement the average grade were 2.9 for both years with a 3.1 median.
Table II. Summary of results; Control Courses in Subjects I and II and Experimental Course (The Subject) (* Data not available).
Table II shows completion in control subjects, Subject I and Subject II, and in the subject in which the experiment was carried out (Subject). No less than 70 % of those registered for the course completed and passed the exam, compared with 49 % the previous year. For two other similar courses, the completion rates for the research year were respectively 32% and 45 %. For the years X-2 and X-1, only 43% and 49% of students registered completed the course and passed the exam. Passing grade is 4.
Figure 7.The use of ICT to support learning goals.
Extensive research was done on learning goals as evaluating construction of knowledge, self-discipline, collaboration, critical thinking, creative thinking and complex thinking, in all 24 questions. The questions went to both the student advisors and the students. Figure 7 show that more than 90 % of the students’ own assessment and more than 75 % of the student advisors’ assessment show that mind tools contribute to some extent or strongly toward reaching specific learning goals. Barely 5 % of the students thought that mind tools were of no value in reaching specific learning goals.
Figure 8. Evaluation by students and student assistants of lecture, auditorium/video and ICT-learning environments for building own learning.
Figure 8 shows that the average values attributed by students and student assistants to lectures and use of ICT learning environments are almost equal; almost 50 % think that lectures, as well as ICT learning environments, are “strongly important” or close to such a label.
For this survey, we see that lectures are still of importance. It was difficult to separate between lectures in auditorium and on broadcasted two-way video, sometimes the students attended auditorium lectures, other times they followed video lectures, and some made use of lectures in auditorium and viewed videos later. They could freely choose. However, based on attendance statistics from use of the video, the majority – possibly all – of the students chose to follow video lectures in some form. The distant learners steadily used broadcasted video lectures and watched videos later.
More than 90 % of the students’ own assessment and more than 75 % of the student advisors’ assessment show that mind tools contribute to some extent or strongly in reaching specific learning goals. Barely 5 % of the students thought that mind tools were of no value in reaching specific learning goals. Almost 50 % think that lectures, as well as ICT learning environments, are “strongly important” or close to such a value, in their own learning.
No less than 70 % of those registered for the course completed and passed the exam, compared with 49 % the previous year. For two other similar courses, the completion rates for the research year were respectively 32% and 45 %. The average final grade improved from 3.2 to 2.5, where 1 is the highest grade and 6 is the lowest. For the years X-2 and X-1, only 43% and 49% of students registered completed the course and passed the exam. Passing grade is 4.
Based on experience from this course and several other courses, it seems that the conditions and the structure around the practice in a course affect the results much more than the lectures. This corresponds well with what to expect from a hypothesis saying that active learning gives better results than mediated education. “It is the hardworking student who learns” (Larsen, 1995 (my own translation)). Both the scientific control group in the course and those students following the new practice system had access to the same lectures, both auditorium- and online lectures. However, the scientific control group only achieved a slightly better result than the two previous years’ classes; their average grade improved from 3.2 to 2.9 (median 3.1), compared to an improvement from 3.2 to 2.5 (median 2.4) for the new arrangement.
Practicing working closely in groups seems to be both demanding and time-consuming. Experiences from the course show that most groups practiced cooperative and not collaborative teamwork.
Most groups quickly divide the tasks among the group members who in turn send their parts to the one student responsible for putting it all together. Or, some groups would meet once a week to link everything together into one solution. However, there were exceptions and particularly one group was able to carry out real collaborative teamwork. Not all tasks lend themselves to collaborative teamwork, some of the students claimed, but this can easily be used as an excuse.
In order to practice such work methods it is not enough to only grade the result of the practice system. One should probably also let the work method count – i.e. the work process. Actually, it is not strange that we expect academics to be able to collaborate with others in solving complex tasks. In the course “Eksperter i team” (Experts in Team) at NTNU this is emphasized and a separate grade is given as part of the final subject grade.
When using a traditional teaching system it is difficult to attack a lecture-based education, which has been the basis of traditional education for decades – not to say centuries. In introducing something new one must expect criticism. But still, I decided to take my chances and did not move far from what the students were used to, the traditional lectures were kept. Having the responsibility for 200-300 students, I chose to take things slowly, one step at a time.
When we earlier pointed out that scarcity of resources were a huge disappointment, this was to emphasize the fact that the new arrangement was not completed because of a “flow of milk and honey”. However, it should be made clear that the exercise teacher and the student assistants did an excellent job.
The teacher’s challenges are not made easier or less demanding, and that was never the intention. It is the responsibility for having done one’s best to organize the students’ learning process so that the students can learn if they actively make an effort that should be the teacher’s responsibility and main task.
Albinussen, T (2003). Navigering i internett og hypermedia, NTNU/IDI, Master’s Thesis.
Cole, M, (1990). Cultural psychology: A once and future discipline? in Bergman, J. J. (Ed). Nebraska Symposium on Motivation, 1989: Cross-cultural perspectives, volume 37, (pp.279-336). Lincoln: University of Nebraska Press.
Gardner, H. (1983). Frames of mind. New York: Basic Books.
Gardner, H & Hatch, T. (1989). Multiple intelligences go to school: Educational implications of the theory of multiple intelligence. Educational Researcher, 18 (11), 4-10.
Hutchins, E, (1995). Cognition in the Wild. Massachusetts, MIT Press.
Jonassen, D. H. (2000). Computers as Mind tools for Schools. Merrill Prentice Hall.
Larsen, S. (1995). ”Man kan ikke lære nogen noget – mod et nyt læringsbegreb”
in: Christensen, A. (red): Individualisering og demokratisering af læreprocesser – muligheder og betingelser. SEL, København 1995.
Lave, J. & Wenger, E., (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Mantovani, G., (1996). New communication Environments - From Everyday to Virtual. Taylor & Francis Ltd, London.
Nonaka, I. (1994). “A Dynamic Theory of Organizational Knowledge Creation”, Organizational Science, Vol. 5, no 1,
Piaget, J, (1985). The equilibration of cognitive structures: The central problem of
intellectual development. Chicago: University of Chicago Press.
Resnick, L. B., (1994). Situated rationalism: Biological and social preparation for
learning, in Hirschfeld, L.A. & Gelman, S.A. (Eds.) Mapping the Mind: Doman.
Senge, Peter (1990). The Fifth Discipline - the art and practice of the learning organization, 5. Paperback edition, Currency Doubleday, New York.
Senge, Peter (1999). Den femte disiplin. Oslo: Egmont hjemmets forlag. (Norwegian edition).
Staupe (2011). Alternative online evaluation in a Blended Learning Environment. Seminar.net, International journal of media, technology & lifelong learning. www.seminar.net (forthcoming).
Tapscott, D., 1998. Growing up digital. The Rise of the Net Generation. McGrawHill.
Warholm, H-O. (2000). Hypermedia i undervisningen. NTNU/IDI, Master’s Thesis.