Department of Didactic Science and Early Childhood Education
Stockholm University
Email: monica.nilsson@did.su.se
For Simon, who is nine years old, reading and writing are obstacles rather than tools. However, when offered the opportunity to create digital stories, Simon becomes deeply engaged. He creates complex stories within different genres based on a variety of subjects. He both expresses and creates meaning through his digital storytelling. He develops what Elbow calls ´voice.´ In the paper I argue that digital storytelling became a resource for Simon which triggered his interest in literacy because of two specific features: multimodality and narrative. Vygotsky’s work on development of literacy and creativity comprise a theoretical point of departure and I claim that new media has the potential to play a significant role in this realm.
Keywords: Digital Storytelling, voice, narrative, multimodality, literacy, creativity.
Simon is a 9 year-old boy who has problems with school, or put differently, school has problems with him. Among other things Simon does not want to read and write and is considered generally restless with difficulty concentrating. He shows no patience for schoolwork and moreover, he has been diagnosed with Tourette’s syndrome —an illness causing him to tic severely—which puts an extra strain on him. Simon’s teacher knows him to be creative and to have several interests such as dinosaurs, sports and origami. When offered the opportunity to create digital stories, Simon becomes deeply engaged and demonstrates accuracy and persistence. Why is this the case?
In this paper I am concerned with literacy. Here I understand 'literacy’ to mean two different things: on the one hand, learning to read and write, on the other hand, such things as drawing conclusions, making associations and connecting text to reality. Moreover, literacy also means creating meaning, understanding and at the same time being critical (Gee, 2008; New London Group, 1996; Säljö, 2005). But I am also interested in literacy and its relationship to creative ‘text based activities’ where text is not limited to script but implies a variety of modes such as script, image, and sound (Kress, 2003). I am particularly interested in how text based or literary activities, broadly defined, can enable children and young people to communicate experiences, explore new meaning and knowledge, and perform self-representation and self-expression.
Literacy in educational contexts is most often approached as a motor skill and not as a complex social, cultural and creative activity (Cole, 1996; Gee, 2008). This reductive tradition of understanding literacy is reflected in the students’ texts, which often lack communicative meaning both for the writer and the receiver, who is often only the teacher (Renberg, 2006). Students look for short cuts to good grades by imitating patterns and models. Renberg (2006) discusses this situation and claims that in the best-case scenario the texts work in the particular context (for example, as a school assignment) but it is obvious that the intellectual material is not processed, the perspective is unclear and the text basically lacks communicative purpose. Thus, writing in school is reduced to a ‘technique’ but never becomes a natural way of communicating. A text that doesn’t ‘want’ anything lacks ‘voice,’ according to Elbow (1981). A text that has a ‘voice’ is loaded with energy and has a clear, perceptible rhythm. Elbow even distinguishes between ‘voice’ and ‘real voice:’
Writing with voice is writing into which someone has breathed. It has fluency, rhythm, energy, and liveliness that are enjoying a conversation. Some people who write frequently, copiously, and with confidence manage to get voice into their writing.
Writing with real voice has the power to make you pay attention and understand – the words go deep (Elbow, 1981, 299).
Elbow holds the teaching tradition and the view of writing in the academic world responsible for texts lacking voice, particularly ‘real voice.’ Elbow claims that in the school tradition language is ‘outside us.’ In school, learning to write is reduced to conquering a code. The exercise becomes a surface imitation of genres and text types without being rooted in what is the core of language; according to Elbow (1981) this core is our need to express thoughts and feelings—to influence others. In his essay The Prehistory of Written Language (1978), Vygotsky discusses the problem of literacy teaching and learning in school:
Instead of being founded in the needs of children as they naturally develop and on their own activity, writing is given to them from without, from the teacher’s hands. This situation recalls the development of a technical skill such as piano-playing: the pupil develops finger dexterity and learns to strike the keys while reading music, but he is in no way involved in the essence of the music itself (105-106).
As discussed above, the implication of teaching literacy can be devastating to the child. Vygotsky writes that since the child has no intrinsic motivation to write, he needs to be encouraged to write about topics that are intrinsically understandable, that engage the emotions and most of all, encourage him to express his interior world in words. Very often a child writes poorly because he has nothing he wants to write about. Vygotsky cites Blonskii (2004, 46),
A child must be taught to write only about what he knows well and has thought about much and deeply. There is nothing more harmful to the child than giving him a topic about which he has thought little and on which he has nothing much to say. This practice tends to develop vacuous, superficial writers. To make a child into a writer one needs to develop in him a strong interest in the world around him. The best thing is for the child to write about what he is very interested in, especially if he understands it. The child must be taught to write about what he is deeply interested in and has thought about much and deeply, about what he knows and understands well. The child must be taught never to write about what he does not know, does not understand, and is not interested in. And yet, the teacher sometimes does exactly the reverse and thus kills the writer in the child.
Thus, the challenge is to create within the child the motivation to write and then to help him or her master the techniques of writing. According to Blonskii, suitable literary works for this purpose would be, for example, notes, letters and very short stories. Autobiographical stories are particularly motivational. “When a child has something to write, he writes with great seriousness” (2004, 52).
In this paper I show that Simon, in his digital storytelling activities, developed what Elbow calls ‘voice.’ I also show that Simon, in close interaction with his teacher, developed his vocal voice (speech), which was impaired due to tics, as well as a need to use written language. I suggest that there are several reasons for this. One is that the themes in his digital stories were chosen by him and thus conveyed his interests, skills and knowledge. Another reason is that his teacher actively and emphatically supported him in his efforts. Moreover, I argue that digital storytelling made Simon’s intense and creative activity possible due to two specific features, (i) a variety of modes (image, music, sound, speech, and writing) are possible to use and combine, and (ii) digital storytelling features narrative and dramatization, a mode of thought which is a natural way for children to express themselves. It is this latter issue that I will focus on in this paper, i.e., the role of narrative and multimodality in development of voice.
Two different kinds of data constitute the basis for analysis and discussion. The first kind is Simon’s digital stories, in particular, one featuring himself as a biker and inline skater and another about dinosaurs. The second set of data is a text written by Simon’s teacher that describes Simon’s creative process when making digital stories.
In sociocultural theory, the basis for my reasoning in this paper, language—spoken and written—mediates development of the human mind and consciousness which Vygotsky (1978) calls higher order functioning. However, there is no evidence that literacy, in and of itself, leads to the cognitive functioning of, for example, logical, analytical, and critical thinking that the ‘literacy myth’ (Gee, 2008) prescribes (Cole, 1996). Rather, it is the social practices in which reading and writing take place that make a difference (Gee, 2008). Thus, in order for the child to be a significant participant in communities such as school, peer groups and society as a whole, literacy is essential for a child to develop.
Vygotsky discusses appropriation of written language as a stepwise utilization of signs. He asserts that the mastery of the complex sign system of written language is “the culmination of a long process of complex behavioral functions in the child” (1978, 106). He describes how this trajectory starts with the use of gestures and continues with the use of drawing and play and is eventually completed with the written word.
Vygotsky asserts, “only by understanding the entire history of sign development in the child and the place of writing in it can we approach a correct solution of the psychology of writing” (106).
His discussion about appropriation of written language should be connected to his thought about creativity. In his essay Imagination and Creativity in Childhood (Vygotsky, 2004) Vygotsky argues that each stage of childhood has its own characteristic form of creativity. Literary and verbal creativity is characteristic of school-aged children following stages where play and drawing dominate as a creative form. For Vygotsky play is imagination and creativity embodied. He states that play is “imagination in action” (21987, 93). On the other hand, literary forms enable the youth to express more complex relationships, especially inner relationships, than does drawing.
In the kind of digital storytelling that I discuss in this paper, these diverse forms of expression are present. They are also merged and intertwined. Digital storytelling is made possible by digital technology (Lambert, 2002; Nilsson, 2008) combining different kinds of sign systems such as gesture, speech, images (still and moving), writing, music and sound. Hull & Nelson (2005) describe a digital story as a digital multimodal text. Multimodality emphasizes the variety and combination of multiple sign systems or modes (Kress, 2003). Kress argues that different modes have different affordances. For example, he argues that language (written and spoken) is regulated by linearity and time; the internal order of the parts offers meaning. In addition, they argue that words are relatively empty of meaning; we have to fill them with our own images.
Bruner (1986) makes a distinction between two language-based modes of thought or knowledge: the narrative and logical-mathematical. In the latter we ask what something is and in this form of knowledge we mean exactly what we say. The scientific discourse is implied. In the narrative we ask what something means and we mean more than we say; otherwise the story becomes flat. This mode values interpretation rather than scientific, unambiguous and logical thinking. Thus, there are different ways to employ language in the knowledge domain with different outcomes. In this paper I am interested in the narrative form. I will therefore discuss narrative in a little more detail but before doing so I will say something about the particular affordances that Kress (2003) ascribes to images.
Kress (Ibid) argues that in comparison to language the logic of the image is regulated by space and simultaneity. In addition, the image is filled with meaning but we do not have to read it in a given order. The positions of things located in the image offer meaning. If I write, ”I, Monica, live in Stockholm” it offers meaning based on how the words are related in sequence. For those who know me and know Stockholm the sentence tells as much as a photo could. But for those who do not know Stockholm or me, a photo could have provided more information than this short meaning, for example, whether I am long or short, blond or dark, that Stockholm is on the water, etc.
The origins of the attempt to define the essence of narrative is probably Aristotle’s Poetics. Here a narrative is a recounting of a whole, which has a beginning, a middle part and an ending. The plot is the essence in any narrative and comprises events that are ordered on a consequential basis. However, this definition of narrative has been problematized and today narrative is not only thought of as an object with a certain internal structure but as an activity. Ochs & Capps (2001), for example, conceive of narrative as a host genre that can encompass a huge variety of specific appearances. In their view, the narrative can be understood in terms of a set of dimensions (tellership, tellability, embeddedness, linearity, moral stance) that it displays to different degrees. They write, “Each narrative dimension establishes a range of possibilities, which are realized in particular narrative performances.” (Ochs & Capps, 2001, 19).
Engel argues (1995) that narratives of children are primarily constructed to satisfy the child’s need to be understood. In their narratives children solve cognitive puzzles in their world and make emotional sense of themselves and the people around them. According to Hakkarainen, “Narratives describe and organize the world in which the child lives and acts” (Hakkarainen, 2006, p. 194).
Narrative (written or spoken) or visual presentations are different ways to represent something in the world and they also offer different ways of making meaning (Kress, 2003). Thus as Kress asserts, “The world told is different from the world shown” (2003, 1). Kress further asserts that the combination of these modalities (language and image) requires more than interpretation of text and image separately. According to Kress (2003), in multimodal texts, fantasy (imagination) is about filling words with meaning and creating order in the placements of the elements in the image. In digital storytelling the modes of language and image are combined which makes it particularly interesting to study as a semiotic means in the context of creative literacy and literary activities.
Putting Vygotsky’s ideas about development of written language and leading creative forms in communication with the theory of multimodality has several implications. First, it becomes clear that the diverse modes are not only preparations for the written sign system but feature particular affordances in their own right. Second, the theory of multimodality presented by Kress offers opportunities to refine and develop our understanding of sign mediation, a concept first introduced by Vygotsky. By taking into consideration the different ways that the diverse modes offer meaning and means for learning and communication, separately and in combination, we might better understand their role in mediated activities. Third, the opportunities that digital technology offers tend to blur and transcend the boundaries between the leading creative forms that Vygotsky discussed. The possibility of mixing, for example, sound, script and image enable children and young people to express complex thoughts and emotions, as we will see in the next section. Another example would be the possibility of making images which information technology facilitates, for example, drawing (Klerfeldt, 2002) and photography. Finely, putting Vygotsky’s thoughts about development of writing and creativity in dialogue with the theory of multimodality hopefully will furnish pedagogues in their task to facilitate children’s learning and development, particularly creative activities as ways of communicating and expressing themselves. Studying digital storytelling is one way to start to investigate these questions.
When analyzing the digital stories I draw on multimodality and visual analysis (Kress & van Leeuwen 1996; Machin, 2007) as well as theory on children’s storytelling. In order to structure the analysis I employ the three meta-functions of language originally conceptualized by Halliday and later appropriated by Kress and van Leeuwen (1996). These are the three basic requirements of any semiotic mode in order to function as a communicative system, that is, the ideational, interpersonal and textual meta-function. The ideational meta-function represents and communicates states of affairs in the world (who or what is involved in what processes or relations) (Kress, et al. 2001). It represents ideas beyond its own system of signs (Machin, 2007). In visual design, for example, blue might represent the idea of the ocean. The interpersonal meta-function represents and communicates the social and affective relations between the participants in the act of communication, that is, between the producer and the receiver. In this meta-function attitudes are expressed towards what is being represented. The color red might be used to create a mood of warning or romance. Another example would be that the depicted person might be turned away from the viewer, which connotes absence of a sense of interaction. The textual meta-function is about a coherent whole, genres, and how parts are linked together. For example, in visual communication the color green could be used in a composition for the color of text headings to show that they are of the same order.
Simon is a third grader. According to his teacher, Simon finds schoolwork boring and he seldom participates in classroom activities. He has been assigned an assistant teacher. Simon’s tics are both verbal and physical. He easily gets angry and when he does he can become violent. Simon’s teacher finds it hard to design assignments or themes that Simon finds interesting even though Simon is creative and has many ideas, according to the teacher. He often does not complete his projects because he lacks patience. If something goes wrong and he is unsatisfied with the outcome he destroys what he has produced. One and a half semesters before the digital storytelling project he refused to read and also he hardly wrote. The teacher got some advice from the local support center suggesting she stress Simon’s creative capacity. The teacher writes “Both the assistant and I felt disconsolate since everything Simon produced, he destroyed, which was also a failure for the boy.” The following is the teacher’s story about Simon and his digital storytelling endeavors.
This spring the class worked with dinosaurs, making them out of papier mâché, which Simon enjoyed. He also liked to draw dinosaurs. One day, after taking a digital storytelling class [at the university], and discovering that the Photo Shop editing software was installed on all the computers in the school, I introduced digital storytelling into the classroom. I decided to try to make a dinosaur movie with Simon. Since he has no patience we kept a fast pace. We started by painting a background and then building a landscape with sand. We borrowed plastic dinosaurs and some plants from the kindergarten. Then we took photos with the digital camera. It turned out that Simon enjoyed taking photos very much. I thought it was enough with 20 to 25 pictures but I could not stop him so I let him continue. When Simon was done he had taken 60 to 70 snapshots. His creative capacity was highly visible in the many series of shots of the dinosaurs moving around and falling. We worked for 70 minutes straight.
At the next lesson we looked at the photos and decided which ones to pick. It was now time for him to get rid of some pictures but even then we had too many. Despite that we decided to use all the remaining pictures in the movie. Then Simon started to put the pictures in order. I knew that his idea was to have a fight between the dinosaurs, but in the process of ordering the pictures I realized that he had framed a more complex story. He was very precise and suddenly he became very patient ordering the pictures to make sure they fit together according to his plan. This took another 70 minutes.
The next phase was to record the sound. In the beginning of the work with the movie Simon had asked if he had to make a voice over. I interpreted this as his nervousness about this part because of his verbal tics. We decided that he could either tell the story orally or just have the sound effects and music. At the beginning he wanted to narrate but later he found it hard to remember what to say. Then he came up with the idea that I would write in the “remember-screen” and he would read. He, who refused to read, now asked for supporting text! The ability of the software to make short recordings helped Simon overcome his problem with tics. He prepared himself for short duration recordings by inducing his ticking behavior. When ready, he would say “now,” and then I pushed the record button. It’s important to note that Simon’s ticking had been severe during this period, and his speech could be unintelligible. But this is not noticeable in the movie.
Music and sound effects were also exciting to Simon. He was very careful about his choices and it was when we were listening to different kinds of music that he heard a piece that he immediately associated with the Second World War. “I will make a movie about the Second World War where I will use this piece,” he asserted and at the same time he asked a classmate to come and listen. He invited the boy to make the movie with him. From this moment on the dinosaur movie became less important. Instead Simon started to plan for his next film. He usually likes to show the class what he has produced, for example, paper airplanes. But this time he did not ask. He was totally absorbed by his new idea. Weeks later I asked him if he wanted to show the movie. He very much wanted to and was proud when he saw how impressed his classmates were.
The second movie was made together with a classmate. It was not always easy for the two to collaborate since Simon is strong willed and knows exactly what he wants. The friend did not have the courage to uphold his own opinion. Simon’s ideas were very good but the editing software didn’t support his creative ideas adequately. In the process of making the Second World War movie a third film idea was born—this time a spy thriller. This happened when Simon was searching for sound for the war movie. This time it seems that it will not be enough with still photos/that still photos will not suffice but that we shall have to work with video recordings as well. Whether we can let him make such a movie will of course depend on resources and the kind of staffing we have.
Simon continued to produce digital stories. He finished the Second World War story and the spy story successfully. He made a story about a black spider and he made several about his hobbies: biking, inline skating, and skate boarding. Below I analyze and discuss two: one features the dinosaur story and the other skating and biking. What and how does Simon tell in and with these two stories?
The film described in the teacher’s narrative featuring dinosaurs is 3.14 minutes long. As told in the narrative, Simon and his teacher created an environment in which to stage the story. The film contains 38 images, all photos taken by Simon. The film starts with a half-length picture of a dinosaur drinking water from a pond. A roar replaces the sound of the drinking animal. Simultaneously a voice says, “the country of the nasty valley” followed by a little laughter. Then the voice says, “once up on a time there was a very beautiful valley. They were always together.” The image is replaced through a transparent transition. Now it shows the trunk of a palm tree. A loud growling is audible and the voice says, “then there was a fight.” A caption on a sky like painted area reads: The country of the nasty valley. The voice continues:
- It was a dinosaur that hit another dinosaur.
- Then he dies.
- Suddenly a tyrannosaurus rex appears.
- He caught a triceratops baby.
- And then he killed it.
- The mom hit him so that he fell.
- Then he left.
- Then a styracosaurus showed up and hit the triceratops.
- Then the horn hit the triceratops’s belly.
- And then the triceratops dies.
- Then the tyrannosaurus returns.
- Suddenly there was a bomb burst with comets.
- They went panic-stricken.
- There were a lot of fights.
- Then another iguanodon arrived to the tyrannosaurus.
- They had a fight and the iguanodon hit the tyrannosaurus rex.
- Then he died.
- Then they lived happily ever after.
- End.
The photos are colorful and of different camera shots: close-ups, mid and long shot. They depict the animals drinking, fighting, and dying. Zooming and transitions give a sense of video, i.e., moving pictures. In between some of the utterances there is silence, leaving the images to carry on with the story. The last picture depicts two dinosaurs from behind, one little and one big, side-by-side. The voice is clear and firm without any signs of ticking. One can hear the gap in the recording and sounds from animals and bombing. The music is turned on and off.
At an ideational meta-functional level of a text one would ask who the subject is and what he or she does. In the story we learn about a valley where the dinosaurs used to live together (in peace?). Suddenly they start to fight and among others a baby dinosaur and his mom are being killed. They are also threatened and panic-stricken by raining meteors, which bombard the valley. Eventually the fights stop, life returns to normal and the dinosaurs live happily ever after.
As Sutton-Smith, et al. (1981) assert, younger children, from the ages of three to eight years, often tell stories about animals—wild and domestic—and monsters. The theme is often about a struggle between a subject and a more powerful force, for example a monster or a wild animal. The outcome of the struggle varies, often dependent on the child’s age, according to Sutton-Smith (Ibid). In Simon’s story the more powerful force wins after the subject’s (the mother) attempt to fight back. So, is this a story simply about dinosaurs and their lives or is it a story about human relationships, or even about Simon himself and his complicated relationship with the school?
The interpersonal meta-function describes the kind of interaction taking place between the participants in the communication displayed, that is, between the producer of a sign and the receiver of that sign. It is clear from the outset that Simon is the author and that he has a story to tell us, his audience.
Finally, the textual meta-function has to do with the composition of the final product: how parts are linked together and what genres they comprise. The Country of the Nasty Valley follows the traditional narrative structure with the conventional beginning of ‘once upon a time…,’ the middle part is filled with conflicts and tensions and ending with “and then they lived happily ever after” though sequenced through what Sutton-Smith et al. (1981, 7) call ‘chronicity’ performed through the conjunction “then.” His narrative is therefore a combination of what Sutton-Smith et al.(Ibid) call verse stories and plot stories where the former structure, according to Sutton-Smith et al. (1981, 13) “parallels major modes in the arts, in particular those of music and decoration.” From a perspective of creative text production this notion is intriguing.
Finally, the images, sound and music contribute to the narrative structure but it is the dialogue that creates the order and causality in the text.
The second digital story, which does not have a title, is about a boy inline-skating and biking. The video is 1.53 minutes long. It is built from 19 photos, two pieces of music, and a text caption at the end identifying the creators.
The video begins with an extended guitar chord from a hard rock song; simultaneously the camera zooms in on a door on the opposite side of a pedestrian bridge. One can assume that the door is to the school. We don’t go there – we stay outside in the schoolyard. The extended chord, lasting about four seconds (half of the time the picture is displayed) turns into intensive and heavy hard rock.
When the zooming ends the first photo is replaced with one of a boy approximately 10 years old. The boy is displayed in profile, in such a way that he appears to be floating or flying in the air. We can’t see his face, which is obscured by his hand. His legs are bent, almost at a 45-degree angle against his body. He is wearing inlines.
Several photos appear in which the boy performs diverse acrobatic motions. For example, in one he skates on top of four concrete boards, which are lined up next to each other. A new theme appears which is biking. We are in a forest but we can catch a glimpse of a house far away in the background. Simon’s house, one wonders? This is homeland. A light metallic sound replaces the hard rock music and turns into to a light flute and drum melody. The music is clear, calm, and rhythmic. A boy pops a wheelie with a mountain bike. In the next picture the level of difficulty has increased since the boy – still on one wheel – turns the front wheel sharply to the left. In the next to last picture the boy appears in the distance still on one wheel. His face reveals deep concentration – it looks like he is trying to move ahead on just one wheel. In the final picture we see the boy from behind on his bike, now bending forward with the back wheel in the air. This last picture has a caption: “the one that did the tricks is Eric, the one that took the pictures is Simon.”
Based on what is fore-grounded (the boy conducting tricks) versus what is back-grounded (school) (Machin, 2007) one could interpret this film as being about a boy who shows his skills in inline skating and biking but also about a boy’s relationship to school versus his sports-and-peer world. I any case, I interpret this film as a self-representation (Lundby, 2008), or call it ‘a report of one’s own personal experiences,’ as Sutton-Smith et al. (1981) calls these kinds of narratives. Moreover, Simon is showing us his coordination and skill in his sports, i.e., inline skating and biking. Throughout the story the body is at the center of attention. In most of the pictures the body is in heavy motion, often shown from an oblique angle, connoting energy and movement (Machin, 2007). The rider flies, jumps, bends and controls his vehicles with his body. We can read these poses as signs of strength and freedom (Ibid). The peaceful and harmonic second piece of music underlines this impression. In most of the pictures the body is exposed in close-up, drawing attention to the subject as an individual (Ibid). The head is seldom discernable and only a couple of images show the face. When we can see the face it shows concentration—we understand that Simon has agency and is powerful (Ibid).
Most of the time the audience see the boy from the side. He does not make eye contact nor does he watch us. The implication is, that Simon does not invite interaction: this is a monologue and he offers us information (Ibid). However, we usually see the boy at a neutral level, that is, he does not look either down or up on us. This displays an equal power relation with the audience (Ibid). Thus, even though (or because) Simon beckons us to watch him (Bäckström, 2004), we are considered ‘pals’ and equals.
As a piece about inline skating and biking, described angles and camera shots make perfect sense because this sport is all about posture. Bäckström (2004) has studied skateboard and snowboard cultures and explains that it is the esthetic expression, not the sensation in the ride, that is crucial in these sports. To pull a trick or fail creates strong emotions of happiness or anger. Moreover, it is important to have style, particularly a unique style, and that the performance be beautiful and ‘clean.’ Skateboarding and snowboarding are highly visual. It is important to see, make visible and to be seen. To ride together means to watch each other’s performance. In order to show others, the performance is often video recorded. Bäckström (2004) summarizes the features of skateboarding and snowboarding with terms such as liveliness, vitality, strength, freshness, speediness, freedom and action.
The story is a narrative built up with a calm yet persistent rhythm and carefully selected images and music. It can be described as a “descriptive digital story” (Nilsson, 2008), that is, a description often of a trip, friend, hobby, pet or place. These kinds of digital stories are usually not centered on a conflict and thus the point of view is neither stressed nor conspicuous yet conveys a message. Images in descriptive stories often serve to illustrate the spoken word. However, in Simon’s story images and music are the basic components and offer meaning separately as well as in combination. For example, the music creates a narrative structure (scenes and ‘chapters’) with levels of meanings. This is done by the choice of music and by how Simon uses it, e.g., where he inserts it and how he changes the volume level.
In applying a detailed reading and analysis of Simon’s digital stories one realizes the kind of work that lies behind them. The films are not randomly assembled images, music, speech, captions and sound. On the contrary, they are consciously, creatively, well-reasoned and well-crafted compositions. It is clear that Simon was immersed in a playful and creative process where he took on many roles: as scriptwriter, director, producer, photographer, actor, editor, etc. His digital stories represent different genres and offer multiple messages and meanings.
Through their content and form, Simon’s multimodal texts express and communicate thoughts and emotions (Elbow, 1981). An overall comment on Simon’s digital stories would be that they convey “fluency, rhythm, energy, and liveliness” which, according to Elbow (1981, 299), are signs of a text with voice. They do have the capacity to influence others. They are texts that, in Elbow’s words “want something.”
Simon is at an age where, according to Vygotsky’s (2004) scheme, he should utilize literary (script based) forms as his major form of creation. Literary forms enable a youth to express complex relationships and dynamics, according to Vygotsky. However, we learned from Simon’s teacher’s narrative that writing in a mono-modal manner (Kress, et al. 2001) was not possible for Simon and that reading and writing were obstacles rather than tools for him. But, when Simon was allowed to use a variety of semiotic modes of expression, particularly image and music, which are not usually valued as much as written language in the educational setting, he was able to create complex stories—complex digital multimodal texts—within the ‘host genre of narrative’ (Ochs & Capps, 2001). He created narratives both in the traditional sense, that is, based on language where meaning primarily was gained from the temporally-ordered structure. But he also created narratives through images and music where meaning was gained from the chosen and amalgamated images and their combination with music and sound. As a result, Simon composed digital stories shaped both by temporal and spatial structure. In both cases they were open to a variety of interpretations. Thus, we might ask: is Simon literate? If understanding literacy as a social and cultural activity where semiotic means of different kinds are used for producing texts in processes of expressing and creating meaning and communicating, then Simon is highly literate. But if literacy is limited to forming and decoding letters then Simon is not. Thus, the way we define literacy also defines children as competent or incompetent. I believe it is important to widen our perception of literacy to include new kinds of multimodal communication and thus texts which the new technology enables but also the diversity in creating meaning based on cultural and social differences (see, for example, Gee, 2008; New London Group, 1996). Hopefully that would imply a better understanding of the significance of giving children access to a variety of semiotic resources also in the educational setting. Such an approach to pedagogy would increase the opportunities for students to develop ‘voice.’ But a precondition is that adults become “new and multi-literacies” literate in order to communicate with those voices. As a result it might enhance children’s script based reading and writing. Simon is a good example of this.
While creating his digital stories, Simon and his teacher collaboratively used tools and developed strategies to overcome obstacles, e.g., the ‘remembering screen’ and the inducement of his ticking behavior. These tools and strategies gave Simon access to the spoken word and developed a need for, and turned written language into, a useful resource (Vygotsky, 1987). In addition, the activity created motivation for further exploration in that Simon imagined a film, requiring tools (video recorder) and resources (staff) that the school could not provide at the time but might be able to in the future. In this regard, digital storytelling created a zone of proximal development for Simon (Vygotsky, 1987).
Summing up, I believe that Simon embraced digital storytelling because it enabled him to create stories about what interested him and about issues, which he had experienced and had knowledge about. He was able to create these stories because he was not limited to spoken and written language as the only semiotic modes but had access to multiple modes of expression. He was given the chance to communicate his experiences, skills, knowledge, interests and views in ways suitable to him.
This case is a promising example of new pedagogical activities featuring multimodality and utilizing multimedia in education aiming at facilitating students’ creativity generally and creative literacy and literary development in particular. The ability to use multiple semiotic modes was made possible by digital tools such as the digital camera, editing software, and Internet connections. Does this mean that digital media offer new ways to be creative in the literacy and literary domain?
Creativity, understood from a sociocultural perspective, that is, as a socio-cultural activity, implies the significance of technical abilities, tools, and traditions (Vygotsky, 2004). Thavenius (1995) stresses the significance that modern media, e.g. video and computers, play for aesthetic and creative production in schools. He claims that different kinds of software enable creativity, such as re-creation and use of images (still and video), music, sound, speech, and writing. From an educational perspective it is this creative potential inherent in new media that we have to embrace. Digital storytelling is just one example of how it can be employed.
Bruner, J. (1986). Actual Minds, Possible Worlds. Cambridge, Massachusetts and London: Harvard University Press.
Bäckström, Å. (2004). Liberty, youth, fellowship: Stereotypes within skateboard- and snowboard culture. Published on Internet, www.idrottsforum.org (ISSN 1652–7224) 2004-12-14.
Cole, M. (1996). Cultural Psychology – A Once and Future Discipline. Harvard University Press.
Elbow, P. (1981). Writing With Power. Techniques for Mastering the Writing Process. New York/Oxford: Oxford University Press.
Engel, S. (1995). The stories children tell. Making sense of the narratives of childhood. New York: Guilford Press.
Gee, P. (2008). Social Linguistics and Literacies – Ideology in Discourses. London and New York: Routledge.
Hakkarainen, P. (2006). Learning and Development in Play. In J. Einarsdottir & J. T. Wagner (Eds.), Nordic Childhoods and Early Education (pp. 183 - 222). Greenwich, Connecticut: Information Age Publishing, Inc.
Hull, G., & Nelson, M., E. (2005). Locating the Semiotic Power of Multimodality. Written communication, 22(2), 224-261.
Klerfeldt, A. (2002). Sagor i ny skepnad – barn berättar med dator. In Säljö, R. & Linderoth, J. (Eds.), Utmaningar och e-frestelser. Stockholm: Prisma.
Kress, G. (2003). Literacy in the Media Age. London: Routledge.
Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal Teaching and Learning: The Rhetorics of the Science Classroom. London and New York. Continuum.
Kress, G., & van Leeuwen, T. (1996). Reading Images: The Grammar of Visual Design.
Lambert, J. (2002). Digital Storytelling. Digital Diner Press.
NewLondonGroup (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66, 60-92.
Lundby, K. (2008). Digital Storytelling, Mediatized Stories: Self-representations in New Media. New York: Peter Lang Publishing AB.
Machin, D. (2007). Introduction to Multimodal Analysis. London: Hodder Arnold.
Nilsson, M. (2008). Digital Storytelling: A Multidimensional Tool in Education. In T. Hansson (Ed.), Handbook of Digital Information Technologies: Innovations, Methods and Ethical Issues. Idea Group Publishing.
Ochs, E. & Capps, L. (2001) Living narrative. Creating lives in everyday storytelling. Cambridge: Harvard University Press.
Renberg, B. (2006). Rytm och rap och retorik – om svenska, språk och estetik. In Eva Alerby & Jórunn Elidóttir (Eds.) Lärandets konst – betraktelser av estetiska dimensioner i lärandet. Lund: Studentlitteratur.
Sutton-Smith, B. (1981). The folkstories of children. Philadelpia: University of Pennsylvania Press.
Säljö, R. (2005). Lärande och kulturella redskap. Om lärprocesser och det kollektiva
minnet. Stockholm: Norstedts Akademiska Förlag.
Thavenius, J. (1995). Den motsägelsefulla bildningen. Stockholm: Östlings bokförl. Symposion.
Vygotsky, Lev S. (1978). Mind in Society. Harvard University Press.
Vygotsky, L. S. (2004). Imagination and Creativity in Childhood. Journal of Russian and East European Psychology, 42(1), 7-97.
All names are fictional in order to protect the human subjects. I want to thank Simon and his parents as well as his teacher for permission to use the material analyzed in this paper. Special thanks to Simon’s teacher for kind and valuable help with files and information.
Tourette’s syndrome is a severe neurological disorder characterized by facial and body tics, often accompanied by grunts and compulsive utterances (The American Heritage College Dictionary).
See, Center for Digital Storytelling: http://www.storycenter.org/index1.html
The remember screen is a small window in the editing program where you type in what you are going to record in the voice over.
The boy in the photos is Simon’s friend Eric. Simon’s teacher says that Simon is good at the tricks but that his friend “Eric” is slightly better. Simon is good at and interested in pictures and therefore he wanted to take the photos. His teacher says that Simon has many ideas about angles etc.