ISSN: 2535-4051

Vol 8, No 4 (2024)

https://doi.org/10.7577/njcie.5743

Article

Assessing Computational Literacy in First Language (L1) Teaching

Roland Hachmann

University College South Denmark

Email: rhac@ucsyd.dk

 

Marie Falkesgaard Slot

University College Copenhagen

Email: mafa@kp.dk

Abstract

This article addresses the multifaceted challenges of assessing computational literacy (CL) within the context of first language (L1) teaching. The article commences by examining the theoretical foundations and the need for an expanded understanding of CL, moving beyond the popular notion of computational thinking (CT). We then present an example from a 5^th-grade classroom in a Danish middle school to illustrate how pupils integrate computational skills into their writing practices through a design-based intervention. Subsequent to this example, we propose a cross-disciplinary framework designed to assess CL in L1 settings, focusing on four principles that bridge traditional language arts with computational approaches. Finally, we reflect on how the application of these principles can help formulate new learning goals that better align with the emerging demands of 21st-century education. Throughout the article, we argue that a CL approach provides a more socially rooted and context-sensitive method for integrating computational methods into non-computer science subjects, offering theoretical clarity and practical benefits for both educators and researchers alike. The article commences with a discussion of the CL approach related to assessment.

Keywords: Computational Literacy, Assessment, Teaching, K9, Language Arts

Introduction: From computational thinking towards computational literacy

The term ‘computational literacy’ (CL) was introduced by diSessa in his book Changing Minds (2001). The basic idea of a CL approach is to move away from using computers merely for skill-based tasks and towards developing a literacy that supports thinking and expression of creative ideas in situated contexts mediated by surrounding technologies. As a former colleague of Seymour Papert, who coined the term ‘computational thinking’ (CT) in 1980 but did not define it further, diSessa continues Papert´s original idea that the computer is a powerful tool for new approaches to learning in the existing school system. In his work, Papert saw the computer as a tool to master ‘powerful ideas’, a tool capable of ‘influencing how people think, even when they are far removed from physical contact with a computer’ (Papert, 1980, p. 4). In this sense, both Papert and diSessa perceive the role of computational technology in schools as a way to engage pupils socially and affectively in interdisciplinary learning processes.

In her widely cited 2006 paper, Jeannette Wing revitalised the concept of CT, orienting it toward problem-solving, the emphasis being on computer science. Later, Wing and her colleagues refined her original definition of CT as ‘the thought processes involved in formulating problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent’ (Wing, 2010). Wing and colleagues' revitalisation of CT emphasises a set of general cognitive skills and problem-solving strategies from computer science, including decomposition, pattern recognition, algorithm design, and pattern generalisation (Denning & Tedre, 2021; Grover, 2022; Wing, 2010). However, as opposed to Papert and diSessa, Wing defines CT more narrowly, as a specific scientific mode of thinking that she claims could and should be applied across all subjects.

Presently, CT continues to be the predominant term in educational policy and is often framed as a 21st-century skill (Cutumisu et al., 2019; Grover & Pea, 2013). How CT can be integrated into school subjects is a topic of debate, and researchers are beginning to call for broader definitions that move away from the focus on generic cognitive skills towards a situated, participatory and critical perspective (Bers, 2021; Kafai & Proctor, 2021; Vee, 2017). In line with these recent developments, we also challenge the idea that CT can be easily transferred and applied across different subjects and problem-solving situations in schools.

Not all disciplinary problem spaces and their solutions are legitimate areas of investigation within school subjects (Hachmann, 2022). Both problems and their solutions are based on and evaluated through the subject and the related disciplines. School subjects derive from natural, social and humanistic science disciplines and traditions that determine what kind of content, methods and teaching approaches are selected and recognised as important. A school subject's content, methods and goals result from an extensive and ongoing dialogue between school teachers, teacher educators, researchers and curriculum developers. In this sense, school subjects are reifications of what is considered valuable within these contexts of development and amongst practitioners (Dolin & Qvortrup, 2013). For this reason, CT should be rejected as a generic set of skills that can be applied and transferred to fit all subjects (Hansen et al., 2023). CT should rather be seen as context-dependent and integrated into the specific subject's existing methods and traditions. In other words, instead of pushing a computer science template onto an existing subject, more consideration should be given to aligning computational approaches with the specific subject.

Computational Literacy – cognitive, social and material aspects

DiSessa’s approach entails seeing CL in relation to the complex system of niches that the subjects consist of (diSessa, 2001, p. 24). From this ecological view, writing, rhetoric, and literature studies are social niches, and CL should be embedded in the practices of those niches as they are specifically linked to the subject area. This suggests that pupils’ development of CL is influenced by the unique social and disciplinary structures of each subject, allowing them to engage with and apply computational concepts in ways that are meaningful within those specific domains. Within these niches, CL becomes ‘a socially widespread patterned deployment of skills and capabilities in a context of material support (...) to achieve a valued intellectual goal’ (diSessa, 2001, p. 19). The CL approach emphasises a move from an instrumental and individual approach to computer use in schools towards a social and explorative way of engaging with powerful ideas in the world. Instead of focusing solely on how to code or build robots, CL should be seen as a way to offer opportunities for expression and engagement with the subject matter that involve computational media.

DiSessa (2001) argues that CL builds on three fundamental pillars (we prefer to call them aspects, as they are intertwined): cognitive, social and material. In brief, the cognitive aspect deals with how external representations can support particular ways of thinking and understanding, nurturing new ideas and creativity. The cognitive aspect of CL draws on the notion that the things we use and interact with are connected to how we think and perceive the world around us. This should not be understood as deterministic. It is just another way of saying that tools mediate our thinking in various ways. An important point here is that if the tool changes, the problem will be approached differently or even perceived in new or different ways. In other words, tools enable thinking and action, but they can also limit us by channelling our focus on specific possibilities, thereby preventing us from seeing others (Brennan, 2015).

In conjunction with the cognitive pillar, the material aspect refers to the awareness of the affordances of physical and non-physical tools surrounding us. The material aspect of CL focuses on externalised representations, including symbols and signs, in which humans can capture aspects of our thinking and social practices in a way that makes them reproducible, transportable, and manipulable. In other words, the material aspect of CL involves materialising parts of our thinking and experiences so that they are made concrete and given material form. In this way thoughts and experiences can be retained, transformed, shared, and negotiated in relation to the communicative practices in which we participate at different times (diSessa, 2001). The material aspect of CL thus encompasses the idea that the things we interact with consist of specific symbolic representations and have specific possibilities for use, interpretive frameworks, conventions, and rules.

The social aspect involves the sensitivity and situatedness of CL in a sociocultural and historical context characterised by specific norms and values. CL is always directed towards something, and as suggested by Vee (2017), what it is will always be an open question that can only be answered provisionally. Therefore, contrary to the basic idea of CT as a general mode of thinking, the literacy approach highlights the sociocultural and historical context in which one finds oneself, a social niche. These social niches allow setting external boundaries for what CL is in school subjects. CL is not just about being able to read or write on a computer but about specific practices and activities involving different types of computers or software.

CL enables pupils to explore the world and themselves in it in new ways through the use of computational media. In this light, CL distinguishes itself from CT by going beyond the idea of being the thought process of formulating problems that computers will then solve. Instead, CL involves a more holistic approach of different models of thought, material aspects, and practices of creativity and skills related to the specific subject matter and activities.

The challenge of assessing computational literacy in a Danish L1 context

In Denmark, computer science is not a specific subject in school. However, recent developments and projects on a national scale have been conducted to experiment with makerspaces, programming, and computer science in school settings. Current educational reforms are being initiated with the goal of integrating aspects of computer science, informatics, and design into the existing subjects. There is also an effort to implement a new elective subject in K7-9 based on the development and testing of the preliminary subject called Technology Comprehension later reframed internationally as Computational Empowerment by Dindler et al. (2020; 2022). Although several projects in Denmark have focused on integrating CT into first language teaching (L1), little progress has been made in developing a framework for evaluating pupils’ progress and development. To this date, the policy documents describing the goals and aims of the L1 subject lack the description and criteria of how skills and competencies related to using computational media can be linked to learning and teaching practices. Specifically, in L1, it remains unclear as to how the concept of computation is analytically and productively related to the concept of text, a metaphor for describing, modelling and thinking about the world through the subject (Ministry of Children and Education, 2019; Slot et al., 2021). As remarked by Caeli and Bundsgaard (2019), many European countries have recently launched reforms that include the integration of CT into compulsory education, and more are planning to do the same. Further, Caeli and colleagues point to the fact that previous attempts were made in the late 1960s and early 1970s to implement computer science into Danish schools (K1-9) with a focus on both teaching programming skills and, on a more ethical level, the impact of computers on society (Caeli & Yadav, 2019). These attempts had little success and have now been replaced by a new ‘media literacy’ perspective that integrates the use of ICT as a functional tool (Christensen & Tufte, 2010).

Contemporary research in Denmark points out how researchers, curriculum designers and educators attempted to develop different approaches to implementing CT in existing school subjects (Riis et al., 2021) and as a new, separate subject across the educational system (Buus et al., 2023). Unlike other Nordic countries, Denmark has made no attempts to develop interdisciplinary approaches to implementing and assessing CL skills across the humanities, social sciences, and natural sciences (Berthelsen, 2023; Bocconi et al., 2018; Niemelä et al., 2022; Rajapakse-Mohottige et al., 2024; Tang et al., 2020). The Danish educational community is actively discussing how computational skills relate to specific school subjects, how integration between existing subjects can be carried out, how these new approaches can be implemented and to what end. For instance, researchers focusing specifically on L1 are acutely aware of the challenges in aligning aesthetic production with CT (Slot et al., 2021; Slot et al., 2023).

It is also well-documented that the increased use of computers and digital media influences reading and writing skills (Mills, 2015; Vee, 2017). However, new genres and text features have found their way into language subjects such as micro-blogs, podcasts, digital stories, memes, and AI-produced content. These new linguistic and visual codes have prompted textual theorists to examine shifts in meaning-making (Mills, 2015). Identifying what role, e.g. ‘visual codes'’ should play in L1, therefore, is undergoing several ideological and pragmatic discourses. The discussions revolve around questions about what best contributes to meaning making, including which representations best support core content in the subject. The existing guidelines for L1 describe how various representations are considered relevant to teaching the subject. However, there is no mention of how CL practices can support pupils in composing and writing or creating programmable digital artefacts (Slot et al., 2023). Based on these historical developments, the effort to integrate computational practices in existing school subjects such as L1 remains at an early stage, and the aim of this article is to contribute to the area of assessment by highlighting how integration of traditional language arts with computational approaches changes assessment criteria.

Towards a cross-disciplinary framework for assessing computational literacy in L1

Danish national criteria for evaluating school subjects are linked to international standards for high-quality assessment (AERA, 2014). Assessments of L1 in schools are based primarily on formative assessments conducted throughout K1-8. In 9th grade, pupils' abilities and literacy skills are cumulatively tested through two fixed examinations carried out on a national scale, one oral and the other written. The written test is further divided into two subtests: a test of reading and spelling skills and a second test of written literacy in a 3.5-hour examination. Here, pupils are asked to choose one of four possible tasks in the examination material and to write a genre-based text.

Evaluation and assessment provide an opportunity to obtain knowledge about how pupils learn and develop in subjects. Typically, there will be a clear understanding underlying the evaluation of competencies in a school subject, framed by criteria such as what the student should learn, which tasks are best suited for this purpose, what characterises the learning objectives and activities, and notably, a clear notion of adaptability, i.e., why time has been allocated to specific content or methods. Evaluation is understood quite differently in the Anglo-Saxon versus continental educational approach. In the Nordic countries, evaluation has traditionally focused on academic transcendence (Biesta, 2013) and depth of subject knowledge, while lower priority has been given to focus on factual knowledge and skills (Graf, 2017). In the past 10-15 years, evaluation in the Nordic countries has been increasingly associated with visible learning and taxonomic thinking, as Biggs and Tang (2007) exemplified. This has led to increased grading and monitoring approaches and numerous precisely formulated learning objectives in the curriculum. In recent years, assessment constructs have once again adopted a distinct formative approach, as illustrated in the Quality Chain Model [Kvalitetslenka] (Jølle & Skar, 2021, p. 297), which we will further describe below. We use this model to illustrate how establishing formative evaluation of emergent interdisciplinary areas in school subjects, such as CL in language one, could be carried out. In the Quality Chain Model, Jølle and Skar combine the assessment of writing and writing processes based on international standards for high-quality assessment (AERA, 2014; Cizek, 2019). Jølle and Skar (2021) list six generic criteria that can be applied across all subjects in school. The criteria are understood as both a reflection on the teacher's behalf when planning lessons but also as a point of departure for dialogue in the classroom with the pupils. The six criteria are as follows:

1.  Definition of learning objectives and aims and how they relate to the curriculum.

2.  Relevant and meaningful tasks and how they relate to the objectives and aims.

3.  Clear criteria for how pupils are expected to solve tasks and how they align with learning aims and objectives.

4.  Sufficient and long-term basis for assessment that goes beyond single tests or a few episodes of assessment.

5.  Opportunities for learning and showing depth of skills and competencies.

6.  Positive consequences for learning and how assessments can motivate pupils and foster self-efficacy.

Due to its generic nature, the Quality Chain Model can provide a basis for developing criteria for assessing CL in L1. Our contribution to an interdisciplinary assessment framework builds on and condenses the six criteria into four principles, all of which are formative. Instead of focusing primarily on pupils' acquisition of knowledge and skills based on learning goals describing what they must learn, we connect the quality of the tasks given, the materials provided to solve them, and the processes involved. Further, the four principles are based on the idea of integrating subject skills relevant to first-language subjects and CL.

Principles for the assessment of computational literacy in first language teaching

Although the Danish L1 curriculum mentions opportunities for expressive writing practices, specific approaches such as creative storytelling and aesthetics, language models, programming, and the creation of computational media are not mentioned. Despite this lack of attention at the curricular level, schools are already embedding computational practices across various subjects using maker technology, robots, microcomputers, and Chatbots. Based on our research and findings, we have tried to create a set of principles for developing assessment criteria. As Brennan and Resnick (2012) argued, assessments should involve creating and critically examining computational artefacts in a way that illuminates the design process, supports further learning and values multiple ways of knowing and applying knowledge. In this sense, the significance of technology use in classrooms lies not in what a device or text 'is' nor what it does explicitly. Its significance lies more in what it enables, in so far as technology mediates the relationship between the user and other individuals (Edwards-Groves, 2012). Technology, in this sense, is a social tool. From our perspective, assessing CL in L1 classrooms calls for more formative approaches in alignment with and expanding the cumulative part. Again, the Danish L1 curriculum contains no such assessment criteria. Our task here is to explore how such assessment criteria might be addressed and formulated. The four principles for developing assessment criteria regarding CL in L1 are an attempt to show how CL supports a material intelligence (diSessa, 2001); the principles thus highlight new ways to be creative, express ideas and engage in subject matter not included (or even foreseen) in the existing curriculum. We stress that teaching and assessing are dialogical processes. They influence each other such that what is taught should be aligned with the assessment criteria and vice versa, reflecting what is meaningful in classroom practices. Therefore, the following four principles should be seen from this dialogical perspective.

Principle 1: Material and expression

The assessment of CL in L1 teaching should consider how pupils' textual meaning-making processes are supported through computational materialities and the context in which material expressiveness contributes to their analytical and productive work. Further, the assessment should address how pupils evaluate the affordances of the material support and how the use constrains or creates opportunities for completing the task.

This principle emphasises how pupils are given opportunities to creatively express and transform ideas and knowledge about themselves and the world surrounding them. Teaching and assessing CL require new approaches for evaluating and experimenting with the qualities of computational materialities. Material expressiveness also captures the idea of CL as a skill to analyse and create computational artefacts. In the empirical example, pupils were engaged in reflective dialogue with the material (Schön, 1992). They utilised computational tools for planning and writing, developed computational artefacts, and analysed and gave feedback to their peers in the classroom. The material support of the computational media was important for how these processes evolved and played out in class. The assessment of CL must, therefore, provide the opportunity for reflections on which materials are used and how these create constraints and opportunities for the pupils to express themselves. At the same time, it is important to be aware of how the pupils recognise the qualities of the material support at hand.

Principle 2: Collaborative participant structures

The assessment of CL in L1 teaching should consider how collaboration, negotiation, and sharing of ideas are part of computational practice.

This principle emphasises how pupils work on computational artefacts or apply computational methods to solve subject-specific challenges. Valente and Marchetti (2020) conclude that learners tend to act individually or one at a time when asked to engage in shared activities on computers. Here, we especially want to emphasise the collaborative processes involved in CL as a socially rooted phenomenon. Overly simplistic or routine assessment tasks do not require pupils to collaborate; they do not need to negotiate creative ideas or pool resources. The task should enable group discussion and challenge their thinking.

In the empirical example below, the task of creating the body of a story was both open-ended and yet prompted by the fictional text and the wooden pieces (described in the example below), somewhat constraining the Twine stories. Twine is an open-source web-based application designed to create nonlinear narratives and will be described further below. However, the pupils found the opportunity to discuss and exchange ideas valuable. For instance, it was observed how encouragement and continuous exchange of feedback among the pupils helped them develop ideas for the story and negotiate how to use different visual effects for expression. One group was observed utilising with the wooden pieces as a way to develop their narrative.

These observations highlight the importance of establishing open-ended tasks and monitoring how the pupils participate in the computational practices (Kafai et al., 2020) that come to life in the classroom.

Principle 3: Computational thinking and methods

The assessment of CL in L1 teaching should consider how different modes of thinking are connected to how pupils apply methods to process subject matter.

This principle emphasises the thought processes and methods involved in creating computational artefacts. As noted earlier, we regard CT as a component of CL, more precisely its cognitive aspect. To achieve CL, one must understand the world through specific modelling and apply relevant methods to solve the challenges set in the subject. Achieving this level of understanding involves thinking about the world in different modes, such as abstraction, decomposition, pattern recognition, and algorithmic thinking (Grover, 2022; Wing, 2006). In the empirical example below, these modes of thinking emerge through the use of the tools or structuring resources. In our study, for instance, we observed how the pupils arranged and re-arranged the pieces and how they built patterns between story elements before transforming them into digital representations in Twine (see Figure 2). An analysis of the digital artefacts (Twine stories) shows that, in many cases, the arrangement of the passages aligns with the wooden pieces observed in the classroom. It is also observed (e.g. on the whiteboard in Figure 2) how pupils decompose the plotline of their story into smaller pieces by creating logical coherence between the choices made by the reader (by clicking on hyperlinks), and this choice leads and is connected to the overall narrative of the story. In sum, the engagement with the provided tools points to the importance of the cognitive aspects of CL and how external representation can support or constrain how pupils think and approach writing.

Principle 4: Computational practices

The assessment of CL in L1 teaching should consider how computational media and the specific computational and communicative practices used empower pupils' communication abilities. 

This principle explores how pupils can be given opportunities to communicate powerful ideas and express themselves through expanding traditional writing practices by means of coding and computational media. In L1, pupils are assessed according to their ability to communicate appropriately and gauge the communicative situation. With the focus on pupils' ability to take part in a rich digital text culture supported by computational texts, it is important to teach and evaluate how this engagement provides cultural and expressive empowerment. This engagement further requires an additional assessment element:  the ability to combine or transform written text (natural language) into coding (computing) language.

In the empirical example below, pupils move from oral language to the semantics of the wooden pieces, writings on a whiteboard, and coding in Twine (as shown in Figure 2). The communicative practices are entangled, with the pupils moving back and forth between different modes of expression. The pupils are encouraged to use their ability to communicate in a variety of modes, in interaction with each other and the supportive materials.

Example: Writing interactive stories in Twine

In the following, we provide further description of the example used in the development of the principles above and the framework presented below. The example builds upon three small-scale design experiments (Edelson, 2002) conducted in three Danish public school classes. However, here we use only an excerpt to illustrate the main features of our suggested framework. For a more in-depth presentation, see Hachmann (2024a; 2024b)

The study we draw on here involves a pedagogical design (Figure 1) in which pupils aged 10-12 (grades 4 and 5) were introduced to the beginning of a fictional text, The Horrible Hand (1). The introduction of the story was read aloud in class as a starting point, after which the pupils continued developing their own stories using Twine.

Twine is an open-source web-based application designed to create nonlinear narratives. Users start by creating passages which represent individual chunks of text. These passages can then be linked through hyperlinks, creating and branching pathways within the narrative. Twine employs a simple scripting language, often based on HTML, to allow for more advanced customisation. Users can incorporate variables and conditions to create dynamic storytelling experiences. In addition, Twine provides built-in functionality for variables, macros, and stylesheets that can further enhance the interactive elements of the story. Once the narrative structure is established, users can preview and playtest their creation directly within the Twine interface. The finished work can be published as a stand-alone HTML file, or it can be hosted online for others to experience.

To further support the pupils in the task, 19 wooden pieces (2) with different motives engraved into them were developed and produced. On the back of the five large pieces, the pupils could access small excerpts of the original story (via a QR code to a homepage). The pupils utilised the wooden pieces as a structuring resource both creatively in their idea-generation process and as part of working computationally with the composition of their story. Collaboratively writing their stories in Twine (3) involved peer feedback (4) from other pupils before rewriting (3) and completing their stories (5), which acted as the cumulative assessment task.

Figure 1. Graphical display of the pedagogical design

Tangible tools to support computational practices

The wooden pieces act as an unplugged structuring tool that can support the pupils in generating ideas and composing and arranging story elements (Principle 1). The tangibility of the tool was essential to how the pupils' computational practices played out as an embodied and situated learning activity (Principle 4). The material support of the tool allowed the pupils to engage in dialogue and actively involve themselves in learner-learner and learner-material interaction as part of the creative process of developing a story. Further, the tools resembled Twine's logical structure in that the design of pieces allowed the pupils to work sequentially with ideas and with the elements of their story. In this perspective, pupils' CL was supported by:

●       Allowing pupils to engage in reflective conversation with the material to support their development and organising of story elements in sequential patterns.

●       The external representation and cognitive support (distribution of memory) provided when the pieces were used to remember the composition and content of their stories.

●       The use of computational media, offering new modes of expression.

Twine as an object-to-think-with in L1 settings

Producing stories in Twine is different from more traditional writing practices in schools. As an object-to-think-with (Papert, 1980) Twine affords a communicative situation that involves hyper texting and symbol manipulation, drawing on HTML coding and using natural language and genre-specific elements. In this way, Twine stories draw on literacies from both L1 and computer science. Twine stories have three essential features:

●       They are based on a specific text genre (interactive story), that moves from the first- or third-person’s perspective to a second person's perspective.

●       They are composed of step-by-step sequences that allow the reader to make choices and move forth and back in the narrative, a contrast to the fixed linear progression of conventional narratives.

●       They allow for multimodal expressions by giving opportunities to embed HTML coding directly into every element or sequence of the story, e.g. input of sound, interactive elements, graphics, etc.

Based on these characteristics, we argue that writing interactive stories in Twine involves literacies and methods related to coding, writing, and composing stories, the latter two being essential to language arts.

Designing for computational literacy

The fundamental idea behind the pedagogical design was that pupils would develop stronger textual and CL skills through the use of tangible objects (Principle 1) and collaborative story writing in Twine (Principle 2).

The tangible materiality of the wooden pieces constrains the pupils' construction of the story to step-by-step sequences, allowing them to decompose their ideas into smaller parts and arrange and re-arrange elements of their story meaningfully (Principle 3).

Further, the pupils' Twine stories show that some used the excerpts provided through the QR codes and that elements from these texts functioned as bridges between plot points of the pupils' own stories. In other words, the pupils abstracted information and ideas from the excerpts and then filled in the blanks to create their narratives. This interaction likewise suggests a pattern recognition approach: the pupils had to find plausible and logical relations between the information provided by the excerpts and combine them with their own ideas. Lastly, the transduction process (Kress & Selander, 2012) between the wooden pieces and Twine is of the essence. As part of the computational practices (Principle 4), the pupils shift between different ways of data representation through symbol manipulations. For instance, as shown in Figure 2, transduction involves moving from oral language (1) to the semantics of the wooden pieces (2), to writings on a whiteboard (3) to coding in Twine (4).

Figure 2. Photo from the classroom showing pupils negotiating and creating their narratives (Principle 4)

The pieces afford structure and visual representation, and their tangibility provides opportunities for discussions and for the re-arranging of story elements. The pupils elaborate on these elements by writing on the whiteboard before their ideas are then translated into computer code. The example shows that through the process, the pupils draw on literacies from L1, while also demonstrating CL and modelling skills.

Merging assessment goals computational literacy

Based on the empirical example, we have (below) visualised how the integration of CL in L1 was addressed and formulated as assessable learning goals (figure 3). The L1 goals were taken directly from the national guidelines published by the Ministry of Children and Education (2019) called Fælles Mål Dansk [Common Learning Goals for Danish]. The official goals were formulated to orient teachers as to which subject knowledge and competencies pupils should be expected to have within the subject at the end of the 9th grade. The goals are also subdivided into sections highlighting which goals are to be reached at the end of the 3rd, 6th, and 9th grades, respectively.

 The CL goals in figure 3 were formulated on the background of related literature and already existing assessment frameworks from the field of computer science (e.g. Brennan & Resnick, 2012; Delcker & Ifenthaler, 2017; Snow et al., 2017; Weintrop et al., 2021). Further, when formulating the goals, suggestions were already established for learning goals in the experimental subject of technology comprehension. These were taken into consideration and informed the development of our final learning goals in the framework.

Figure 3. Chart showing how assessment goals from L1 are informed by CL goals and merge into cross-disciplinary assessment goals

The basic idea here is to show how applying CL goals to already existing learning goals in the L1 subject has a transformative effect (indicated by the blue arrows) on the why, what and how writing is taught. Consequently, the goals transform into what we label cross-disciplinary goals, underpinning that both disciplines (with L1 as the primary offset) are adding to the final set of assessment goals as they merge.

Figure 3 reflects our preliminary work and has its limits. It highlights cumulative assessment goals, but it does not provide any guidelines for assessing the processes in the classroom from a more formative perspective. The lack of guidance is partly due to the cumulative nature of the L1 goals and CL goals, respectively. We need to ask whether the pupils are learning to become computationally literate or whether they are learning something else by applying CL skills. Another question that needs to be addressed from a CL perspective is how the pupils evaluate and recognise the opportunities and constraints afforded by the material. Lastly, the framework does not evaluate how new writing identities (Krogh & Piekut, 2015) on the pupils' behalf are shaped through new modes of expression and whether (or how) these allow for new ways to participate in writing practices in L1. The questions raised do not capture the full complexity of assessing pupils' learning outcomes. However, they need to be considered when developing assessment criteria, as they will frame the assessment task, and the kind of data needed for evaluation.

Concluding remarks and next steps

In this paper, we have endeavoured to set forth some guiding principles for assessing computer literacy in school subject matter, with an example taken from L1. According to UNESCO, literacy is a means of identification, understanding, interpretation, creation, and communication in an increasingly digital, text-mediated, information-rich and fast-changing world. Literacy is a continuum of learning and proficiency in reading, writing and using numbers (UNESCO, 2023).

In this article, we have discussed several challenges to assess CL. The most significant of these challenges is the need for more holistic assessment criteria, considering that literacy is not only about hard skills but also about engaging in collaborative, creative processes and critical thinking. We tried to address some of these perspectives by proposing four principles that could effectively guide computer literacy assessment.

That said, we also need to consider how to assess skills in a way that reflects a progression in the subject. We still must consider what pupils need to learn from subjects. Taking a literacy perspective, as opposed to the more popularised CT approach, entails the need to incorporate the powerful ideas of different learning domains within specific subjects instead of applying a generalised template to all subjects. In our example, writing stories in L1, the decomposing of texts (also called deconstructing) would be different from tasks requiring decompositions in mathematics or computer science. Therefore, a pedagogical framework is required that can provide meaningful assessment criteria that align with the aims and learning goals of the specific subject. It is a two-way process where both assessment and the way a subject is taught inform each other, making each other aware of what central questions are operating, how they operate, for whom and why. This contextual, interactional relationship between computation and subject complicates the effort to establish assessment criteria for CL. In the Danish context, we are only beginning to ask the key questions and to realise the nuances and varied relations between computation and school subject.

We posit that a focus on CL instead of a more general thinking framework can help pupils engage analytically and productively in new modes of expression. Returning to diSessa's argument, if CL is to enter the classroom in an efficient way, it must be infrastructural and in line with other literacies (diSessa, 2001). By ‘efficient’ we mean that pupils will be using CL skills all the time, just as they use specialised ways of reading and writing skills across subjects.

If CL is to become a part of the school curriculum, it must not just be implemented, but fully integrated. As diSessa shows in his work on CL in physics (2001) and mathematics, integrating CL into the subjects is a long-term endeavour that changes how we think about subjects, their content, their goals and how to assess progression in learning. Any form of literacy needs literature (diSessa, 2018) and both the above definition from UNESCO and the recently published report by the U.S. Government (2023) mark shifts in the landscape of how we organise our intellectual terrain and the importance of CL in, across and beyond school and education. New ways to think about the ‘big ideas’, subject-specific content, curriculum and assessments have become relevant considering developments in artificial intelligence, automation and data security. The US report focuses on STEM education and how computational and communication technologies are rapidly evolving, creating challenges to traditional instructional methods. The report concludes that it is necessary to emphasise the importance of developing content that responds to current and emerging advanced technologies. Further, professional development around CL is crucial in order to take advantage of educational technology for learning, work, and everyday life (U.S. Government, 2023, p. 29).

However, focusing on CL in STEM education alone is insufficient. The idea of CL as a way to enhance pupils' skills and opportunities to express their knowledge through the use of technology and pave new ways for collaboration, problem-solving, and expression is valuable precisely because it is not limited to a single group of subjects. On the contrary, a CL approach can also be applied to all non-computer science subjects, not as a general applied skillset but in alignment with the subjects and their social niches. L1 education focuses not simply on learning to code or using technology as an add-on skill. Instead, CL can be aligned with the core goals of L1, such as reading, writing, and critical analysis.

In conclusion this means integrating CL into how students structure texts, explore narrative forms, or even analyse linguistic patterns. This requires assessments that measure both traditional language skills and how effectively students use computational tools to augment their understanding and expression of language. Assessment should reflect this integration by focusing on how computational tools and thinking contribute to the development of language and literacy skills.

In Nordic countries such as Sweden and Norway, CT has been integrated into national curricula through various approaches that are often related to digital skills, problem-solving, and programming. In 2017, Sweden significantly updated its national curriculum to incorporate digital competence and programming into various subjects (Heintz et al., 2017). Norway’s curriculum update, Kunnskapsløftet 2020 [Knowledge Promotion 2020], was fully implemented in 2020 (Norwegian Directorate for Education and Training, 2020). This reform provided a broader foundation for future competencies, including CT. Mifsud et al. (2023) examine how CT is represented in educational policy documents in Norway, Denmark, and Finland, providing insights into how CT practices are integrated into these countries’ curricula. The study found that CT is interpreted not only as programming but also as a broader problem-solving process in the policy documents of these countries. This suggests that CT is seen as relevant across various subjects, not just within computer science or mathematics. Further, the authors find that while the governmental documents reflect strong intentions to integrate CT, the qualitative analysis highlighted the ongoing challenge of translating these intentions into practice. The presence of CT keywords does not always align with practical curriculum implementation at the classroom level, suggesting a gap between policy and practice.

We are at the beginning of a lengthy endeavour. From our Nordic perspective, we suggest that all subjects should contribute to the development of CL and assessments, both as an integral part of the subjects and across subjects. Developing CL assessment criteria within a Nordic tradition should combine research and practical work. Over time, this combination of research and practice will ensure a consistent and valid progression through K9. To achieve this, however, we need to develop an assessment approach that will renew and revitalise CL into the core of subjects while paving the way to new and long-term research fields.

References

American Educational Research Association (AERA). (2014). Standards for educational and psychological testing. AERA.

Biggs, J. B. & Tang, C. S. (2007). Teaching for quality learning at university: what the student does. McGraw-Hill/Society for Research into Higher Education & Open University Press.

Bers, M. U. (2021). Coding as a playground: Programming and computational thinking in the early childhood classroom. Routledge.

Berthelsen, U. D. (2023). Digital humaniora og teknologiforståelse [Digital humanities and computational empowerment]. Tidsskriftet Læring og Medier (LOM), 16(28). https://doi.org/10.7146/lom.v16i28.136744

Biesta, G. J. J. (2013). Receiving the gift of teaching: from ‘learning from’ to ‘being taught by’. Studies in Philosophy and Education, 32(5), 449–461. https://doi.org/10.1007/s11217-012-9312-9

Bocconi, S., Chioccariello, A., & Earp, J. (2018). The Nordic approach to introducing Computational Thinking and programming in compulsory education. Report prepared for the Nordic@BETT2018 Steering Group. https://doi.org/10.17471/54007

Brennan, K. (2015). Objects To Think With. Constructivist Foundations, 10(3), 313–314.

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 Annual Meeting of the American Educational Research Association (Vol. 1, pp. 1–25).

Buus, L., Georgsen, M., & Lindenskov Tamborg, A. (2023). Teknologiforståelser i uddannelsessystemet: Editorial [Technology Comprehension in the Educational System: Editorial]. Tidsskriftet Læring og Medier (LOM), 16(28). https://doi.org/10.7146/lom.v16i28.142410

Caeli, E. N., & Bundsgaard, J. (2019). Computational thinking in compulsory education: A survey study on initiatives and conceptions. Educational Technology Research and Development. https://doi.org/10.1007/s11423-019-09694-z

Caeli, E. N., & Yadav, A. (2019). Unplugged Approaches to Computational Thinking: A Historical Perspective. TechTrends (2020) 64, 29-36. https://doi.org/10.1007/s11528-019-00410-5

Christensen, O., & Tufte B. (2010). Media Education: Between Theory and Practice. In S. Kotilainen, & S.-B. Arnolds-Granlund (Eds.), Media Literacy Education: Nordic Perspectives (pp. 109-120). Nordicom.

Cizek, G. J., Andrade, H. L., & Bennett, R. E. (2019). Formative Assessment: History, Definition, and Progress. In H. L. Andrade, R. E. Bennett, & G. J. Cizek (Eds.), Handbook of Formative Assessment in the Disciplines (pp. 3–19). Routledge.

Cutumisu, M., Adams, C., & Lu, C. (2019). A Scoping Review of Empirical Research on Recent Computational Thinking Assessments. Journal of Science Education and Technology, 28(6), 651–676. https://doi.org/10.1007/s10956-019-09799-3

Delcker, J., & Ifenthaler, D. (2017). Computational Thinking as an Interdisciplinary Approach to Computer Science School Curricula: A German Perspective. In P. Rich, & C. Hodges (Eds.), Emerging Research, Practice, and Policy on Computational Thinking. Educational Communications and Technology: Issues and Innovations (pp. 49-62). Springer. https://doi.org/10.1007/978-3-319-52691-1_4

Denning, P. J., & Tedre, M. (2019). Computational thinking. The MIT Press.

Dindler, C., Iversen, O. S., Caspersen, M. E., & Smith, R. C. (2022). COMPUTATIONAL EMPOWERMENT. In H. Abelson, & S.-C. Kong (Eds.), Computational Thinking Education in K–12: Artificial Intelligence Literacy and Physical Computing (pp. 121–139). MIT Press.

Dindler, C., Smith, R., & Iversen, O. S. (2020). Computational empowerment: Participatory design in education. CoDesign International Journal of CoCreation in Design and the Arts, 16(1), 66–80. https://doi.org/10.1080/15710882.2020.1722173

diSessa, A. A. (2001). Changing minds: Computers, learning, and literacy. The MIT Press.

diSessa, A. A. (2018). Computational literacy and “the big picture” concerning computers in mathematics education. Mathematical Thinking and Learning, 20(1), 3-31. https://doi.org/10.1080/10986065.2018.1403544 

Dolin, J., & Qvortrup, L. (2013). Fagdidaktik – viden og undervisning. Hans Reitzels Forlag.

Edelson, D. C. (2002). Design research: What we learn when we engage in design. Journal of the Learning Sciences, 11(1), 105–121. https://doi.org/10.1207/S15327809JLS1101_4

Edwards-Groves, C. (2012). Interactive Creative Technologies: Changing learning practices and pedagogies in the writing classroom. The Australian Journal of Language and Literacy, 35(1), 99–113. https://doi.org/10.1007/BF03651876

Graf, S. T. (2017). Dannende undervisning - en begrebsafklaring. Studier i læreruddannelse og profession, 2(1), 6-29. https://doi.org/10.7146/lup.v2i1.27706

Grover, S. (2022). Computational Thinking Today. In A. Yadav, & U. D. Berthelsen (Eds.), Computational Thinking in Education—A Pedagogical Perspective (pp. 18–40). Routledge. https://doi.org/10.4324/9781003102991-2

Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38-43. https://doi.org/10.3102/0013189X12463051

Hachmann, R. (2022). Cybervåbnet: Computationel praksis i 8. Klasse. I: S. S. Fougt, J. Bundsgaard, T. Hanghøj, & M. Misfelt (Red.), Håndbog i scenariedidaktik (pp. 489–501). Aarhus Universitetsforlag.

Hachmann, R. (2024a). Designing for Computational Literacy in Non-Computer Science Subjects. In Creating Design Knowledge in Educational Innovation (pp. 232–245). Routledge. https://doi.org/10.4324/9781003391432-25

Hachmann, R. (2024b). Computationel Literacy in Schools: Cognitive, social and material aspects. Learning Tech, 13, 1–27. https://doi.org/10.7146/lt.v8i13.149529

Hansen, S. B., Hachmann, R., & Dohn, N. B. (2023). Computational thinking beyond computer science. In S. H. Klausen, & N. Mård (Eds.), Developing a Didactic Framework Across and Beyond School Subjects (pp. 232–242). Routledge. https://doi.org/10.4324/9781003367260-21

Heintz, F., Mannila, L., Nordén, L.-Å., Parnes, P., & Regnell, B. (2017). Introducing Programming and Digital Competence in Swedish K-9 Education. In V. Dagienė & A. Hellas (Eds.), Informatics in Schools: Focus on Learning Programming (Vol. 10696, pp. 117–128). Springer International Publishing. https://doi.org/10.1007/978-3-319-71483-7_10

Jølle, L., & B. Skar, G. (2021). 13. En modell for planlegging, gjennomføring og evaluering av morsmålsfaglige læringsprosesser. I: L. Jølle, A. S. Larsen, H. Otnes, & L. I. Aa (Red.), Morsmålsfaget som fag og forskningsfelt i Norden (pp. 291–309). Universitetsforlaget. https://doi.org/10.18261/9788215050997-2021-15

Kafai, Y. B., & Proctor, C. (2021). A Revaluation of Computational Thinking in K–12 Education: Moving Toward Computational Literacies. Educational Researcher, 51(2), 146-151. https://doi.org/10.3102/0013189X211057904

Kafai, Y., Proctor, C., & Lui, D. (2020). From theory bias to theory dialogue: Embracing cognitive, situated, and critical framings of computational thinking in K-12 CS education. ACM Inroads, 11(1), 44–53. https://doi.org/10.1145/3381887

Kress, G., & Selander, S. (2012). Multimodal design, learning and cultures of recognition. The Internet and Higher Education, 15(4), 265–268. https://doi.org/10.1016/j.iheduc.2011.12.003

Krogh, E., & Piekut, A. (2015). Voice and narrative in L1 Writing. L1 Educational Studies in Language and Literature, 15, 1-32. https://doi.org/10.17239/L1ESLL-2015.15.01.10

Mills, K. A. (2015). Literacy theories for the digital age: Social, critical, multimodal, spatial, Material and Sensory Lenses. Multilingual Matters. https://doi.org/10.21832/9781783094639 

Ministry of Children and Education (2019). Dansk Fælles Mål [Common Goals for Danish]. Retrieved May 10th, 2024.https://emu.dk/sites/default/files/2020-09/GSK_F%C3%A6llesM%C3%A5l_Dansk.pdf

Mifsud, L., Andersen, R., Frågåt, T., Boğar, Y., & Jensen, J. J. (2023). Representations of computational thinking in policy documents in an educational context: The cases of Denmark, Finland, and Norway. In P. Blikstein, J. Van Aalst, R. Kizito, & K. Brennan (Eds.), Proceedings of the 17th International Conference of the Learning Sciences (pp. 35-42). International Society of the Learning Sciences. https://doi.org/10.22318/icls2023.719530

Niemelä, P., Pears, A., Dagienė, V., & Laanpere, M. (2022). Computational Thinking – Forces Shaping Curriculum and Policy in Finland, Sweden and the Baltic Countries. In D. Passey, D. Leahy, L. Williams, J. Holvikivi, & M. Ruohonen (Eds.), Digital Transformation of Education and Learning—Past, Present and Future (pp. 131–143). Springer International Publishing. https://doi.org/10.1007/978-3-030-97986-7_11

Norwegian Directorate for Education and Training (2020). Norwegian Curriculum for primary, lower secondary and upper secondary - Kunnskapsløftet 2020. https://www.udir.no/in-english/curricula-in-english/

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.

Rajapakse-Mohottige, N. U. S., Andersen, R., & Bjerke, A. H. (2024). Computational thinking in Norwegian teacher education: An analysis of mathematics and science course descriptions. Nordic Journal of Digital Literacy, 19(1), 25–38. https://doi.org/10.18261/njdl.19.1.3

Riis, M., Andersen, B. L., Dybdal, M., Jørnø, R. L., Slot, M. F., & Hjort, M. (2021). Teknologiforståelsens fagdidaktik. Learning Tech, (10). https://tidsskrift.dk/learningtech/issue/view/9409/1585

Schön, D. A. (1992). Designing as reflective conversation with the materials of a design situation. Research in Engineering Design, 3, 131–147. https://doi.org/10.1007/bf01580516

Snow, E., Tate, C., Rutstein, D., & Bienkowski, M. (2017). Assessment Design Patterns for Computational Thinking Practices in Exploring Computer Science. SRI International.

Slot, M. F., Gissel, S. T., & Hjorth, M. (2021). Teknologiforståelse i fag. Learning Tech, 6(10), 191–217. https://doi.org/10.7146/lt.v6i10.125676

Slot, M. F., Rasmussen, H. F., & Kjærgaard, T. (2023). Teknologiforstyrrelser i læreruddannelsens danskfag: på vej med danskfaglige teknologiforståelser i læreruddannelse . Tidsskriftet Læring og Medier (LOM), 16(28). https://doi.org/10.7146/lom.v16i28.136093

Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148. https://doi.org/10.1016/j.compedu.2019.103798

UNESCO. (2023). What you need to know about literacy. https://www.unesco.org/en/literacy

U.S. Government. (2023). Building Computational Literacy Through Stem Education: A Guide For Federal Agencies And Stakeholders. A Report by the Computational Literacy Interagency Working Group, Federal Coordination in STEM Education Subcommittee, Committee on STEM Education of the National Science and Technology Council.

Valente, A., & Marchetti, E. (2020). Playful Learning and Shared Computational Thinking: the PaCoMa case study. In H-J. So, M. M. Rodrigo, J. Mason, & A. Mitrovic (Eds.), Proceedings of the 28th International Conference on Computers in Education, ICCE 2020 (Vol. II, pp. 180-189). Asia-Pacific Society for Computers in Education.

Vee, A. (2017). Coding Literacy: How Computer Programming Is Changing Writing. The MIT Press. https://doi.org/10.7551/mitpress/10655.001.0001

Weintrop, D., Rutstein, D., Bienkowski, M., & McGee, S. (2021). Assessment of Computational Thinking. In A. Yadav & U. D. Berthelsen (Eds.), Computational thinking in compulsory education: A pedagogical perspective (pp. 90–111). Routledge. https://doi.org/10.4324/9781003102991-6

Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.

Wing, J. M. (2010). Computational Thinking: What and Why? Unpublished manuscript Computer Science Department, Carnegie Mellon University.

 

©2024 Roland Hachmann and Marie Falkesgaard Slot. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.