"Would you tell me, please, which way I ought to go from here?" [said Alice]. "That depends a good deal on where you want to get to," said the Cat.
(Alice's Adventures in Wonderland, Lewis Carroll)
In this section, I discuss and illustrate how composition teachers can deal with the second challenge of using digital storytelling, namely the challenge of assessment. While there is limited literature on digital storytelling per se within composition, composition scholars have addressed the challenge of assessing multimodal work in general for some time.
A number of articles published in journals like Kairos and Computers and Composition have addressed the specific issue of assessing multimodal work (Takayoshi, 1996; Yancey, 2003; Ball, 2004; Sorapure, 2006; Sheppard, 2009). However, a search of literature on digital storytelling specifically shows that there is not much written about the issue of assessment in this particular case. In fact, even within the broader domain of multimodal composition, the challenge of assessment that Takayoshi highlighted in an article in 1996 seems to largely persist today (also see Raymond, 2008; Clark, 2010). As a result, composition teachers like me who decide to adopt digital storytelling in their classrooms continue to struggle with the assessment of students’ work and, in fact, even with how to define the term within college composition. The issue of assessing multimodal composition seems to have drawn the attention of composition scholars for the first time around the middle of the 1990s. In an article titled "The shape of electronic writing: Evaluating and assessing computer assisted writing processes and products," published in Computers and Composition, Takayoshi (1996) stated that "[a]s we have fleshed out the effects of computers on our pedagogies and writing theories, however, one aspect of teaching writing with computers has generally remained amorphous: assessment of student writing" (246).
What Takayoshi pointed out fifteen years ago seems to be still true because not many new frameworks for the assessment of multimodal work have been developed and implemented. Echoing Takayoshi’s concerns, Meeks and Ilyasova said in an article published in Kairos in 2003 that the assessment of multimodal composition can be challenging because there are a number of other factors to consider than the "text"' itself. Traditional frameworks of assessing academic composition are not capable of accounting for the multi-languagedness and multimedia affordances of digital storytelling; as a result, even teachers who understand the educational potentials of digital storytelling and would like to use it in their classrooms are unable to adopt it with confidence.
One of the most prominent challenges regarding the assessment of multimodal composition including digital storytelling is that the use of media is often seen as an addition of "cool" new components like image, sound, and video to traditional text-based composition. To illustrate this point, let us consider a digital story composed by a college, student Rocio Palacios which is available on her professor's course website. * Palacio uses family photographs, callout texts, cartoon images, music and other media assets apparently to enhance the message of her script, but those assets often overwhelm the audience’s attention. The "tone" of those visuals often jars with the message of Rocio's voiceover.
This issue about the difficulty of getting students beyond the "coolness" of new media is explained in a striking way by Zoeteway and Staggers (2003) in their article "Beyond ‘current-traditional’ design: Assessing rhetoric in new media." The authors remember the time when composing web text was considered a cool new form of composition so instructors and students alike did not have the standards of the assignment figured out very well:
In those days, having links that actually linked to other things, text that showed up where you meant for it to show up, and--splendor of splendors--a free-floating disembodied head that actually rotated was enough to merit an "A." The fact that our pages were virtually devoid of meaningful content and were at the same time spectacularly ugly escaped our notice. And it escaped the notice of our instructors. None of us--students or instructors--had any clear ideas about what we were doing. We were thrilled with our index page, our long, endlessly scrolling page full of text, our list of links, and, of course, our giant disembodied spinning head. (135)
To some extent, the same kind of situation continues to persist in the composition classroom where multimodal composition including digital storytelling is used. After students spend a lot of time learning and using new technologies and putting together the materials into whatever form that they could within the limits of the time available, teachers are more likely to focus on the "nice work" that the students did within those limits than to start asking critical questions. Part of the reason for the lack of critical assessment is because multimodal composition is often an add-on rather than an integral part of the course, as it was in my case.
As a result, instructors do not want to grade what they did not give much time teaching—in some cases did not have much expertise themselves—and instead give most students a good grade. Zoeteway and Staggers argue that "words, images, sound, color, shape, motion, features of the interface, grid structures, navigation systems and devices, and connections between and among all of these temporally and spatially are rhetorical and contribute to making meaning" (134) and therefore they must be accounted for while assessing students’ work. It is necessary to develop analytical frameworks in order to account for all those ingredients of digital storytelling. The new framework will also help teachers who tend to uncritically celebrate the technology involved and use digital storytelling without a clear sense of its educational goals and without explicit pedagogical goals and strategies.
Another challenge with the assessment of digital storytelling comes from the common assumption that students are generally tech-savvy so that we can just tell them to use the new media to complete the assignment, rather than teaching them the necessary technological skills. In other words, we should only grade what we teach. With regard to effective pedagogy, which I argue is an important prerequisite to effective assessment, Borton (2005) provides a list of suggestions:
1. Use the multimodal narrative as an opportunity to connect with your students and help them to "enter the conversation."
2. Encourage students to use the medium they know to compose their narratives.
3. Use the multimodal narrative to teach audience.
4. Use the multimodal narrative as an opportunity to teach rhetorical choices, critical reflection, and analysis.
In that sense, the amount and effectiveness of Rocio’s use of images, callouts, and cartoons in her story could be a great opportunity for a teacher to assign the student or her class a useful task of rhetorical analysis.
Digital storytelling is most commonly used as a mode of composing narrative and reflective genres, and therefore it tends to involve emotional issues, painful experiences, and subjects that make it harder to critique the work. As authors of a web document titled "Digital storytelling: Multimedia archive" at the Georgetown University note, "[d]ue to their affective involvement with this process and the novelty effect of the medium, students are more engaged than in traditional assignments. These factors can create a 'spiral' of engagement, drawing students into deeper and deeper engagement with their topics." In spite of the benefit of better engagement, however, emotional content makes it generally harder to evaluate students' work, whether multimodal or print-based; and it is even harder to assess emotionally charged content when it is presented in multimodal form. For example, in the case of Rocio’s story, it could be harder for her teacher to discuss the "appropriateness" of using images of her family members and childhood memories.
The challenge of assessing student works is rarely an isolated problem; assessment almost always has a wash back effect on the design and execution of our curriculum and pedagogy. Powell, Alexander, and Borton (2011) argue that digital multimodal assignments "can be a useful tool to teach students complex concepts and skills that we typically aim for in first-year composition" such as the following that the authors cite from Takayoshi and Selfe (2007): "audience awareness, exigence, organization, correctness, arrangement, and rhetorical appeals" (web). Advocates of multimodal composition have praised it because it engages students in finding, evaluating, and synthesizing different types of information into the final product. But it is only when we actually teach students how to communicate complex concepts and skills by using digital storytelling, or any medium for that matter, that we can grade them on those issues. As Rubin (2008) points out in "Digital storytelling: A powerful technology tool for the 21st century classroom," digital storytelling, if properly used, has the potential to promote students’ skills and understanding in multiple literacies, including
digital literacy—the ability to communicate with an ever-expanding community to discuss issues, gather information, and seek help; . . . technology literacy—the ability to use computers and other technology to improve learning, productivity, and performance; visual literacy—the ability to understand, produce, and communicate through visual images; and information literacy—the ability to find, evaluate, and synthesize information. (224)
Some scholars of multimodal composition who have addressed the challenges of assessing student work composed in multimodal forms have suggested that we need to develop new frameworks altogether; other scholars have argued that it is both possible and necessary to adapt the conventional frameworks of rhetorical analysis to the new media. In "HyperRhetoric: Multimedia, literacy, and the future of composition" Heba (1997) suggests that we need to broaden the traditional text-based framework of assessing print-based composition into a "semiotic" framework (19). He argues that "[p]rimarily, multimedia bypasses earlier media by blurring the boundaries among print, video, photography, audio recording, animation, and film by allowing us to combine and integrate varied and disparate media into one discursive space" (21). This means that we cannot take the components apart and understand them separately; we need to analyze and understand the synthesis created by multimodal composition.
That is, in the words of Heba again, because multimodal composition "involves more than textual communication, a broader definition of language is necessary to explain the phenomenon of multimedia communication, and semiotics provides a base" (29). As Ball (2004) suggests, digital multiliteracies also "offer readers a chance to enact and interpret an author’s argument through multimodal elements and navigational strategies" (416). Therefore, reading of multimodal work means attending to the synthesis of multiple sources of meaning at the same time, and from this perspective, using traditional frameworks of assessment may be insufficient for assessing multimodal work like works produced with digital storytelling. Whether we consider it first or last, we need to assess and assign credit for the semiotic synthesis or the mutual animation of meaning by the multiple media elements in a digital story.
Other composition scholars believe that it is necessary or pragmatic to adapt traditional framework of textual analysis to assess multimodal composition. These scholars argue that the principles of text-based composition can be adapted and that we must use what we are familiar with instead of trying to invent entirely new frameworks: "Instructors and students both might be better served if we move away from design guidelines and focus instead on transforming the rhetorical criteria we already understand, such as coherence, clarity, relevance, so that we can read and evaluate them as they operate in new media" (Zoeteway & Staggers, 2003). I believe that a pragmatic approach that adapts traditional terms and frameworks of analysis and assessment will help to a certain extent, but we need to add considerations of new affordances of new media in order to adapt conventional assessment frameworks to digital storytelling. Indeed, such adaptation is necessary for specific types of multimodal composition even after we develop general frameworks. Shorapure (2006) states that because there is a wide variety of new media composition, it is hard to develop a general rhetorical framework for assessment. "A broadly rhetorical approach can accommodate these differences—that is, an approach that focuses assessment on how effectively the project addresses a specific audience to achieve a specific purpose" (3). Shorapure admits that the "weakness of a broad rhetorical approach is that it doesn’t in itself offer any specific guidance or criteria for handling the multimodal aspects of the composition" (ibid.). If we take this approach in assessing Rocio’s work, we would start by assigning credits for conventional, text-based rhetorical elements, then add credits for the affordances of new media.
Yet other composition scholars see both overlapping and non-overlapping areas between traditional and multimodal composition with regard to assessment. In "The rhetorical work of multimedia production practices: It’s more than just technical skill," Sheppard (2009), argues that "rather than viewing development of multimedia as just technical skill" (122) composition teachers who assign multimedia work in the classroom must pay sufficient attention to the practices of multimodal composition as processes that involve "critical negotiations" with the multimodal assets:
[The multimodal] development activities build on traditional print-based literacies and rhetorical practices but require additional considerations in order to achieve the desired effects on intended audiences. Beyond established rhetorical concerns such as audience, purpose, and context, designers must also make rhetorical choices speciﬁc to the development of multimedia. These technological rhetorical considerations include decisions such as the appropriateness of technologies for a given situation and the selection and integration of media to facilitate reader/user comprehension of the text. (122-23)
Composition teachers who have any of the views above suggest that we must teach students how to pay attention to "additional rhetorical concerns"; this means that at the very least we need to learn how to assess those additional concerns when assessing students’ work. To borrow Sheppard’s words again, "[e]xploring the communicative affordances of different media and planning their integration with other modes, understanding the technical capabilities and contexts of use for the target audience, and developing an interface and set of interactions that make a text usable and intuitive are just some of the many technological rhetorical considerations speciﬁc to successful multimedia production" (128). If the process of teaching multimodal work involves "sophisticated integration of knowing how and when to use appropriate technologies, where to ﬁnd or how to create the necessary media resources, how to interact with the people involved with a project, and how to prepare the material for the context in which it will be used by its intended audience" (130), the assessment of those processes needs equally sophisticated understanding and appreciation of all of them.
Among the many unique characteristics of multimodal composition, which composition teachers need to pay attention to while teaching or assessing multimodal work, the "synthesis" of different media and materials is perhaps the most important. That synthesis is also one reason why traditional rhetorical frames of analysis are useful but insufficient. In "Looking for sources of coherence in a fragmented world: Notes toward a new assessment design," Yancey (2004) states that the "key to these new ways of writing, these new literacies, these new textures . . . [is] a composition made whole by a new kind of coherence" (89). Yancey suggests that the complex coming together of different modes is one of the reasons why even composition teachers who decide to adopt digital, multimodal composition "seem decidedly discomforted when it comes time to assess such processes and products—regardless of whether by assess we mean responding to student texts or putting a grade to them…." (90).
One way to tackle that challenge of assessment, which Yancey rightly says is discomforting even to multimodal enthusiasts, is to attend to the "coherence" among the components of a multimodal work, or, more specifically the "function of a pattern that is created through the relationships between and among context, screen, image, the visual, the aural, the verbal, and with repetition and multiplicity as the common features" (95). We can also see the coherence in digital, multimodal composition as the "weaving" together of rhetorical and media components : "Digital compositions weave words and context and images: They are exercises in ordered complexity—and complex in some different ways than print precisely because they include more kinds of threads" (95).
The suggestions of scholars who see commonalities between traditional and multimodal composition encourage us to focus on what is new and how we can connect them to what we already know and what we must recognize as unique, can greatly help us meet the challenges of assessing multimodal composition. As Takayoshi (1996) writes, the changes brought about by the use of computer technology "necessitate a transition from assessment practices based in theories about print literacy to assessment practices based in computer-assisted composition theory" (246), and those new theories may need to be the framework into which theories traditional print literacy may be adapted as appropriated. In the next section, I demonstrate this position with a rubric and discussion of how I use it for assessing students’ digital storytelling works in the composition classroom.
* This digital story is not from a college composition classroom but I chose it as a sample for analysis because it best represents the kind of work that my composition students produce and, therefore, poses the kinds of challenges that I intend to address in this article; not many digital stories composed by students in college writing classes are available online, other than a few that are used by scholars of multimodal composition (e.g., Perl, 2010) in order to illustrate other types of specific issues and arguments.