8.3 Digital Composition and Multimodal Texts

Almost every aspect of our communication is, in some way, digitally based. To be a writer in the 21st century means that you are a digital composer. Digital composition involves writing based in digital creation that incorporates multimodal elements. If you type your research essay on a computer using Google Docs, then you are a digital composer. But digital composition goes beyond the standard essay typed into a word processor—it includes using other digital tools and elements to explore the topic and persuade your audience. To begin with, most digital texts are considered multimodal. In this chapter, we will discuss multimodality within the digital composition realm, but you should know that multimodal texts can be created without a digital device.

What Are Multimodal Texts?

Multimodal texts utilize sensory elements to further their rhetorical purpose and persuade an audience. These can include audio, visual, and/or physical elements. You can create a multimodal text using a digital technology tool, but you can also create a multimodal text by hand as well. The following are some examples of digital multimodal composition:

• Infographic
• YouTube video
• Podcast
• Website
• Blog
• Text message
• Word Document

And since we’re looking at types of multimodal texts, let’s also look at some examples of multimodal composition that take place outside of the digital realm:

• Collage
• Poster
• Speech
• Sculpture
• Painting
• Architectural models

Why Use Digital Composition and Multimodal Texts?

Since our world communicates through mostly digital means, learning how to compose in a digital environment is key to your success not only in school, but also in your current and future jobs, and as a member of society.

In addition to multimodal assignments preparing you for your future writing endeavors, multimodal assignments also allow students to use what they know. Melanie Gagich, in her essay “An Introduction to and Strategies for Multimodal Composing,” writes that students already have knowledge of multimodal composition. She writes in her essay, “Understanding that you are already composing multimodally in many digital spaces will help you transfer that knowledge and experience into your academic assignments. This understanding might also help alleviate any fears or anxiety you may have when confronted with an assignment that disrupts what you think writing should look like” (Gagich 74). If creating a multimodal text seems new to you, chances are you already have an applied understanding of multimodality, just by virtue of living and socializing in the 21st century.

Gagich continues and writes:

Perhaps the most significant reason for learning how to compose multimodally is that it provides “real-life” skills that can help prepare students for careers. The United States continues to experience a “digital age” where employees are expected to have an understanding of how to use technology and communicate in various ways for various purposes. Takayoshi and Selfe argue that “[w]hatever profession students hope to enter in the 21st century . . . they can expect to read and be asked to help compose multimodal texts of various kinds . . .” (3). Additionally, professionals are also using the benefits of digital tools and multimodal composing to promote themselves, their interests, research, or all three. Learning how to create a multimodal text will prepare you for the workforce by allowing you to embrace the skills you already have and learn how to target specific audiences for specific reasons using various modes of communication. (74)

Gagich writes that there are five steps to creating a multimodal text:

1. Determine your rhetorical situation.
2. Review and analyze other multimodal texts.
3. Gather content, media, and tools.
4. Cite and attribute information appropriately.
5. Begin drafting your text.

Making Visual Choices

According to Foundations of Communication, it may be a cliché to say, “A picture is worth a thousand words,” but visual images have power. Good communication is a multisensory experience. Pre-literate children gravitate toward books with engaging pictures. As adults we graduate to denser books without pictures, yet we still visualize ideas to help us understand the text. That’s because a strong image in a poem or a story appeals both to the readers’ senses and emotions or intellect. Advertisers favor visual media—television, magazines, and billboards—because they are an effective way to hook an audience. Websites rely on color, graphics, icons, and a clear system of visual organization to engage Internet surfers. Visuals bring ideas to life for many readers and audiences in multiple ways:

• As a link between raw data and usable knowledge
• To provide concrete, vivid, and quick representations
• To save space
• To speak in a universal language
• To be persuasive

There are many types of visuals you can incorporate in digital and physical multimodal composition to illustrate and emphasize your point. The rest of this section describes how visuals can support and enhance your ideas in a multimodal text.

Symbols

Symbols include a range of items that can be either pictographic or abstract. In the image above, mathematical symbols and the image of a heart are used to convey the concept of love.

It’s a visual way to represent love, which is an abstract noun that means different things to a wide range of people.

Maps

Maps sometimes include map charts or statistical maps. In the image to the right, two human palms are displayed, and a map of the world is painted onto them. In the background there is a blue sky with white, fluffy clouds. A map can represent more than geography. This map is a representation of our world, but the image could also speak to lines drawn by humans. One ironic feature of this picture is that the world’s territorial lines are drawn over the lines of two human hands. The image could also speak to matter, how nearly three-fourths of the world is made of water, and how up to 60% of the human body is made of water. Could the image speak to the interconnectedness of all beings on earth? That’s up to the reader to interpret. Images can also persuade.

Graphs and Tables

Graphs can take a variety of forms, the most common being line graphs, bar graphs, and pie charts.

In the graphs above, the creator is using visual representations of numbers to represent growth and decline in their topic. Graphs are a helpful way to visually illustrate change.

Diagrams

This visual illustrates a process. One example of a diagram would be a flow chart. The diagram below illustrates a workflow process.

Photographs

Photographs (still or moving) depict concrete objects, tell a story, provide a scenario, and persuade an audience.

In the image to the right, a picture of a baby is displayed clearly on a smart phone in the foreground. In the background of the picture, an elderly woman smiles, and her face is blurry. There is contrast between the baby, whose image is clear, and the woman, whose image is blurred. The contrast piques the viewer’s interest. Contrasting images, colors, and subjects can draw a reader into an image and force them to ask questions.

Illustrations

Illustrations can be realistic or abstract. The illustration pictured here displays a cartoonish picture of a Polaroid camera, the iconic camera from the eighties. For some readers, the illustration may invoke nostalgia, while for younger viewers, the camera may have a slightly historical feel. The reception of the illustration varies depending on the audience; regardless, the illustration can help persuade a wide range of audience members.

Why Use Visuals?

There are a number of reasons you might consider including visuals in documents, presentations, and other communications. Four reasons are detailed below:

Decorative: Visuals that do not represent objects or actions within the text but are added, instead, for aesthetic effect are considered decorative. Decorative visuals are often added to gain attention or increase the audience’s interest. Visuals can be used this way but can detract from the message you are trying to communicate and, thus, should be used with caution.

Representational: These visuals physically represent or physically resemble objects or actions in the text and are relevant to the content of the text. For example, rather than giving a detailed textual description of a new playground, you might include an image or render of the new playground and use the text to highlight specific features or information.

Analogical: Analogical visuals are used to compare and contrast two things and explain their likeness or correspondence. For example, a marketing consultant might try to clarify the difference between targeted marketing and mass marketing by including images of a single fisherman with a single fishing rod and line next to an image of a bigger boat with a fishing net. By using the fishing analogy, the marketing consultant is attempting to connect possible prior understanding of the audience, a visual, and the concepts of targeted marketing versus mass marketing.

Organizational: The purpose of organizational images is to provide structure to information, visually define relationships, and illustrate connections. A chart of the hierarchical structure of a company is one example of an organizational image.

There are many considerations to keep in mind when choosing visuals. When possible, use a variety of types of visuals, but remember that any visuals you use should enhance the content of the text. For example, only add photos if viewing the photos will clarify the text. Near each visual, explain its purpose concisely. Do not expect your readers to figure out the values of the visuals on their own.

For repositories of openly licensed photos, you can search Wikimedia Commons and Pixabay. Website creation software like WordPress, social media applications like Twitter, and other types of software like graphics makers such as Canva and video creators like Powtoon have their own repositories of free images you can use as well.

Making Audio Choices

Including audio in your multimodal project can enhance your text and move a reader both logically and emotionally. Audio enhances your message. You may want to consider the following audio choices:

• Music
• Spoken word
• Sound effects

Music: From pop to classical to Bollywood, music can be any use of vocalizations or instrumentals. Music can help convey theme in a video or podcast, and music can help heighten tension and advance plot in a story.

Spoken word: Spoken word audio choices include recording a voice over of the written text or a narration.

Sound effects: Sound effects include any kind of sound, from nature or manmade. A couple of examples include crickets, glass shattering, or applause. Sound effects can help characterize people and convey action. Sound effects can also affect the tone of a text, creating humor or suspense.

National Public Radio develops a wide range of podcasts that integrate music, narration, and sound effects to tell stories, as in this sample podcast episode aired on the NPR show Hidden Brain.

Why Use Audio?

There are a number of reasons you might consider including audio in documents, presentations, and other communications. According to the Advisory Group on Computer Graphics (AGOCG), using audio in multimedia has the following advantages:

It can convey meaning, providing an extra channel of information. It allows redundancy to be incorporated into the presentation of information, so that if the meaning is unclear to a user using visual information alone, the audio may clarify it.

Different learners use different learning strategies, and audio can provide additional information to support different learning styles—for example, some users may learn more by hearing than reading a piece of text.

Audio can add a sense of realism. Cultural associations with music allow you to convey emotion, time period, geographic location, etc.

It is useful for directing attention to important events. Non-speech audio may be readily identified by users—for example, the sound of breaking glass to signify an error. Since audio can grab the user’s attention so successfully, it must be used carefully so as not to unduly distract from other media.

It can add interest to a presentation or program.

Ease of communication—users may respond better to the spoken word than other media. For example, in a company presentation, ‘sound bytes’ from satisfied customers can be used.

The AGOCG also writes that there can be disadvantages to using audio:

• Like most media, files can be large.
• Audio can be easily overused, and when sounds are continually used users tend to tune them out.
• For most people, audio is not as memorable as visual media.
• Good quality audio can be difficult to produce, and like other media most commercial audio, particularly music, is copyright.
• Users must have appropriate hardware and software.

Accessibility

Being a digital writer means that you have to be consciously aware of your audience and their ability or inability to participate in the texts that you create. Not everyone can view a meme or infographic and not everyone can hear the sound on a YouTube video.

Closed Captions

Whenever possible, include captions for all videos that you create. This allows those who are hearing-impaired and deaf to access your message. You can edit the videos yourself to add captions, or you can use a platform like YouTube that will auto-generate captions that you can edit.

Audio Description

If you are making a video that contains scenes with any type of action, you will want to create an audio description.

Transcript

For any type of audio or video, you want to consider including a transcript of the spoken dialogue.

Alternative Text

For any photo or graphic you include in your text, make sure to provide alternative text by right-clicking on the image in Word, and then selecting edit Alt-text. Using Alt-text is a principle of web accessibility. Users with screen readers will be read an alt attribute to better understand an on-page image.