Running Head: EFFECTS OF IMAGE COLOR AND CAPTIONS ON MEMORY
The Effects of Image Color and Captions on Long-Term Memory
Lanie Castro Amanda Harbin San Jose State University
Abstract
In areas such as academia and marketing, recall of visually presented stimuli is often necessary. Research has shown that recall of images may be affected when captions are included, and may also be affected by the presence or absence of color in images. However, research regarding the effect of images with captions on recall was not found. Eighty university students viewed one of four conditions of stimuli: color images with captions, color images without captions, black and white images with captions, and black and white images without captions. Participants described as many images as possible with a one minute delay between the last stimulus and response. Students who saw images with captions, regardless of color, remembered more than those who did not see captions. These findings have implications for marketing, in that customers viewing advertisements including images have higher probabilities of recalling them, if the images are coupled with captions. The study also implies that learning activities assisted by visual stimuli could be augmented by using captions to assist learner recall.
The Effects of Image Color and Captions on Long-Term Memory
Knowing the effect that color in an image can have on the long-term memory is valuable in many areas including academics and cost effective marketing. Many technology companies create their software user interfaces in color stating that this yields better user interactivity (Apple Human Interface Guidelines, 2008). Color is used to make a software program or webpage look more interesting, stand out, and to enhance people's memories of what particular buttons do. For instance, Microsoft's Internet Explorer uses a large blue “e” as their icon to launch the application, and the majority of people recognize that icon as Internet Explorer.
In addition to the impact of color, the presence or absence of captions accompanying an image is also important to companies. Online shopping companies highlight recently viewed items to remind shoppers of items they may be interested in purchasing. It is possible that the image paired with a one word description or caption, as opposed to simply highlighting, could be effective in aiding their customers memory of an item (Yang, 1997). Companies will know how to more effectively advertise products by presenting their products in a format that customers are more likely to remember
(Tavassoli, 2001). Presenting advertisements in a way that customers are most likely to remember them means that those customers will recall those advertisements and will be more likely to purchase the items advertised.
Current research primarily looks at the effects of sentences coupled with images on memory, and indeed, Amazon and similar companies present their customers with images of recently viewed items paired with a sentence description. While the present study does not deal directly with sentences, the observed effects of single word captions may serve to warrant further study on these captions versus sentences.
Long-Term Memory
Long-term memory is a complex process which results in retaining and subsequently recalling or recognizing a stimulus after a relatively long delay between the 'learning' or 'memorizing' and the retrieval process. Long-term memory consists of four components or “systems.” They are episodic, semantic, procedural, and perceptual representation memory (Nyberg & Tulving, 1997).
Episodic Memory
Episodic memory is the mechanism by which the mind organizes collections of temporally related events in a structured manner. This not only allows people to spontaneously remember
individual experiences but also allows them to consciously pick out particular collections of memories or episodes (Tulving, 1993).
Semantic Memory
We use semantic memory when we absorb complex abstract information (words, i.e. linguistic signals or cues) from the external environment. In parallel, Semantic memory processes this
information while retrieving meanings for those words (Kutas & Fredermeier, 2000). Procedural Memory
The procedural memory system is often seen as implicit and unquestioned, happening over time and growing with each experience (Gupta & Cohen, 2002). An example of procedural memory would be the process of learning to ride a bicycle. Each step is stored as greater knowledge of how to
accomplish this task, and are available in procedural memory when necessary. Perceptual Representation Memory
The perceptual representation memory system is thought to be a non-conscious system which reveals itself through implicit memory tests (Tulving & Schacter, 1990). The concept of perceptual memory itself refers to the neural and cognitive processes underlying the storage of sensory
information. This particular system is located early in the visual processing stream and consists of a series of parallel special-purpose perceptual mechanisms with independent but limited processing resources (Magnussen & Greenlee, 1999). With this system a person has the ability to have long-term memory of images.
Visual Memory
Visual memory, is a component of the perceptual representation memory system. In addition, visual memory has three sub-components. The first analyzes basic features, while the second has attention focused on details of the visual stimuli (Vogt & Magnussen, 2007). Long-term visual
memories are classified as being in the third sub-component of visual memory which pays attention to scene schema, and provides basic information about the arrangement and details of a given scene. Together these three systems are thought to contribute to the long-term memory of images (Vogt & Magnussen, 2007).
Many variables contribute everyday to visual memory. In any given day a person is bombarded with objects and printed images that they either have to remember or happen to remember. Some of these images are representations of something (pictures) and some of these images are symbols of something (words). Paivio's dual coding model demonstrates that pictures are remembered better than words. Pictures are dually coded, and are stored in image memory, which happens to be the more effective code for item memory, whereas words are not dually coded or stored to image memory (Snodgrass & Vanderwart, 1980).
Images, whether pictorial or symbolic, have shapes, sizes, spatial arrangement, coloring, shadowing, etc. There is research on recognition tasks and color and there is also research on the memory and recall of images. However the research regarding memory and recall of images has used mostly black and white line drawings (Spence, et al., 2006). It was once believed that color did not
play an important or any role in visual memory (Spence, et al., 2006). However, more recent research demonstrates that color plays a very significant role in visual memory, especially in that of images of natural scenes (Spence, et al., 2006). When participants saw the images in color, they demonstrated a noticeable increase in the recall of images of natural scenes (Gegenfurtner & Rieger, 2000). There is however, little research examining the underlying difference between color and black and white images.
Image Color
Color plays a leading role in an individual’s experience of the visual world, but plays an equally important role in the retrieval of images seen while having a visual experience. Initially, color leads to an image coding advantage at the very early stages of sensory processing (Gegenfurtner & Rieger, 2000) most probably by easing the image segmentation task. Later, color leads to an advantage in retrieval because it enhances the image stored in memory. In other words, color adds an additional cue which can be used on both sensory and cognitive levels of processing.
For subjects given color images of natural scenes as well as matching black and white images, recognition accuracy was higher for the color images and it was found that color information
contributed to the recognition process (Gegenfurtner & Rieger, 2000). Also, highlighting text with a color marker has a positive effect on retention of selected text material (Fowler & Barker, 1974). This suggests that color has a role in visual memory in regards to the encoding and retrieval process of recognition.
Captions
Research has examined how people remember pictures with sentences, such as presenting the participants with a picture of a wagon in a kitchen and a sentence such as "The wagon is in the kitchen" printed next to the picture (Radvansky & Copeland, 2006). Not much is known about how people
remember a picture varied in relation to the presence or absence of a caption. For example, how someone would remember seeing a picture of a computer with the simple caption"computer" next to the image, as opposed to a descriptive sentence, or the image by itself. However, previous research shows that when an image is paired with a caption, participants recall more frequently and accurately the caption that is related to the image versus captions that are unrelated to the image (Bahrick & Gharrity, 1976).
Interaction of Image Color and Captions/Words
Pictures and words are two of the most popular types of stimuli in memory research. When comparing them, they have different observed effects on memory (Radvansky & Copeland, 2006). For example, when participants are presented with words and pictures serially during a memory task, the participants remember more pictures than they do words and recall them in serial order more accurately (Whitehouse, et al., 2006). Pictures are recalled better than words not only for adults but also for children, demonstrating the picture superiority effect in all age groups. However, the effect did increase with age, the adults exhibiting the effect more than the children (Whitehouse, et al., 2006). Researchers had thought that younger children would demonstrate the superiority effect greater than adults because of the primacy of pictorial information, however this was not the case. This discovery shows compatibility with the arguments that the picture superiority effect is the result of dual coding (Whitehouse, et al., 2006).
It is clear that color is an important element and coding mechanism for the recall of images. Researchers have assumed that there is a link which exists between mental imagery and other forms of processing for quite some time (Holmes, et al., 2008). Recently, this assumption was put to the test by comparing imagery and alternative types of encoding, such as verbal-semantic. This research found that in order for the words to provide an additional cue for the image, there needed to be a semantic
relationship between the image and the words (Holmes & Mathews, 2005). It is still unclear whether images and words are processed independently or interdependently.
Method Participants
The aim of this experiment was to assess the effects of images and captions in a long-term memory free recall task. 80 San Jose State University students participated in this study. These participants consisted of male and female adults receiving higher education at a public California university. The study used a convenience sample which incorporated students with range of majors. The participants fell between the ages of 18 and 54 years old. Also, 69% of the participants were female and 31% were male. All participants had between one and seven years of college education. Materials
The stimulus was a slide show of 20 images, and there were four conditions of the stimulus (see Appendix A). The four conditions were: color images with captions, color images without captions, black and white images with captions, and black and white images without captions. All four of the conditions used exactly the same base images, varied only by their color saturation. Likewise, the two conditions with captions used the exact same set of captions. The captions were one word descriptions of the image with which they were paired. When choosing the specific images shown, the researchers took into account previous findings concerning the grouping of like objects. In particular, they chose images, each belonging to a unique category, in order to minimize the enhanced recall brought about by the association of related items. Also, to further the distance between each image, the researchers chose images which ranged in frequency according to the MRC Psycholinguistic Database. The researchers' list of images includes words ranging in frequency from one to two-hundred and five.
researchers asked them to perform. These typewritten instructions reiterated what the researchers told the participants verbally. The bottom three-quarters of the page had lines for the participants to write their responses (see Appendix B).
Participants also responded to a Background Questionnaire (see Appendix C) regarding their age, gender, and level of education. The Background Questionnaire also asked if the participants had any visual impairments such as colorblindness or dyslexia, or if they wear any kind of corrective lenses.
Procedure
All participants in this study were tested in university classrooms while seated in desks facing a projection screen. Before performing the experiment, participants signed a consent, and were reminded that they could abandon the experiment at any time. Participants were then instructed to fill out a background questionnaire. The background questionnaire did not ask for any personal, identifiable information, only information which was pertinent to the study. After signing the consent form and filling out the background questionnaire, the participants viewed a practice slide for seven seconds (see Appendix D). After the seven seconds the practice image was replaced with a blank slide and the researchers then gave the participants verbal examples of proper and improper responses. This trial run served only to give the participants a clear idea of what the researchers expected of them in the actual experiment. The participants were told to pay attention to the following twenty images, and informed that they would be asked to recall the images later in the experiment. The participants viewed each of the twenty images one at a time for seven seconds each (Radvansky & Copeland, 2006). The slide show had a total duration of two minutes and forty seconds. After the slide show, participants were asked to sit quietly while the researchers passed out the response sheets. Passing out the response sheets served as a distracter task in order to flush out the participants' working and short term memory,
this activity took not less than one minute. The participants were then given one minute and twenty seconds, half the time of the slide show, to recall the twenty images they saw (Radvansky & Copeland, 2006). After this time, the participants were instructed to stop writing their responses and set down their pens. The researchers then collected the response sheets, debriefed the participants, and gave the participants the opportunity to ask any questions they may have had.
Scoring
The researchers measured, on a scale from one to twenty, the number of images correctly recalled by each participant. Each correct answer on the response sheet counted as one point. The researchers determined acceptable correct answers based on a coding sheet they had constructed before the experiment. The coding sheet included common misspellings and synonyms (see Appendix E).
Results
This study evaluated the effects of image color and captions on Long-Term Visual Memory. Effects of Image Color
Participants who saw black and white images recalled more images (M = 9.94 , SD = 3.43) than participants who saw color images (M = 9.65, SD = 2.72). However, this effect was not statistically significant, F(1, 80) < 1. This result was not the expected outcome.
In a separate analysis, participants who recalled fewer than five images were normalized to five and the data went under reanalysis. The researchers chose five because it was two standard deviations away from the mean. There was no statistically significant effect F(1, 80) < 1. Also, participants who recalled more than 15 images were normalized to 15. Again, there was not a statistically significant effect F(1, 80) < 1.
Another analysis using Images and Corrective Lenses as the factors, showed that participants who wear corrective lenses recalled more images if they saw color images (M = 12.22, SD = 2.68) than
if they saw black and white images (M = 10.00, SD = 2.26). Participants who do not wear corrective lenses recalled fewer images if they saw color images (M = 9.76, SD = 2.40) than if they saw black and white images (M = 13.89, SD = 2.57). This interaction was statistically significant, F(1, 80) = 4.986, p < .029.
Effects of Words
Participants who saw the images with a caption recalled more images (M = 11.06, SD = 2.91) than participants who saw images without a caption (M = 7.96, SD = 2.22). The effect was very statistically significant, F(1, 80) = 29.70, p< .0001.
For a separate analysis, the data of participants who recalled fewer than five images was normalized to five and the data went under reanalysis. There was a statistically significant effect F(1, 80) = 29.70, p< .0001. Also, participants who recalled more than 15 images were normalized to 15 and the data went under reanalysis. There was a statistically significant effect F(1, 80) = 30.25, p< .0001. The results from this analysis confirm the significant effects found in the previous two analyzes.
In another analysis, using Captions and Corrective Lenses as factors showed that participants who wear corrective lenses and saw images with words recalled more images (M = 11.17, SD = 2.67) than participants who also saw images with words but did not wear corrective lenses (M = 11.00, SD = 3.08). However, this effect was not statistically significant F(1, 80) < 1.
Interaction of Images and Words
The interaction of images and captions was not significant, F(1, 80) < 1, for the current data set. There was no difference between color and black and white images for the groups which saw captions. The highest average was for the condition with black and white images with captions (M = 12.05, SD = 3.09). The lowest average was for the condition with color images and no captions (M = 7.85, SD = 1.79). These results were not the expected outcome.
When analyzing the interaction images, words, and corrective lenses have on each other, participants who saw color images, with captions, and wear corrective lenses recalled more images (M = 12.22, SD = 2.68) than participants who saw black and white images, without captions, and do not wear corrective lenses (M = 7.00, SD = 1.26). These results showed a statistically significant effect F(1, 80) = 10.75, p< .002.
Discussion
This study investigated the effects of image color and captions on long-term visual memory. Previous research documents the effects of color images versus black and white images on recall (Spence et al., 2006). The effects of the presence of words on recall is also documented (Tehan, Hendry, & Kocinski, 2001), but to a lesser degree. The present study built on previous research by using a free recall task which uses four conditions involving color or black and white images, with or without captions in order to determine if there is a difference.
Image Color
Although the effects of image color (color and black and white) on long-term visual memory were not statistically significant in this study, participants tended to have a higher rate of recall after viewing black and white images than for color images. Participants who saw black and white images remembered on average two images more than participants who saw color images. Color is supposed to enhance peoples' visual experience and aid in long-term memory (Spence, et al., 2006). This study's results suggest the opposite.
Previous research suggesting that color images are more likely to be remembered than black and white used only printed images. Participants in the present study viewed images via a projector onto a projection screen. In addition, each condition of the study was performed in a different classroom and used a different projector and screen. According to experts in the field of computer
science, color display from device to device can vary greatly (Lindbloom, 1989). Also, lighting in the different classrooms could have contributed to lower contrast color images and higher contrast black and white images. Research suggests that contrast perception is not independent of luminance. This means that a brighter classroom could have “washed out” the color projections, while dark rooms could have emphasized the black and white images (Peli, et al., 1991).
However, the data did show a statistically significant effect of the participants' use of corrective lenses. In Figure 1, note that participants who wear corrective lenses perform better on the recall task when they saw color images and recalled fewer images when they saw black and white images. In addition, participants who do not wear corrective lenses recalled more images when they saw black and white images, and that their recall performance decreases when they saw color images. The findings also showed that participants who do not wear corrective lenses and saw black and white images recalled more images than participants who do wear corrective lenses and saw color images.
There was not any previous research found regarding the effects that corrective lenses have on the perception and memory of color images. However, some corrective lenses not intended for color enhancement (like that of those for color blind patients) are tinted and could possibly create a
difference between how participants with corrective lenses and without corrective lenses perceive, encode, and remember the color images.
Captions
This study focused on using words as a caption for what the image was displaying. The significant effect found in the increased ability to recall images when coupled with a caption was consistent with the literature (Tehan, Hendry, & Kocinski, 2001), although the word length chosen for this study was four to five words shorter than commonly seen in other such studies.
very strong effect is interesting considering that previous research suggests that the effect should not be as significant as this study demonstrated (Whitehouse, et al., 2006). When words are presented serially with pictures during a memory task, pictures are better remembered and recalled more accurately than words, demonstrating a picture superiority effect, for both adults and children. This effect is also found to increase with age (Whitehouse et al., 2006). This previous research suggests that in the present study participants recall images better than words due to the fact that the participants were all adults. Although this does not imply that images coupled with a caption would not be recalled even better, the picture superiority research suggests that the large difference between the groups who did and did not see words should be smaller.
Research shows that when participants view an image paired with a caption they do not encode them separately but encode the image and the caption together as one functional stimulus (Bahrick & Gharrity, 1976). This could explain the results of the present study. Also, because the participants in the present study viewed images with related captions, the participants were more likely to encode and remember the stimulus than if the captions were unrelated to the image. Previous research suggests that participants tend to encode and remember either: both parts of the stimulus or none of the parts. The former is more likely when images and captions are related and the latter is more likely when they are unrelated (Bahrick & Gharrity, 1976).
Interactions
When comparing images and captions on memory, the two stimuli have different effects (Radvansky & Copeland, 2006). There was no statistically significant interaction between the two. This suggests that images and captions act independently from one another on long-term memory.
However, Figure 2 illustrates that when analyzing images, words, and whether the participants wear corrective lenses or do not wear corrective lenses as factors, there was a significant three-way
interaction. This suggests that corrective lenses may have an important impact on visual memory. More research is needed to determine this, and to study how corrective lenses might be altering how image color and captions affect memory. It is possible that corrective lenses, not meant to enhance color for color blind patients, may alter color perception. However, this demands further research. A more detailed background questionnaire, or a standard colorblindness test performed beforehand could lead to more specific results with regards to factors such as colorblindness, and corrective lenses. These details will aid further research.
Some changes that should be considered for further research are, having a more detailed and specific coding sheet. For instance with the image ship, the response “yacht” should be on the coding sheet, and for the image of the bride, the words “young girl” should be on the coding sheet, because the image is indeed a young girl.
One important direction for further research would be to investigate further the influence of wearing corrective lenses on memory. Another direction would be to study what such findings mean for those who are colorblind, specifically with regard to long-term memory of images and/or natural scenes (opposed to sketches or silhouettes). The findings of such research would be very important as they reach into many areas of everyday life including academia, and be extremely valuable for
advertising companies.
This study is a part of a line of research that is important for understanding how a person's memory might be affected by the types of images they see and the captions which may accompany those images. The pattern of results from the present study taken together show that images and words influence long-term memory independently and do not influence each other. More specifically, use of corrective lenses seems to be an important factor found to affect memory.
References
Apple Human Interface Guidelines. (2008). Apple Developer Connection. Retrieved December 6, 2008, from <http://developer.apple.com/documentation/userexperience/Conceptual/AppleHIGuide lines/XHIGIntro/chapter_1_section_1.html#//apple_ref/doc/uid/TP30000894-TP6>.
Baddeley, A. D., Thomson, N., & Buchanan, M. (1975). Word length and the structure of short-term memory. Journal of Verbal Learning & Verbal Behavior, 14, 575-589.
Bahrick, H. & Gharrity, K. (1976). Interaction among pictorial components in the recall of picture captions. Journal of Experimental Psychology, 2, 103-111.
Cowan, N. (1999). An embedded-process model of working memory. In A. Miyake & P. Shah (Ed.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 102- 134). New York: Cambridge University Press.
Fowler, R. L. & Barker, A. S. (1974). Effectiveness of highlighting for retention of text material. Journal of Applied Psychology, 59, 358-364.
Gegenfurtner, K. R. & Rieger, J. (2000). Sensory and cognitive contributions of color to the recognition of natural scenes. Current Biology, 10, 805-808.
Gupta, P. & Cohen, N. J. (2002). Theoretical and computational analysis of skill learning, repetition priming, and procedural memory. Psychological Review, 109, 401-448.
Holmes, E. A. & Mathews, A. (2005). Mental imagery and emotion: a special relationship? Emotion, 5, 489–497.
Holmes, E. A., Mathews, A., Mackintosh, B. & Dalgleish, T. (2008). The causal effect of mental imagery on emotion assessed using picture-word cues. Emotion, 8, 395-409.
Kutas, M. & Federmeier, K. D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends is Cognitive Sciences, 4, 463-470.
Lindbloom, B. (1989). Accurate color reproduction for computer graphics applications. Computer Graphics, 23, 117-126.
Magnussen S. & Greenlee, M. W. (1999). The psychophysics of perceptual memory. Psychological Research, 62, 81-92.
Nikolic, D. & Singer, W. (2007). Creation of visual long-term memory. Perception & Psychophysics, 69, 904-912.
Nyberg, L. & Tulving, E. (1997). Searching for memory systems. European Journal of Cognitive Psychology, 9, 121-125.
Peli, E. Yang, J., Goldstein, R., & Reeves, A. (1991). Effect of luminance on suprathreshold contrast perception. Optical Society of America, 8, 1352-1359.
Radvansky, G. A. & Copeland, D. E. (2006). Situation models and retrieval interference: pictures and words. Psychology Press, 14, 614-623.
Russo, R. & Grammatopoulou, N. (2003). Word length and articulatory suppression affect short-term and long-term recall tasks. Memory & Cognition, 31, 728-737.
Snodgrass, J. G. & Vanderwart, M. (1980). A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6, 174-215.
Spence, I., Wong, P., Rusan, M. & Rastegar, N. (2006). How color enhances visual memory for natural scenes. Association for Psychological Science, 17, 1-6.
Tavassoli, N T. (2001). Color memory and evaluations for alphabetic and logographic brand names. Journal of Experimental Psychology: Applied, 7, 104-111.
Tehan, G. & Tolan, G. A. (2007). Word length effects in long-term memory. Journal of Memory and Language, 56, 35-48.
Tehan, G., Hendry, L. & Kocinski, D. (2001). Word length and phonological similarity effects in simple, complex, and delayed serial recall tasks: implications for working memory. Memory, 9, 333-348.
Tulving, E. & Schacter, D. L. (1990). Priming and human memory systems. Science, 247, 301-306. Tulving, E. (1993). What is episodic memory? American Psychological Society, 2, 67-70.
Vogt, S. & Magnussen, S. (2007). Long-term memory for a common theme. Experimental Psychology, 5, 463-470.
Vogt, S. & Magnussen, S. (2007). Long-term memory for 400 pictures on a common theme. Experimental Psychology, 54, 298-303.
Whitehouse, A. J. O., Maybery, M. T. & Durkin, K. (2006). The development of the picture-superiority effect. British Journal of Developmental Psychology, 24, 767–773.
Yang, C. (1997) An exploratory study of the effectiveness of interactive advertisements on the internet. Journal of Marketing Communications, 3, 61-85.
Tractor Shoes Apple
Bicycle Campfire Ship
Computer Rabbit Skateboard
Pencil
Telephone Rose
Response Sheet
You will now have one minute and ten seconds to write down all of the images you were shown. You may use as many words as you like to describe the image, and you may write the images in any order.
___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________ ___________________________________________________________________________________
Background Questionnaire
Gender: Male Female Other:________________________
Age: ______ years
Are you color blind? Yes No
Do you have dyslexia? Yes No
Do you wear corrective vision lenses of any kind? Yes No
Are you near sighted, far sighted or do you have 20/20 vision?
Near Sighted Far Sighted 20/20
Have you ever been diagnosed why ADD or ADHD? Yes No
Is English your first language? Yes No
How many years of college education have you had?_____________________________
Figure 2 0 3.75 7.50 11.25 15.00 Color BW
Means of Recall (Color, Lenses, Captions)
Captions
w/ Lenses No Captionsw/ Lenses Captionsw/ No Lenses No Captionsw/ No Lenses
0 3.75 7.50 11.25 15.00 Lenses No Lenses
Means of Recall (Color, Lenses)
