Tools and Technologies for Web Credibility Dr. Jonathan Lazar
Department of Computer and Information Sciences Universal Usability Laboratory
Towson University 8000 York Road
Towson, Maryland 21252 USA Phone: 410-704-2255
Fax: 410-704-3868 E-mail: [email protected]
1. Introduction
The diversity of Internet communication and content is celebrated as one of its strengths. We receive e-mails from friends, family, colleagues, and strangers. We read web pages from schools, non-profit organizations, corporations, and governmental agencies. We take part in online
communities, where we share and communicate about our hobbies, our religion, our problems, and our health. When we receive all of this
information, we rarely wonder about how credible this information is.
However, the credibility of web-based information is a very important issue. When you receive an e-mail, how do you know that it’s actually from the person or organization listed as the sender? When you check out a web page, how do you know that the content on the web page is valid? How do you know that the organization is who they are presenting themselves to be? How do you know that the information being provided within the online community is accurate and from a recognized authority? While this might not make a difference if you are in an online community for fans of the new baseball team from Washington, D.C., this will make a big difference if you are online to learn more about a rare form of cancer that you are facing. This paper will focus on the issues related to web site credibility. While the main focus of this paper will be on the credibility of web sites, the credibility of e-mails and online communities will also be discussed. Implications for various stakeholders will also be addressed.
2.0 Credibility of Information on The Web
Information comes in many different forms on the Internet and Web. We get information from web pages, we get information through e-mails, and we get information from posted messages in online communities. While the specific interfaces for these information sources might be different, these are all forms of information. Credibility of this information is therefore an
important concern. What is credibility? Credible information can be defined as believable information, information that can be relied upon as being accurate and correct (Fogg & Tseng, 1999). Trust, a closely related concept, can be defined as the belief that a person (or information) is reliable and dependable (Fogg & Tseng, 1999).
How do people determine the credibility of information? There are many different judgments that are used. Some judgments are made consciously after much consideration, while other judgments are made intuitively by the user. For instance, if a user is very familiar with the subject content, they might be able to determine the level of credibility simply on how well the information matches up what they already know to be credible (Fogg & Tseng, 1999). Other times, users judge the credibility based on
appearance. For instance, in the physical world, there are expectations regarding professional dress for certain professions. You wouldn’t interact with a mortgage loan officer who wears shorts and a t-shirt, as this person would not appear to be credible. In a similar vein, design features of
interfaces can help project the idea of credibility (more information on this will be included in later sections of the paper). In the physical world,
certifications can help project credibility. You only want to go to a board-certified doctor. You only want to have your taxes done by a board-certified public accountant. Web sites also have certifications that can help project
credibility.
Credibility of web-based information is an increasing concern. There are multiple reasons for this. The amount and scope of information being delivered through the Internet and Web is expanding. Most business communication now takes place through e-mails. When political scandals break, e-mails are usually the form of documentation that are in the news. Our interaction with businesses or individuals through face-to-face contact or telephone calls is now limited. For instance, many companies are now forcing consumers to use web sites as their primary form of communication with the company, charging extra if the individual even wants to speak with a person. For example, Northwest Airlines charges $5 if you want to speak to an individual and make a reservation through their call center.
Southwest Airlines, and many other companies, offer special discounts available only on their web site. Some banks now charge extra if you want to use a teller. Transactions are only free if they are done over the web or at an ATM machine. Because we no longer have these face-to-face experiences, the interactions are either human-human (mediated by a computer), or human-computer. The ability to determine the credibility of these interactions is therefore paramount. This is especially true as the severity and importance of the task increases. As transactions increase
from purchasing a t-shirt to purchasing a car, purchasing a home, or choosing which medical information to follow, the importance of establishing credibility also increases.
For the purposes of this paper, online information will be divided into three categories: one-one (e-mail and instant messaging), one-many (web pages), and many-many (online communities). One-one communication is when individuals or organizations are sending an e-mail or instant
message to another individual or small group. The communication is personalized. It is only meant for specific recipients. One-many
communication is when an individual or organization posts a web page, which has information that is not targeted towards a specific individual, but rather any and all individuals who can access the web site. Many-many communication is when any number of individuals or organizations are both the source and the recipient of information. Online communities, which include tools such as listservers, chat rooms, blogs, and bulletin boards, are the most common form of many-many communication. While this is a simple classification scheme for web-based information, it can help illuminate some of the differences in how people perceive the
credibility of information, and how information can be presented in a way that is more credible.
2.1 E-Mail and IM
E-mail is the most personal form of web-based communication. People accept an e-mail as your word. For many, e-mail is their primary form of business communication. Organizations communicate with their clients or members using e-mail. Family members keep in touch using e-mail. One study from 1985 says that e-mail increases the number of people whom you regularly communicate with by a factor of 10 (Hiltz & Turoff, 1985). By the year 2005, it might be possible that e-mail increases the number of regular communication partners by a factor of 100. Instant messaging (IM) has also become important, as a shorter, quicker way of communicating with others. While e-mails are asynchronous, instant messaging is synchronous. The obvious downside of IM is that both users must be logged in and at their computer at the same time. In addition,
communication partners must be using the same IM application. While there are a number of differences between e-mail and instant messaging (Nardi, Whittaker, & Bradner, 2000), they are outside of the scope of this paper.
Simply put, some people rely on e-mail and IM as their main form of communication. However, the credibility of e-mail can sometimes be unclear. You sometimes receive e-mails from names that you know, but
the e-mail somehow doesn’t seem to be from the sender. In the physical world, people rely on face-to-face interactions, or interactions over the phone, to determine the credibility of their communication partner. There are many aspects of communication that are considered when determining face-to-face credibility, such as vocal intonations, facial expressions, body language, and proxemics. On the phone, we can still rely on vocal
intonation, emotion in the voice, loudness, or whispering in the voice. If you say, “She’s a good friend,” depending on the vocal intonation or hints of sarcasm in the voice, there can be different intended messages. All of these aspects of communication (vocal intonations, facial expressions, proxemics) can help support the development of trust between
communication partners (Feng, Lazar, & Preece, 2004). And all of these aspects of communication are more effective than plain text, in developing trust among communication partners (Bos, Olson, Gergle, Olson, & Wright, 2002). However, none of these aspects are present in e-mails or instant messaging. Despite this fact, research has found that individuals tend to be more honest in e-mails than in phone calls or instant messaging (Sussman & Sproull, 1999); (Hancock, Thom-Santelli, & Ritchie, 2004). However, if you don’t know the sender of the e-mail, how can you assess the credibility of what they are saying? How do you decide if you want to open and read the e-mail? Does the subject of the e-mail play a role? The next section discusses how users determine the credibility of incoming e-mail messages.
2.1.1 Determining the Validity of Senders
It is certainly important to determine the credibility of the sender of an e-mail, when it arrives in your inbox. Most e-mails are actually from the sender listed. However, there are a number of situations which could cause false sender information. For instance, if a virus takes over your e-mail program, it could send e-e-mails, replicating the virus, to all those in your addressbook. These e-mails are sent from the e-mail program of the sender listed, although the sender did not write those e-mails. Another example is when an individual is able to hack an e-mail account and
essentially perform the same function as a virus. The e-mail is coming from the e-mail program of the sender listed, but in reality, the sender did not write the e-mail. If the recipient of the e-mail knows the sender personally, they will usually be able to tell if the e-mail is valid, due to the style and content of the message (Metzger, Flanagin, Eyal, Lemus, & McCann, 2003). When unsure, a follow-up phone call can confirm the validity of the message. If the recipient of the e-mail message does not know the sender, it can be harder to determine the validity of the message.
Another topic related to the credibility of e-mails is spam. Spam is an issue that regularly makes the headlines. Unwanted e-mails, encouraging you to refinance your mortgage, purchase medicine online, make parts of your body bigger, and recover $35 million dollars from your relative in Nigeria, fill e-mail inboxes on a regular basis (Preece et al., 2003). These are clearly non-credible mails, and users can determine this simply by the e-mail subject line. Frequently, the sender’s e-e-mail address also gives away the fact that an e-mail is not credible. However, what about e-mails that could be credible, and could be spam? For instance, one commonly sent e-mail appears to be from Washington Mutual Bank, noting that there have been problems with the user’s bank account. If the user does not have an account at Washington Mutual, this is an easy fraud to identify. However, it is much harder for the Washington Mutual customer to determine the credibility of this message. How should the user act in a situation like this, where it is unclear whether the e-mail is credible or not? Probably the best approach is for the consumer to actually call the individual or organization involved.
2.1.2 Users with Disabilities and Spam
The validity of e-mail is an especially great problem for users with
disabilities. For users with disabilities, spam and non-credible e-mail poses a greater problem than for typical users. Whereas typical users without disabilities can just see an e-mail in preview format, determine that it’s spam, and delete it, this is not possible for a user with a disability. For instance, a blind user might need to hear a good part of the e-mail (using a screen reader) to determine if it is credible or not. The cost of dealing with spam, in terms of time, is very high. Other disability populations face similar challenges, and spam just makes their interactions with the computer that much harder.
2.1.3 Tools for Assessing the Credibility of E-mail
A popular way to address e-mail credibility (especially spam) is e-mail filtering (either organizational or personal), however, this technology is imperfect. At the organization level, spam filters may keep out e-mails with certain keywords or from certain domains. At a personal level, spam filters may also keep out e-mails from certain individuals. A stronger level of spam protection is when only e-mails from certain individuals (on a “white list”) will make it through to the inbox (Preece et al., 2003). For users with disabilities, these approaches are risky. To have a high level of spam protection will save time, but important e-mails may be deleted or
overlooked. To have a lower level of spam protection, more e-mails may make it through, but time will be wasted dealing with the spam.
How could you determine if incoming e-mails are credible? There are new features that could be added to e-mail programs to address credibility of incoming e-mails. Consider the e-mail message in figure 1. It asks the user to verify their account information on ebay.com. The credibility is already questionable. However, there is one hint that this e-mail is not credible and is very likely spam. The e-mail address that the message was sent from indicates ebay.com. Yet, the link that the user is requested to click on (where it says “click here to verify your account”) is actually directing the user to a non-ebay domain in Germany. A user would only note this if they closely examined the link (by placing their mouse on top of the link to determine the target URL). This is a hint that this is spam. A feature could be added to an e-mail program that could check e-mails, such as this one, that come from unknown individuals, and look to see if the stated e-mail address matches the domain that is listed as a link.
Another potential tool for assessing e-mail credibility is a linguistic parser. If, for instance, e-mails regularly come from a friend, there will be a
documented history of the friend’s writing style. An e-mail program could store general information about the writing style, and note to users when the writing style from that individual changes drastically. This could signify to the user that an e-mail was automatically generated by a virus, or was written by a hacker. With notification, it would be up to the user to examine the message more carefully to determine credibility. In addition, in the future, digital signatures might be used to help establish that the sender of an e-mail is really who they say that they are.
2.2 Web Pages
There are literally billions of web pages. The problem isn’t a limited amount of information, but rather, too much information. Sifting through what’s on the web can be a challenge. Which information is credible, and which information is not? There are many implications: health information on the web can come from multiple sources. Which sources do we trust?
Businesses want us to enter our data (for reservations, purchasing
products, etc.) on web sites, rather than in person or over the phone. Can we reliably enter our data? Governmental agencies want for us to pay our taxes and register for government permits online. Are these the actual sites, or frauds, representing themselves as governmental agencies? If we know that information is credible, there are many benefits to all of these interactions. But how do we know if web sites are credible? The next sections will discuss the credibility of information on the web. There are really two separate, but related topics: 1) how do sites make themselves appear to be credible? and 2) how do users sort through and determine which sites are credible?
2.2.1 What Determines Whether a Site is Viewed as Credible?
There are a number of different approaches to making a web site appear credible. Good interface design may lead to an appearance of credibility. However, while users may primarily evaluate credibility based on the graphical appeal of a web site, in reality, users evaluate the credibility of information based on multiple dimensions (Fogg, 2003). However, it’s important to note that design features do not automatically mean that a site is authentic and credible, rather that it has the APPEARANCE of credibility. There are many design features that help support the perception of
the perceived credibility of a web site: 1) conveying the real world aspect of the web site sponsor, 2) making sure that the web site is easy to use, 3) including markers of expertise, 4) including markers of trustworthiness, and 5) providing a personalized user experience (Fogg et al., 2001b). Two design features seem to detract from the perception of credibility: overly commercial elements, and design features that make the site appear amateurish (Fogg et al., 2001b). In a follow-up study, Fogg et. al. determined that the design look of the site has the greatest impact on credibility (Fogg et al., 2003). The “design look,” in that study, included topics such as visual layout, white space, and use of appropriate color schemes (Fogg et al., 2003). In addition, banner ads tend to decrease the perception of credibility, while information on the author of the content (both a biography and/or pictures) can help increase the perception of credibility (Fogg et al., 2001a). Testimonials from well-known sources can also help increase the perception of credibility (Ceaparu, Demner, Hung, Zhao, & Shneiderman, 2002).
While not a part of the design look, an important interface issue related to credibility is the response time. A number of research studies have
examined response time, that is, how long it takes a web site to respond to a user request. The findings generally state that the shorter the response time, the higher the perceived credibility of the site. An increased response time changes the user’s perception of the quality of the content (Jacko, Sears, & Borella, 2000), and whether the content is interesting (Ramsay, Barbesi, & Preece, 1998). In fact, a long response time can change the user’s perception of the organization that sponsors the web site, and even impact on the perception that a transaction is secure (Bouch, Kuchinsky, & Bhatti, 2000). While the user’s local connection speed is outside of the control of the site designer, the designer can attempt to minimize response time by keeping web page file sizes low (Lazar, 2006, in press).
Users are likely to feel that the information provided by an organization is more credible if the organization providing information in cyberspace is an organization that the user is familiar with in the physical world (Lazar & Sears, 2005, in press). This is obviously something that develops over time. Experience with a physical organization is developed over a number of years, and cannot be added overnight with a software application or a development tool. For instance, many users trust the web sites of
newspapers, when they are familiar with those newspapers in print over a long period of time (Metzger et al., 2003). A newspaper’s reputation and credibility does not just develop overnight, but in other cases, it can be helpful if a web site can be designed so that it highlights the long-term history of the physical organization. Similarly, the web sites of neutral
government organizations, such as the National Institutes of Health, are perceived as having more credibility than web sites for hospitals such as Johns Hopkins Hospital, which may be more concerned with their rankings than accuracy, and Intelihealth.com, which is owned by Aetna, a for-profit HMO (Luo & Najdawi, 2004).
Another way to make a site appear credible is to use some sort of certification seal (Metzger et al., 2003). For instance, a number of e-commerce sites use certification seals to convince users that their computer networks are secure (Diller, Lin, & Tashjian, 2003). These certification seals include Verisign and TrustE. The Better Business Bureau offers a certification seal, known as BBBOnline
(http://www.bbbonline.org/) that certifies that the company sponsoring the web site follows the BBB Code of Ethics. The BBB also offers a separate certification seal related to the handling of personal information.
Sometimes, credit card seals (such as those from Visa or Mastercard) can also help create the appearance of credibility (Diller et al., 2003). Users using these sites can be sure that the site is secure and that their
information will be protected.
A number of certification seals exist for determining the credibility of health information on the web. These programs, including URAC
(http://www.urac.org/) and Hi-Ethics (http://www.hiethics.com/), help establish the credibility of health related web sites (Luo & Najdawi, 2004). Other types of certification seals can relate to site accessibility. Some sites use the “BOBBY-approved” symbol (see http://bobby.watchfire.com). This symbol informs users that the site has passed Bobby certification for
accessibility. Other sites may state that they are accessible, as certified by the National Federation of the Blind certification program (see
http://www.nfb.org/seal/intro.htm).
Another way to promote credibility is to include a statement of some type, detailing a policy of the web site. For instance, the most popular type of policy is a privacy policy, where the site will clearly state how information collected on the site is being used (Milne & Culnan, 2002). Another type of policy is the universal usability policy, which states how usable the site is for diverse user populations (such as users using assistive technology, old browsers, or slow connection speeds) (Hochheiser & Shneiderman, 2001).
There are a number of ways that users evaluate credibility of the
information that appears on web sites. Users tend to evaluate credibility by using the design appeal of the web site (Fogg et al., 2003). This is not necessarily the best methodology for evaluating credibility, however, it is one of the most frequent approaches used. Certification symbols, such as those discussed in previous sections, can help users establish credibility, however, in reality, this only transfers the need to determine credibility from the web site to the sponsor of the certification symbol. Users must
determine if they feel that the certification symbol itself is credible. If the user has long-term experience with the sponsor of the web site, this might encourage the user to believe that the information provided on the web site is credible. Users may perceive a higher level of credibility if it has a
privacy policy (Fogg et al., 2001b). Non-design related aspects of credibility that can be evaluated by users included accuracy, bias, and completeness of information (Metzger et al., 2003). Finally, users may feel that credibility is in question if information presented is either out-of-date or has misspellings.
Health information is probably the most important and serious type of web information that users look for. The consequences for using non-credible information are the highest when looking for health information (Luo & Najdawi, 2004). It is therefore disturbing to note that, in a 2002 study from the Pew Internet and American Life Project, only 25% of those who acquire health information online always check to determine the credibility of the information source (Fox & Rainie, 2002). This can be problematic,
especially considering that one study found that less than 50% of the health-related information available on the web has been reviewed by medical doctors for accuracy (Fox & Rainie, 2000). When users actually do check to ascertain the credibility of information, they check the source, date, and privacy policy of a web site. Another study found that, as a health crisis increased in severity, users were actually more likely to evaluate health information to determine the credibility of that information (Sillence, Briggs, Fishwick, & Harris, 2004). Overly commercial elements of a web site tend to “turn off” the users and make them question the
credibility of the health information. These users typically find the health information using a search engine, which, again, can be problematic (see section 2.2.5 on search engines).
2.2.3 Tools to Assess the Credibility of Web Pages
There are a number of tools that can be used to help users assess the credibility of information on the web. One way for users to evaluate the credibility is to use a list of questions developed by content experts (Fritch, 2003). When users access a web site that they are not previously familiar with, they can use these series of questions, relating to topics such as accuracy and currency of information. Examples of credibility evaluation question lists are available at:
http://www2.widener.edu/Wolfgram-Memorial-Library/webevaluation/webeval.htm
and
http://www.lib.purdue.edu/ugrl/inst/evaluationchart.pdf
Of course, no set of questions would fit the needs of every web site, and the responsibility for determining the credibility falls on the user. Some more focused question lists have been developed for specific topics, such as the DISCERN questionnaire (http://www.discern.org.uk/), which focuses specifically on health information for consumers (Fritch, 2003). In addition, a questionnaire can be presented while the user is actually utilizing a site. A tool called the Information Quality tool (http://hitiweb.mitretek.org/iq/) is essentially a pop-up window that asks questions of the user, related to the credibility of the site, while the user is utilizing that site. After users have responded to all of the questions, giving details about the web site, a score is calculated by the IQ tool, rating the credibility of the web site examined (Fritch, 2003).
The limitation here is that all of these methods for determining credibility are based on information provided by either the site sponsor, or based on the previous experience or knowledge of the user. What if such information is lacking, or the user does not have the necessary experience or
knowledge?
One solution is for users to utilize a pre-screened list of sites. A series of sites might be pre-screened by librarians or other content experts to determine the credibility of the information (Fritch, 2003). One example of such a site is the Librarian’s Index to the Internet (http://www.lii.org). The limitation here is that this is only feasible if the type of content that the user is interested in has been included in the pre-approved list. While the LII is a very general resource list, more specific lists exist, where sites have been examined and pre-approved. For instance, http://www.ds-health.com/ is a web site run by Dr. Len Leshin. It provides articles on Down Syndrome and links to other sites which have been pre-screened by Dr. Leshin. Since
this isn’t a site like the National Institutes of Health, or the New York
Times, which would already have a certain level of assumed credibility, this site (See figure 2) provides a number of features to help establish
credibility. For instance, it offers a certification seal. It notes the
professional credentials of the site sponsor. It also provides a biography of the site sponsor (noting that he is both a pediatrician and a parent of a child with Down Syndrome).
Figure 2. DS-Health, a site with a number of features to establish credibility Reputation systems (also known as recommender systems) are another way that users, based on the experience and knowledge of others, can evaluate the credibility of information, products, services, or individuals on a web site. Reputation systems are areas of a web site where users can post their perceptions and experiences with information, products,
individuals, or organizations (Preece, 2000); (Chen & Singh, 2001); (Jensen, Davis, & Farnham, 2002). The concepts of credibility and trust are increasingly being added to reputation systems (O'Donovan & Smyth, 2005). The idea is that if other users provide their feedback, it can help establish the credibility of information, or let the user know that certain information or a certain individual is not credible. This information can be as simple as a numerical rating, or can be complex, rating multiple aspects of credibility or providing lengthy feedback. This type of reputation
manager has been applied most often in e-commerce companies. This is especially important when there are various parties involved in a
transaction (such as those on Ebay), where the parties will never meet, and the intentions and credibility of the strangers must be established (Shneiderman, 2000). An example of Ebay’s reputation manager is in figure 3. However, this approach can also be applied to establishing the credibility of information on a web site. For instance, users can comment on how credible information is on a health web site. In 2004, the Pew
Internet and American Life Project found that 26% of adult Internet users in the U.S. had used a reputation system to rate a product, service, or person (Rainie & Hitlin, 2004).
Figure 3. The reputation manager from EBay
2.2.4 Site Accessibility for Users with Disabilities (Does Accessibility=Credibility?)
There are many resources available to help developers, designers, and webmasters make their web sites accessible. There are guidelines (such as the Web Content Accessibility Guidelines, http://www.w3.org/wai), tools (such as A-Prompt, InFocus, or Ramp), and even laws to encourage the development or accessible sites (Section 508 in the US,
http://www.section508.gov). Despite that, most web sites continue to be inaccessible (Loiacono & McCoy, 2004; Lazar, Beere, Greenidge, &
that has not followed design guidelines, and does not have the back-end equivalents needed, so that it can be successfully used by individuals with disabilities. There are ramifications of accessibility, related to the credibility of information.
From the discussions in previous paragraphs, it is clear that visual
presentation of web content helps users to determine whether information is credible. What happens when a user is visually impaired? All of that presentation, which can influence the perception of credibility, is absent. In its place, accessibility, the ability to access content, the time spent making sites accessible, is what leads to a perception of credibility. For users with disabilities, the sites that are viewed as most credible are the sites that are accessible. Sites that are inaccessible are viewed as the least credible (and since the information is not possible to ascertain, the credibility is irrelevant). From personal experience, users with disabilities tend to view the information that they can access (on an accessible web site) as credible information. Users with disabilities return again and again to the sites where they know that they will not experience problems accessing the information. These sites, in turn, become the sites that are viewed as credible. Similar to how graphical design influences the perception of credibility, accessibility also influences the perception of accessibility, although neither of these are necessarily the best ways to ascertain the credibility of web-based information.
2.2.5 How Do Search Engines Impact on Credibility?
As previously discussed, users frequently do not evaluate the credibility of the information that they find on the web. This is especially true with
search engines. For instance, a Pew Internet and American Life Study found that only 38% of users actually are aware that there is a difference between search results that are sponsored (and paid for) and those that are not sponsored (Fallows, 2005). Among those 38% who are aware of the differences, only half can always tell which search results are
sponsored and which are not. Despite this, 64-73% of users (depending on frequency of usage and experience) feel that search engines are a fair and unbiased source of information (Fallows, 2005). Aside from the difference between sponsored and unsponsored links, there are also techniques that are used by webmasters to “trick” the search engine into ranking a specific site higher than others (Lynch, 2001). Therefore, the results of a search engine are, at all times, questionable.
2.3 Online Communities
Online communities consist of multiple individuals interested in a specific topic, sharing resources, and communicating using group computer-mediated communication tools (Preece, 2000). These tools include listservers, chat rooms, bulletin boards, and blogs.
Online communities vary on the amount of face-to-face contact among community members (Lazar & Preece, 1998). The amount of face-to-face contact may impact on the credibility of information. Some communities, such as those based on physical communities such as the Blacksburg Electronic Village or the Seattle Community Network, often have frequent face-to-face contact among community members. Some communities are based on periodic face-to-face contact, such as those related to religious communities, professional communities, and sports communities.
Community members may meet face-to-face at annual events. Other communities, such as those related to healthcare, emotional support, or roleplaying, tend to have no face-to-face contact (Lazar & Preece, 1998). The amount of face-to-face contact can impact on credibility, since with frequent face-to-face contact, users might be familiar with other community members and be able to judge the credibility of the information that is posted by those specific members.
The major factors in an online community that can help users determine the credibility of information are registration, moderation, identification of expertise, and reputation feedback. These types of approaches fall under the umbrella of what is known as “sociability” (Preece, 2000). Sociability, unlike usability, focuses on what is needed to help ensure that an online community is successful as a social place. Sociability details include
moderation, registration rules, rules on posting, rules on flaming, and rules on lurking.
2.3.1 Tools for Determining the Credibility of Information in Online Communities
Registration can be the first tool used to assess credibility. In some online communities, registration is only allowed to people with certain credentials. For instance, in one online community for anesthesiologists, only
credentialed, certified anesthesiologists were allowed to register, post messages, or even read posts (Preece, 2000). Membership in the community itself established credibility. In another online community for folk music, an e-mail, establishing the user’s identity, as well as reasons for wanting to join, were required before posting was allowed.
Another tool that can influence credibility in an online community is strong moderation. If a communication tool (such as a listserver, newsgroup, or bulletin board) is strongly moderated, it means that an expert in the topic of study must approve all posts. This role could be considered similar to someone like a librarian who is a content expert, and reviews and
approves content, to ensure the credibility of information. Moderators are especially important in online communities related to health care
information, as they can help separate informed, credible advice, from other advice that is simply someone’s opinion.
One step further to this approach is to make clear the source of the
information. On a bulletin board related to athlete’s foot, for instance, there should be a clear distinction between the informed advice of a podiatrist, the experiences of someone suffering with athlete’s foot, and a
representative of a pharmaceutical company that sells a cream to treat athlete’s foot. Even noting, as some bulletin boards on
http://www.webmd.com do, which information is coming from a licensed source, and which information is not coming from a licensed source, could be helpful. Figure 4 displays a screenshot from WebMD, where it is noted in the screen name that the person posting is a WebMD staff member.
Figure 4. A screenshot from WebMD, where a message post is identified as coming from a WebMD staff member
Another example is an online community/digital library known as CalFlora, which helps track plant growth and distribution in California (VanHouse, 2002). Data can be added by any individual across the state, who can record their reports of where various plants and fauna have been found. The challenge is to determine the skill level of those reporting the plant growth, as many plants look similar. To address this issue of credibility, the CalFlora asks users, when they register, to report their bio credentials, their institutional affiliation, specific expertise, and experience level (VanHouse, 2002). The idea is that based on that data, other users and scientists can make judgments about the credibility of the information, based on the skill level of the individual that added the information. Reputation systems are also useful to help establish what information is credible within an online community. Reputation systems can help establish what information other users found credible or useful. More information on reputation systems is available in previous sections. It should be noted that the nature of most online communities are to be places of free and open speech. If community members want to keep it that way, then it should be clear from the start that the credibility of information within the community (especially in a health-related
community) is questionable. As long as it is clear that the credibility of information is questionable, users can make informed decisions on the use of that information.
3.0 Implications
Given the previous discussion, possible next steps are presented for various stakeholders. These stakeholders include browser designers, web site developers, users, and librarians.
3.1 Implications for Browser Designers
More features need to be added to web browsers, which can address information credibility. Currently, web browsers include many features to highlight issues related to security. For instance, there is usually a padlock or other icon that indicates that you are using a secure site. A dialog box pops up, noting that you are about to enter a secure site (or that you are about to leave a secure site). There is no reason that similar features,
related to web credibility, could not be developed. For instance, rather than a stand-alone tool (the Information Quality tool described earlier in the paper), a short pop-up, which allows users to evaluate the credibility of information, could be integrated into the browser. Since there are a
number of known certification seals for information credibility, the browser could alert the user when they are going to a certified credible site, and when they are leaving a certified credible site. An icon could be added, similar to the padlock icon that indicates a secure site, and the icon could indicate when the user is visiting a credible site.
3.2 Implications for Web Site Developers
In this paper, suggestions for highlighting the credibility of a web site were presented. Like most usability guidelines, the guidelines are complex, and will not be read by large numbers of people who are actually developing web sites. To help with this effort, those leading the web credibility effort should take a page from those focusing on web accessibility. Knowing that most people would never read their 50+ page guidelines document on web accessibility, the Web Accessibility Initiative (http://www.w3.org/wai)
created a set of 10 “quicktips” for making a web site accessible. These quicktips fit on a business card, and they are short (even shorter than typical usability heuristics), to the point, and can be understood by a wide range of information technology professionals. These cards are given away for free, and up to 500 cards can be ordered at no charge. In my personal opinion, I think that this is one of the most useful documents ever created related to web usability. I therefore propose that a short list of “quicktips,” which address design features that highlight whether a site is credible, be developed and distributed by the American Library Association to thousands of web site developers.
As part of these quicktips, developers should be encouraged to incorporate a number of features into their web sites. Certainly, privacy policies and noting the source and date of all information can help establish the credibility of information. In addition, where possible, reputation systems should be incorporated into web sites. When users can add information to the web site, there should be some feature where they can note their expertise and experience, which can be used to help evaluate the credibility of the information that they post.
3.3 Implications for Users
Users need to have clear tools that help them evaluate the credibility of information on the web. In earlier sections, I proposed adding design
features into web browsers, that help users evaluate the credibility of web sites, such as pop-up tools, and new dialog boxes and interface widgets. I also propose that, similar to the quicktips described for web developers, that a similar set of quicktips be developed for users. Since users will have to evaluate multiple web sites for credibility on a regular basis, I suggest limiting the number of quicktips to five. Again, these quicktips for
evaluating web sites can be distributed via business cards. I propose the following 5 heuristics for assisting users in determining the credibility of a web site:
1. Is the site sponsor clear, and would it be considered unbiased? 2. Is the source of the specific information clear, and would it be considered a knowledgeable source?
3. Is there a certification seal?
4. Is the date that the information was created and/or posted clear?
5. Have outside parties (librarians, knowledgeable experts, recommender communities, etc.) established the credibility of information on the site? Another issue for users is determining the credibility of incoming e-mails. While the press has not done a sufficient job of educating the public about issues related to the credibility of web-based information, I feel that the press has done a sufficient job of educating users about the credibility of e-mails. Due to the spread of viruses via e-mails, it has become a public concern. Many users know not to open file attachments from those that they don’t know. The press frequently reports events related to new
viruses and/or worms, and tells the public what keywords, phrases, or hints to watch out for. Users are savvy about assessing the credibility of
incoming e-mails.
3.4 Implications for Librarians
Based on the research described in the paper, it is clear that users DO need assistance in determining the credibility of web-based information. Efforts such as the Librarian’s Index to the Internet are helpful. The
number of similar efforts needs to be expanded. This is not to be confused with censorship. Censorship occurs when users are limited in what they can see. Users can still visit whichever sites they prefer, however, when librarians help sort sites by information quality, users then have the option of visiting sites that are more credible. And, of course, librarians will need to continue with their Herculean effort to educate users about the topic of web credibility.
4. Summary
Web credibility is a multi-dimensional problem. While this paper focuses on the tools and technologies, other papers at this symposium focus on users, institutions, and government policies. Only when attention is paid to all four aspects of this problem can solutions truly be found. If users want to
determine the credibility of a web site, the web site sponsor will also need to be interested in proving their site credibility. If a web site sponsor wants to make clear that their web site is credible, the user must spend the time to evaluate the credibility. Institutions, such as libraries and schools, must help to educate users about web credibility. Some tools do exist now to help users evaluate credibility, such as spam filters, evaluation lists, and certification seals. In the future, new approaches for evaluating credibility, such as built-in browser features, and language parsers, will need to be developed. Those hoping to deceive users on the web have taken the first steps. It’s time for those interested in establishing credibility on the web to catch up.
5. References
Bos, N., Olson, J., Gergle, D., Olson, G., & Wright, Z. (2002). Effects of four computer-mediated communications channels on trust
development. Paper presented at the ACM Conference on Human Factors in Computing Systems (CHI), 135-140.
Bouch, A., Kuchinsky, A., & Bhatti, N. (2000). Quality is in the eye of the beholder: meeting users' requirements for Internet quality of service. Paper presented at the ACM Conference on Human Factors in Computing Systems (CHI), 297-304.
Ceaparu, I., Demner, D., Hung, E., Zhao, H., & Shneiderman, B. (2002). In Web we trust: establishing strategic trust among online customers. In R. Rust & P. Kannan (Eds.), E-Service. Armonk, NY: E. Sharpe Publishers, 90-107.
Chen, M., & Singh, J. (2001). Computing and using reputations for internet ratings. Paper presented at the ACM Conference on E-Commerce, 154-162.
Diller, S., Lin, L., & Tashjian, V. (2003). The evolving role of security, privacy, and trust in a digitized world. In J. Jacko & A. Sears (Eds.), The Handbook of Human-Computer Interaction (pp. 1213-1225). Mahwah, NJ: Lawrence Erlbaum Associates.
Fallows, D. (2005). Search Engine Users: Internet searchers are confident, satisfied and trusting- but they are also unaware and naive.
Available at: http://www.pewinternet.org/report_display.asp?r=146. Pew Internet and American Life Project.
Feng, J., Lazar, J., & Preece, J. (2004). Empathy and online interpersonal trust: A fragile relationship. Behaviour and Information Technology, 23(2), 97-106.
Fogg, B. (2003). Motivating, Influencing, and Persuading Users. In J. Jacko & A. Sears (Eds.), The Handbook of Human-Computer Interaction (pp. 359-370). Mahwah, NJ: Lawrence Erlbaum Associates.
Fogg, B., Marshall, J., Kameda, T., Solomon, J., Rangnekar, A., Boyd, J., & Brown, B. (2001a). Web credibility research: A method for online experiments and early study results. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHi), 295-296.
Fogg, B., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A., Shon, J., Swani, P., & Treinen, M. (2001b). What makes web sites credible? A report on a large quantitative study. Paper presented at the CHI 2001: Human Factors in Computing, 61-68.
Fogg, B., Soohoo, C., Danielson, D., Marable, L., Stanford, J., & Tauber, E. (2003). How do users evaluate the credibility of web sites?: a study with over 2,500 participants. Paper presented at the
Proceedings of the 2003 Conference on Designing for User Experiences, 1-15.
Fogg, B., & Tseng, H. (1999). The Elements of Computer Credibility. Paper presented at the ACM Conference on Human Factors in Computing Systems (CHI), 80-86.
Fox, S., & Rainie, L. (2000). The Online Health Care Revolution: How the Web helps Americans take better care of themselves. Available at: http://www.pewinternet.org/report_display.asp?r=26. Pew Internet and American Life Project.
Fox, S., & Rainie, L. (2002). Vital Decisions: How Internet users decide what information to trust when they or their loved ones are sick. Available at: http://www.pewinternet.org/report_display.asp?r=59. Pew Internet and American Life Project.
Fritch, J. (2003). Heuristics, tools, and systems for evaluating Internet information: helping users assess a tangled Web. Online Information Review, 27(5), 321-327.
Hancock, J., Thom-Santelli, J., & Ritchie, T. (2004). Deception and design: The impact of communication technology on lying behavior. Paper presented at the ACM Conference on Human Factors in Computing Systems, 129-134.
Hiltz, R., & Turoff, M. (1985). Structuring computer-mediated communication systems to avoid information overload. Communications of the ACM, 28(7), 680-689.
Hochheiser, H., & Shneiderman, B. (2001). Universal usability statements: Marking the trail for all users. Interactions, 16-18.
Jacko, J., Sears, A., & Borella, M. (2000). The effect of network delay and media on user perceptions of web resources. Behaviour and
Information Technology, 19(6), 427-439.
Jensen, C., Davis, J., & Farnham, S. (2002). Finding others online:
Repuation systems for social spaces online. Paper presented at the ACM Conference on Human Factors in Computing Systems, 447-454.
Lazar, J. (2006, in press). Web Usability: A User-Centered Design Approach. Boston: Addison-Wesley.
Lazar, J., Beere, P., Greenidge, K., & Nagappa, Y. (2003). Web
Accessibility in the Mid-Atlantic United States: A Study of 50 Web Sites. Universal Access in the Information Society, 2(4), 331-341. Lazar, J., & Preece, J. (1998). Classification schema for online
communities. Paper presented at the 1998 Association for
Information Systems Americas Conference, Baltimore, Md., 84-86. Lazar, J., & Sears, A. (2005, in press). Design of E-Business Web Sites. In
G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics. Hoboken, NJ: John Wiley & Sons.
Loiacono, E., & McCoy, S. (2004). Web site accessibility: an online sector analysis. Information Technology and People, 17(1), 87-101.
Luo, W., & Najdawi, M. (2004). Trust building measures: A review of consumer health portals. Communications of the ACM, 47(1), 109-113.
Lynch, C. (2001). When documents deceive: trust and provenance as the new factors for information retrieval into tangled web. Journal of the American Society for Information Science and Technology, 52(1), 12-17.
Metzger, M., Flanagin, A., Eyal, K., Lemus, D., & McCann, R. (2003). Credibility for the 21st Century: Integrating perspectives on source, message, and media credibility in the comtemporary media
environment. In P. Kalbfleisch (Ed.), Communication Yearbook 27 (pp. 293-335). Mahwah, NJ: Lawrence Erlbaum Associates.
Milne, G., & Culnan, M. (2002). Using the Content of Online Privacy
Notices to Inform Public Policy: A Longitudinal Analysis of the 1998– 2001 U.S. Web Surveys. The Information Society, 18(5), 345-360. Nardi, B., Whittaker, S., & Bradner, E. (2000). Interaction and outeraction:
Conference on Computer-Supported Cooperative Work (CSCW), 79-88.
O'Donovan, J., & Smyth, B. (2005). Trust in recommender systems. Paper presented at the ACM 2005 Conference on Intelligent User
Interfaces, 167-174.
Preece, J. (2000). Online Communities: Designing Usability , Supporting Sociability. New York: John Wiley & Sons.
Preece, J., Lazar, J., Churchill, E., DeGraaff, H., Friedman, B., & Konstan, J. (2003). Spam, spam, spam, spam: how can we stop it?. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems, 706-707.
Rainie, L., & Hitlin, P. (2004). The Use of Online Reputation and Rating Systems. Available at:
http://www.pewinternet.org/report_display.asp?r=140. Pew Internet and American Life Project.
Ramsay, J., Barbesi, A., & Preece, J. (1998). A psychological investigation of long retrieval times on the World Wide Web. Interacting with Computers, 10(1), 77-86.
Shneiderman, B. (2000). Designing trust into online experiences. Communications of the ACM, 43(12), 57-59.
Sillence, E., Briggs, P., Fishwick, L., & Harris, P. (2004). Trust and mistrust of online health sites. Paper presented at the ACM Conference on Human Factors in Computing Systems (CHI), 663-670.
Sullivan, T., & Matson, R. (2000). Barriers to use: Usability and Content Accessibility on the web's most popular sites. Paper presented at the ACM Conference on Universal Usability, 139-144.
Sussman, S., & Sproull, L. (1999). Straight talk: Delivering bad news through electronic communication. Information Systems Research, 10(2), 150-166.
VanHouse, N. (2002). Trust and epistemic communities in biodiversity data sharing. Paper presented at the ACM 2002 Joint Conference on Digital Libraries, 231-239.