Methods of Incorporating Serendipity

The section above introduces the setup process for the filtering environment. The next important issue in this study is how to incorporate serendipity in the filtering environment. In order to reduce the impact of ranking on serendipitous recommendation, we adopt a fixed position in the presented list to introduce serendipity. This means that our study does not test placement location. Placement is not an independent variable in this study.

Figure 3.3 shows how articles are presented to users in this study. In order to clarify the whole process of article retrieval and presentation, an example is given here. Let us assume a user selects 8 out of 30 medical topics to build system’s profile. These selected topics (SP topics) are ranked by the strength of user’s interest on each topic and are arranged in descending order. The other 22 medical topics (NSP topics), which are not selected by the user, are presented to the user by serendipitous recommendation. The filtering process involves the stages of article retrieval and presentation. In the retrieval stage, one article is selected from the dataset for each SP topic. As a result, 8 articles are retrieved from the

experimental dataset. In the presentation stage, only the top 5 articles are listed according to the rank of their topics due to the limitation in total number of presented articles. Then, one article from a NSP topic is added below each article on SP topics, generating a total of 10 articles shown to the user in each session (see Figure 3.3).

Figure 3.3: Article List Presented to Users in MedSDFilter System

From the description above, topic-X in Figure 3.3 is selected from all NSP topics. Presenting the articles on the NSP topics represents this study’s strategy for incorporating serendipity. In this context, how to pick topic-X from all NSP topics in each session becomes

especially important. Based on the article list described in Figure 3.3, three methods for

picking topic-X are introduced respectively. In order to double check users’ interests, topic-X is presented repeatedly for 2 times in consecutive sessions. In other words, users view 2 different articles on each topic-X. Below, we describe the three methods that introduced serendipity in this study.

1. Article on SP topic-1

2. Article on NSP topic-X (serendipity incorporated)

3. Article on SP topic-2

4. Article on NSP topic-X (serendipity incorporated)

5. Article on SP topic-3

6. Article on NSP topic-X (serendipity incorporated)

7. Article on SP topic-4

8. Article on NSP topic-X (serendipity incorporated)

9. Article on SP topic-5

1) RA Method (Based on Randomness)

RA method is the simplest solution of incorporating serendipity since it relies on randomness. It works as a baseline for comparing the other methods of incorporating serendipity. When RA method was implemented in this study, topic-X in every location was randomly selected from all NSP topics across all the sessions.

2) KA Method (Based on Knowledge of Topic Association)

In KA method, the topic-X is not randomly selected from all NSP topics. Instead, the topic-X is selected through analyzing its association (to be described later) with the SP topic above it in the presentation. For instance, as for the topic-X in the second article in Figure 3.3, all NSP topics are sorted by their association with topic-1. Based on this rank, these sorted topics are sequentially presented below topic-1 over all the 10 sessions. When KA method was implemented in this study, the NSP topic of the highest rank was selected as topic-X first, followed by that of the second highest rank.

For implementing the procedures above, it is important to establish topic associations. This study adopted manual methods instead of any technical solution in evaluating topic association because such an evaluation involves many contextual factors and should represent the real life clinical situations well. Two experienced physicians were involved in this study. Their long-term experience in the area of healthcare ensures that the data obtained about health topic association are valid. In the implementation, these two physicians were invited to fill out a survey that lists 435 topic pairs (based on 30 topics). As for each pair, the physicians were asked to select an integer value between 0 and 3 to best describe its association, based on their professional experience and knowledge. Here ‘3’ indicates the strongest association and ‘0’ means no association. In order to minimize judgment error

across topics, the physicians were requested to adopt the same rating standards for all of these topic pairs when they conducted the evaluation. It is highly challenging to achieve this goal, considering the fact that a large number of judgments have to be made by each physician. As a result, a short range of topic associations (0-3, not 0-5 or more) was adopted to compress the range of selectable options for a judgment, making the workload of evaluation manageable. After the survey, the data obtained from the physicians were compared and analyzed through statistical methods. Based on average value of their judgments on topic association, the KA method was implemented for topic-X selection.

3) KAA Method (Adaptive KA method)

If NSP topics are not relevant to users’ interests but presented repeatedly, users may be distracted by noise introduced. Based on the KA method, the KAA method is developed to resolve the problem of over-presentation of irrelevant NSP topics. The KAA method is similar to KA but it incorporates user feedback representing their level of interest. In order to display more relevant NSP topics to users in a limited number of sessions, a NSP topic is excluded if it has been presented to users multiple times and users have low or no interest. In this study, two articles on topic-X (one NSP topic picked for presenting) were shown to users in consecutive sessions. If users were not interested in these articles, the topic-X was not shown to users again until all other NSP topics were presented.

To summarize, the 3 methods adopt different resources and strategies for incorporating serendipity in filtering settings. RA method uses a random mechanism. KA method employs physician’s knowledge on topic associations. KAA method uses physician’s knowledge on topic associations as well as user’s feedback on topic relevance. Based on

these methods, the influence of serendipity was analyzed in the study. Additional details of system implementation are discussed in the next section.