Developing the discrete choice experiment
About 25 out of 10,000 people with ocular hypertension will develop
severe glaucoma in 10 years if not treated.
About 25 out of 10,000 people with ocular hypertension will develop visual impairment in 10 years if not treated.
FIGURE 60 Graphical explanation of risk of developing glaucoma, severe glaucoma and visual impairment given to FIGURE 59 Describing the clinical condition. Images reproduced with permission from the Belgian Glaucoma Society
(http://www.glaucoma.be/en/index.html). (continued)
In other words:
Now In 10 years’ time…
About 25 will develop visual impairment About 1600 people will develop glaucoma About 180 will develop severe glaucoma Of 10,000 people with high eye pressure
Willingness-to-pay question
Now, I would like you to imagine the best monitoring programme. For some of you it is going to be at the hospital, being seen by an ophthalmologist, or it might be going to an optometrist every 6 months. Imagine the service that you really like, the service that you are really comfortable with. Then, obviously, the question is how much you might be prepared to pay for that type of service.
The monetary value of a service was a difficult concept to understand for the participants. Initially, participants expressed reluctance to pay for the service. However, after it was explained that this was an exercise to infer value, and that money is one way of looking at value, they engaged with the exercise.
Willingness to pay was based on previous experience with eye services or other health services in Scotland. Other factors considered when giving a value were a person’s own health risk perception, financial circumstances, family history, previous experiences (good or bad) and current diagnosis (all suggesting engagement with the exercise). For example:
How much is the dentist? Is the dentist about £15? The same as the dentist.
(P4)
I would have paid it. I would have paid the £40 because my £40 gave me peace of mind.
(P3)
Something happened to my mother and in the light of today and prices and things, to cover National Health, I think I would be prepared to pay what about £40 for an hour’s consultation, once a year.
(P5)
Well if you’re wanting a figure, I would be comfortable in my wallet with £50.
(P1)
I think we’re giving a kind of philosophical stun point here and the issue is not I think – to put a value on what you’d pay for a consultation then you need to know I guess you would maybe be prepared to pay a percentage of the cost of the consultation inclusive of everything involved, whatever that comes out to for a consultation. If it comes out to for a number £100 yes I would happily pay 10/20/30%, but without knowing the cost of the work then I can’t really put a figure on it but perhaps the question should be ‘would you pay or would you not pay’ and I would pay – ‘. . . because of the situation I am in I would pay but it would be with extreme reluctance.
(P2)
Based on this discussion participants indicated values ranging from £15 (P4) to £50 (P1), with intermediate values of £30 (P2) and £40 (P3, P5). Such values provide a guide for levels of the cost attribute. However, given the small sample size of the focus groups, the levels for the cost attribute were extended beyond the maximum stated within the focus group, with an upper range of £70.
The final set of attributes and levels are summarised in Table 26.
Determining choice sets
The six attributes with four levels and one attribute with two levels resulted in 8192 possible choice sets (i.e. 46× 21). Experimental design techniques were employed to reduce the number
TABLE 26 Summary of the monitoring attributes and levels
Attribute included in the DCE Value for each level included in the DCE No monitoring alternative
Risk of developing glaucoma Number of people out of 10,000 developing glaucoma in 10 years
740; 960; 1190; 1410 1600
Risk of developing severe glaucoma Number of people out of 10,000 developing severe glaucoma in 10 years
25; 60; 100; 130 180
Risk of developing visual impairment Number of people out of 10,000 developing visual impairment in 10 years
2; 6; 10; 15 25
Unwanted effects of treatment None – means that you have not noticed any discomfort or difficulties Some – means that you have noticed occasional discomfort or difficulty Quite a lot – means that you are aware of these discomforts or difficulties most of the time
Severe – means that you need or think you need additional treatment to control one or more of these difficulties
None
Communication and understanding of information provided by the health professional
Made me feel at ease and made sure I understood the purpose of monitoring Made me feel at ease but did not make sure I understood the purpose of monitoring
Did not make me feel at ease but made sure I understood the purpose of monitoring
Did not make me feel at ease and did not make sure I understood the purpose of monitoring
Not applicable
Location Hospital eye clinic
Local optician
No testing
Cost per year (£) 15; 30; 50; 70 No cost
of choice sets to a more manageable number while still being able to infer WTP for all possible configurations. More specifically, SAS version 9.1.2 software244 (SAS Institute Inc., Cary, NC, USA) was employed to generate a main-effects D-efficient design, ensuring that uncertainty around parameter estimates was minimised (by minimising the determinant of the covariance matrix).245 This resulted in 32 choice sets.
An opt-out ‘no monitoring’ alternative was added to each of the 32 choice sets to allow individuals to choose the realistic option of not being monitored. The levels for the opt-out option were defined in consultation with experts in the field. An example of a choice set is shown in Figure 61. Respondents were presented with an example of a choice set before proceeding to the task themselves. The order of the choice questions presented for each participant was randomly generated.
Additional information in the questionnaire
In addition to the 32 choices derived from the experimental design, four rationality tests were added to test whether respondents were engaged in the experiment. The rationality tests applied Sen’s contraction test.246,247 Here, individuals were initially presented with a choice set involving three choices (monitoring service A, monitoring service B and no monitoring service). This choice was then repeated later in the questionnaire, with the choice set contracted to two options. These two options depended on what they chose initially, with the option they chose and one of the other options from the original three choices being presented. The individual should choose the same option when the choice set is contracted. The tests were included at four points within the questionnaire (choices 2 and 5, choices 11 and 15, choices 20 and 26 and choices 28 and
34). This allowed testing of whether or not including a larger than average number of choices increased the cognitive burden or task complexity for respondents.248–250
If respondents ‘fail’ such tests, the question arises of what to do with them. Although deletion of such responses is common,240 Lancsar and Louviere251 argue that such deletion may be inappropriate because such responses may be valid, and random utility models are robust to errors made by individuals in forming and revealing preferences. Decisions on what to do with those who ‘failed’ were based on discussions once responses were analysed. Given that the rationality checks were not derived from the experimental design, the additional contracted questions were not included in the econometric analysis (because adding choices to the data matrix would compromise the statistical properties of the experimental design).
Before completing the DCE the following characteristics of the participants were collected: gender, age, region of UK, level of education (secondary school, college, university, none of the above and other), socioeconomic status (level of income per household and number of habitants) and experience of eye testing. Following the choices information was collected on general health, using the European Quality of Life-5 Dimensions (EQ-5D) (www.euroqol.org).
The full questionnaire is available from the authors.
Pre-piloting
The questionnaire was pre-piloted opportunistically among 10 members of staff from the University of Aberdeen. Comments on the format of the questionnaire were given in terms of the ordering of the different sections. One participant argued that having the ‘easier’ questions concerned with personal characteristics at the beginning of the questionnaire would help to ease respondents into the rest of the survey. Other useful comments related to the format were: ‘useful to have an access link to the explanation of each characteristic and its levels throughout
FIGURE 61 Example of a DCE question.
Which monitoring service would you choose? (please tick one box below)
Monitoring service A Monitoring service B No monitoring service
Number of people out of 10,000 developing glaucoma in
10 years 740 1410 1600
Number of people out of 10,000 developing severe glaucoma in
10 years 25 130 180
Number of people out of 10,000 developing visual impairment in
10 years 2 15 25
Unwanted effects of treatment None Severe None
Communication and understanding of information
provided by the health
professional
Made me feel at ease and made sure I understood the
purpose of monitoring
Did not make me feel at ease and did not make sure I
understood the purpose
of monitoring
Not applicable
Place of testing Hospital eye clinic Local optician No testing
Cost per year £15/year £30/year No cost
Service A Service B No monitoring
the choice section in the same screen shot’, ‘provide a hyperlink to the example choice as a guide for answering the choices’, ‘highlight key words to emphasise meaning and capture attention’. Most people agreed that pictorial representation of disease stages together with the graphical risk explanation made information more understandable. Changes were made to the format of the survey accordingly.