The design of the DCE plays a critical role because it determines how the attributes and their levels are combined to form **choice** sets. One common method is to use fractional factorial **designs** (FFDs) to construct DCEs; see Street and Burgess ( 2007 ) and Bush ( 2014 ), among others. These **designs** are based on a starting design that is either a full factorial or an FFD, for which the entries represent the ﬁrst option in each **choice** set. Generators are then added component-wise to the starting design to form the remaining options in each **choice** set. These methods are ﬂexible for constructing DCEs for estimat- ing main eﬀects only. Because both main eﬀects and two-factor interactions can jointly determine whether DCEs are successfully used to accurately assess real-life decision- making processes, **designs** that can also accurately estimate two-factor interactions are more desirable. Our work in this article focuses on constructing more eﬀective **designs** for estimating main eﬀects and two-factor interactions simultaneously.

Show more
23 Read more

15 Read more

• Evaluation of individual variables for each agent. Some variables are examined in more detail, i.e., not just **minimum**/average/maximum values over all agents are recorded, but the specific value for each agent. This is done for variables such as degree of capacity utilization in the warehouse or of production, and also allows the recognition of interesting patterns in the simulation results. This approach uses CSV files as well and was also applied to track a fitness variable which will be presented in the method section. • Individual agent evaluation. This approach traces some randomly selected agents in detail over the whole simulation period. Almost all important data about an agent is stored in an XML file for each simu- lated point in time, and thus permits further evaluation with a separate evaluation program. For example, this data enables an understanding of the reasons why each and every incoming or outgoing request was accepted or rejected. This approach is especially valuable if bugs are to be traced back to their source. • Visual evaluation. It is possible to run the simulation program with a GUI that includes a map containing

Show more
22 Read more

H Comparison:
Single binary (SB) – First questions of PoolRB (CrossRB1)
All **choice** sets were created using a Bayesian D-efficient design (Bliemer et al. 2008). Bayesian D-efficient **designs** are statically efficient **designs** (see, for example, Ferrini and Scarpa 2007; Rose and Bliemer 2008; Rose et al. 2008). Statistically efficient **designs** aim to maximize the amount of obtained information. A commonly used measure to express the global level of efficiency is the D-error, which minimizes the determinant of variance-covariance matrix. The smaller the D-error, the more statistically efficient is the design. Therefore, a statistically efficient design can be used to increase efficiency while holding the sample size fixed. The Bayesian D-efficient **designs** (100 Halton draws) used in this study are developed based on the calculation of the Db-error of randomly selected **designs** (10,000 iterations). Attribute levels are randomly assigned to each attribute in each **choice** set of the change options while accounting for attribute balance. The base level (zero cost option) is held constant but included in the design process. Priors ware obtained from pilot studies targeting the population of Sydney and Canberra 20 . Following a suggestion of Rose and Bliemer (2005), the rows and columns related to the constant term are excluded from the calculation of the Db-error in order to avoid the dominance of the unproportionally large standard errors of the constant. Dominant and redundant **choice** sets are removed through restrictions and swapping of attribute levels marginally reducing the Db-efficiency (3%). The Bayesian D-efficient **designs** are developed for multinominal logit (MNL) models without accounting for covariate effects. Estimating different models may alter the design efficiency (Rose and Bliemer 2005). The results

Show more
34 Read more

Methods This paper systematically reviews health-related DCEs published between 2009 and 2012, using the same database as the earlier published review (PubMed) to obtain citations, and the same range of search terms. Results A total of 179 health-related DCEs for 2009–2012 met the inclusion criteria for the review. We found a continuing trend towards conducting DCEs across a broader range of countries. However, the trend towards including fewer attributes was reversed, whilst the trend towards interview-based DCEs reversed because of increased computer administration. The trend towards using more flexible econometric models, including mixed logit and latent class, has also continued. Reporting of monetary values has fallen compared with earlier periods, but the proportion of studies estimating trade-offs between health outcomes and experience factors, or val- uing outcomes in terms of utility scores, has increased, although use of odds ratios and probabilities has declined. The reassuring trend towards the use of more flexible and appropriate DCE **designs** and econometric methods has been reinforced by the increased use of qualitative methods to inform DCE processes and results. However, qualitative research methods are being used less often to inform attribute selection, which may make DCEs more susceptible to omitted variable bias if the decision framework is not known prior to the research project.

Show more
20 Read more

7. Discussion
The **choice** of experimental points is an important issue in planning an efﬁcient computer experiment. Various authors have suggested intuitive goals for good **designs**, including “good coverage”, ability to ﬁt complex models, many levels for each factor, and good projection properties. At the same time, a number of different mathematical criteria have been put forth for comparing **designs**. We have proposed here a new criterion, based on the alias matrix for a simple model, and compared it to three criteria that have been advocated by other researchers, the integrated mean squared error, the design entropy and **minimum** distance criteria. We have compared these criteria with respect to ability to guide the **choice** of sample size and to produce good **designs** with respect to lack of replication and projective properties. We also compared several classes of **designs** using the criteria..

Show more
17 Read more

14 Read more

5 Conclusion
This study provides an overview of the applications and methods used by DCEs in health. The use of empirical DCEs in health economics has continued to grow, as have the areas of application and the geographic scope. This study identi- fied changes in the experimental design (e.g. more frequent use of D-efficient **designs**), analysis methods (e.g. mixed logit models most frequently used), validity enhancement (e.g. more diverse use of internal validity checks), quali- tative methods (e.g. upwards trend of qualitative methods used for attribute and level selection) and outcome measures (e.g. coefficients most frequently used). However, a large number of studies not reporting methodological details were also identified. DCEs should include more complete infor- mation, for example, information about design generation, blocking, model specification, random-parameter estimation and model results. Developing reporting guidelines specifi- cally for DCEs might positively impact quality assessment, increase confidence in the results and improve the ability of decision-makers to act on the results. How and when to integrate health-related DCE outcomes into decision-making remains an important area for future research.

Show more
26 Read more

Perhaps the most signiﬁcant ﬁnding of this study is that different cost vectors can affect decision strategies employed by **choice** experiment survey respondents. Statistically signiﬁcant differences in attribute non-attendance (AN-A) for the cost attribute point to differences in **choice** behaviour in response to different cost vectors. Differences in AN-A could have implications for WTP estimates, but we believe that further investigation of such effects requires a better understanding of reasons that underpin AN-A and that explain differences between alternative approaches to identify AN-A incidence. We also ﬁnd signiﬁcant differences across cost vector treatments for a decision strategy that might reﬂect unwillingness to trade-off cost and other non-cost attributes: always choosing the cheapest alternative when a non-status quo alternative was selected. Our results suggest that varying use of this decision strategy can obscure effects of cost vectors on marginal WTP estimates present for the part of the sample that did not exhibit these decision strategies. Indeed, after omitting respondents who use this decision strategy from the sample, we ﬁnd a monotonic increase as the magnitude of the cost vector increases, and signiﬁcant differences between all cost vector treatments for all of the attributes. This suggests that in moving forward, research investigating cost vector effects should take a more detailed look at decision strategies used. This can be considered as non-trivial: many different strategies may be used by respondents, and impacts of these on a naïve model (that does not account for them and might thus suffer from misspeciﬁcation bias) might counteract systematic differences in preferences and WTP found for respondents who make trade-offs as assumed by random utility maximisation models. As an aside, our results suggest that efﬁcient **designs** may not always be the ideal experimental design **choice** in the presence of non-compensatory strategies such as always choosing the cheapest alternative. This aspect deserves further investigation.

Show more
21 Read more

Additionally to the effect on the data by simply including such an opt-out option, our study results indicate that **choice** behavior changes which influences DCE results when respondents are given the opportunity to opt-out. Including an extra **choice** option automatically implies reduced effectiveness, as there are more answering categories included. Specifically an opt-out option does not provide any insight on attribute level trade-offs. This is not an issue, if the **choice** to opt-out is due to the low perceived personal utility of the other scenarios. However, our analysis showed that it is likely to assume that a considerable number of respondents chose to opt-out for other reasons than a dislike for lifestyle programs. It can therefore be suggested that including an opt-out option in a DCE, leads to an ‘unnecessary’ loss of effectiveness. This is of special interest in the light of designing DCEs in an efficient manner (e.g., by minimizing D-error). Such **designs** strive to create **choice** sets with an optimal utility balance between the scenarios of each **choice** task, by optimizing the variance- covariance matrix (22–24). This designing procedure results in a DCE that requires a minimal number of **choice** tasks per respondent and a minimal number of respondents per experiment (aside from model specifications (e.g., level restrictions or interactions)). At the same time, this may induce complexity of the generated **choice** tasks. Since there are indications for higher levels of opt-out when **choice** tasks become more complex, the efficiency of designing DCEs in such a way may be at risk. Future research is necessary to identify subgroups among study popula- tions that are most likely to opt-out due to other reasons than solely personal utility. Moreover, it should be explored how DCEs can be designed in an efficient manner while keeping in mind this phenomenon.

Show more
This chapter reviewed the main issues with the experimental design used to con- struct **choice** **designs** in health economics in general, and for estimating health state utilities within the QALY scale in particular, and identified the main requirements in the experimental **designs** for successful implementation of DCEs to estimate health state utilities. Generally, the review showed that the main design issue is that most of the **choice** **experiments** constructed for valuing health state utilities are generated using orthogonal array **designs** or are based on other required statistical properties such as level balance, and minimal overlap. Restricting the construction of **choice** de- signs to these properties might result in dominant and implausible combinations of the attribute levels that reduces the design efficiency. However, imposing constraints on the attribute level combinations together with other health evaluation design require- ments (particularly including the death state in the **choice** design to anchor health state utilities into the QALY scale) and the complexity of nonlinear design problems, require deviation from orthogonal design principles and more advanced design methods to construct an efficient **choice** design.

Show more
276 Read more

2.2 Experimental **Designs** With/Without Severity‑Stratified Restriction
We implemented heterogeneous DCE design algorithms to create for each study arm a unique experimental design com- prising 168 **choice** tasks, distributed over eight sub-**designs** [ 21 ]. The algorithm optimizes for Bayesian D error for the total design, while simultaneously optimizing for the Bayes- ian D errors of each of the eight sub-**designs**. In essence, this strategy produces a blocked design with eight blocks, where the design within each block is optimized in addition to the optimization of the overall design across blocks. A Latin hypercube sample optimized for maximum **minimum** distance between points and a greedy optimization algo- rithm was used to optimize the weighted averaged Bayesian D error with one-third of the weight assigned to the aggre- gated efficiency and two-thirds on the individual efficiencies of the sub-**designs**. Note that the design algorithm controlled for left–right randomization of the two states by including both options A and B in comparison with option C in the Bayesian design criterion, even though only one of the two **choice** options was presented (in random order) to the survey respondents.

Show more
13 Read more

Although the **experiments** were presented in the early 1990’s, it was not until Marley and Louviere (2005) that mathematical probabilities and properties were for- mally determined and published. Marley and Louviere (2005), Marley et al. (2008), and Marley and Pihlens (2012) provided the probability and properties to best-worst scaling **experiments** for the three cases. Additionally, Lancsar et al. (2013) provided the probability and utility definition for case 3 **experiments** that include the sequen- tial best-worst **choice** from a set of choices. Other work found in the literature with regards to these **experiments** are the design of the **experiments** and dealing with taste heterogeneity. Louviere and Woodworth (1983) stated that orthogonal, main effects, and fractional factorial **designs** provide better parameter estimates than other **designs**. In application to best-worst scaling **experiments**, balance incomplete block **designs** (BIBD)(Louviere et al. 2013; Parvin et al. 2016) and orthogonal main effects plans (OMEPs) are popular **designs** (Flynn et al., 2007; Knox et al., 2012; Street and Knox, 2012). These **designs** and their properties are examined by Street and Burgess (2007). Louviere et al. (2013) looked at the design of **experiments** for best-worst scaling **experiments** and stated that it is possible to determine individual parameter estimates for the respondents.

Show more
136 Read more

2009). These so-called utility-neutral partial profile **designs** are created using or- thogonal **designs** based on linear design principles. To improve the quality of the **designs**, innovative construction methods have been developed based on optimal design theory for linear models (see, e.g., Atkinson et al. 2007). Nevertheless, the most recent utility-neutral optimal **designs** created by Grossmann et al. (2009) only allow for the construction of DCEs with two profiles per **choice** set and two groups of attributes where the number of levels of the attributes is fixed in every group. There are also limitations on the allowable number of constant attributes. This approach is therefore not applicable in a wide variety of practical problems. However, a more fundamental problem with the use of utility-neutral optimal **designs** is that they do not match the **discrete** **choice** models. This is because **discrete** **choice** models are nonlinear in the parameters, implying that the quality of the design of a DCE depends on the unknown parameters (Atkinson and Haines 1996). One justification for the use of the utility-neutral design approach is that the nonlinear design problem can be transformed into a linear one by assuming zero prior parameter values. A key feature of the utility-neutral optimal **designs** is thus that they are optimal for one specific set of parameter values. Therefore, utility-neutral optimal **designs** belong to the class of locally optimal **designs** which are constructed using prior point estimates for the parameters (Huber and Zwerina 1996).

Show more
23 Read more

hypothetical, the alternatives are experienced, not described using vignettes.
BC’s symposium presentation focused on the potential value of experience-based methods for DCE **designs**. In this work, the argument was advanced that decision-makers are likely to be more interested in the views of individuals with a clear understanding of what they are trading (e.g., patients). His presentation provided two examples, asking patients in a chemotherapy clinical to prioritize their symptom relief and asking patients who used two forms of delivery (infusion and infection) in a crossover trial about their preferences (15). The use of experience-based methods has a number of important advantages; in particular that it will reduce hypothetical bias. This has to be balanced against the potential for sample selection bias, small samples for rare alternatives, and a possible incompatibility between preferences of experienced individuals and the general population (16).

Show more
However, the application of this methodology is relatively complex, as the construction of **choice** **experiments** requires the understanding and application of advanced notions in experimental design theory [52]. Although some software programs, such as SAS [74], now provide tools to help researchers construct optimal **designs**, proper experimental design remains a very technical and evolving field that might limit the use of DCE methods. A large literature devoted to **choice** **experiments** has already underlined some of the limitations of certain study **designs** [51], while design constraints or limitations can limit the validity of results obtained from such **choice** **experiments** [66,67,75,76]. The econometric analysis of DCE data also requires fairly advanced statistical methods and there is no consensus in the literature at present on the best models to use [53,77]. Although **choice** experi- ments may be useful in informing decision-making in developing countries, HR researchers should be aware of the technical expertise required to use them, as well as their potential limitations.

Show more
10 Read more

Deterministic and stochastic simulation models both re- quire careful planning and execution of experimental de- sign strategies. Based on the type of computer simulation – deterministic or stochastic – a researcher must make choices with respect to **choice** of design (i.e. space-filling or optimal) and **choice** of model fitting technique (i.e. li- near model or GASP model). This research demonstrated a new way of comparing **designs** for computer simulation **experiments**. Theoretical prediction performance of space-filling and optimal **designs** with respect to the second order polynomial model and the GASP model were illustrated. The theoretical results showed that there was a dominate design in both cases. The I-optimal design dominated the linear model and the GASP IMSE design dominated the GASP model. These results are somewhat intuitive as both of these **designs** are intended to minimize the variance in the design region with respect to the spe- cific model. While the FDS graphical strategy is useful for comparing design types, the FDS plotting capabilities also allow the assessment of the other effects on predic- tion variance such as sample size, dimension, and un- known theta parameters (in the case of the GASP model).

Show more
Apart from the important impact of some socio- demographic variables and the health status, the characteristics connected to the participants' past experiences seem to have the greatest influence on the involvement level. Thus, our data suggest that preferences for a different involvement level could be relatively controllable by the caregivers, considering that, to a large extent, they seem to depend on the attitude of the GP in the previous visits and to the long- term ‘personal’ relationship between a patient and his/her GP, that is different from the occasional contacts with a specialist. Patients, in Italy, are in fact free to choose their GP at the age of 18 and once they make their final **choice** they hardly change it [37]. Consequently, this long term relationship, built over the years and most of the times based on reciprocal trust should not require patients’ total involvement during each consultation. The presence of heterogeneity was also confirmed when taking into account information preferences.

Show more
The concept and importance of tradability may be related to reference-dependent preferences in the area of behavioral economics (Kahneman, 2003). Evidence from lab and field **experiments** shows that the point of reference influences the monetary valuation of a good, sometimes with surprising results. People may have evaluated the job in a single profile case scenario as the only offer in hand, whereas they are unlikely to use as the point of reference any particular one of the three job offers that a multi-profile case scenario presents. Once the job is perceived as given, the non-salary characteristics could increase in monetary value. 19 A test of this explanation could proceed with a random allocation of two contextual prompts to single profile case respondents, which would include one stating that the job is the only available offer and the other stating that it is but one of several, and comparing the preferences elicited by each treatment method with those elicited by the multi-profile case method.

Show more
37 Read more

In this article, I will analyze the different theories about "cult" membership and conversion, specifically focusing upon whether or not conversions to cults ought to b[r]

29 Read more