M od el checking

A more formal way is norm ally required for the selection of th e best am ong com peting models. Obviously in the class of models fitted, some are nested w ithin others, which implies th a t a likelihood ratio approach for choosing between them is valid. B ut if a selection is required between a linear an a non-linear m odel then the final decision should be based on a criterion th a t rewards a good fit b u t punishes for model complexity. One such criterion is th e Akaike’s Inform ation C riterion (AIC) defined as follows:

A I C = —2/ri(likelihood) -f 2(num ber of estim ated param eters)

A lthough AIC has received some criticism as a m odel selection tool (especially in the tim e series literatu re), it is still the m ost pop u lar criterion used by p rac­ titioners. In a Bayesian analysis, th e posterior d istrib u tio n of AIC is evaluated and model choice is based on a sum m ary statistic of th a t distrib u tion . In w hat follows the posterior mean of AIC calculated from th e last 1000 M CM C runs is

com pared am ong com peting models and sum m arized in Table (3.12),

Table 3.12: Posterior Mean of AIC

M2 M il M12 MO

Linear -238.705 -240.433 -239.853 -242.096 Non-linear -238.693 -240.115 -240.876

The model which has the lowest value of AIC is selected as th e best one. Following th a t principle th e model w ith no carry-over term s (MO) is th e preferable one. For model M1 2 the non-linear fit gives slightly b e tte r results when com pared to its lin­

ear counterpart. T he reverse argum ent is true for models M i l and M2. In th e lin­ ear case AIC gives the following model ordering: MO -< M i l -< M 12 -< M2, while

in the non-linear context we get the more sound result MO -< M 12 -< M i l -< M 2. The operator -< m eans th a t the model on the left hand side provides a b e tte r fit com pared to the right hand side one. In conclusion the non-linear approach gives sensible results for treatm en t effect irrespective of the type of residual term (if any) fitted in the model. Moreover it tends to provide accurate outcom e during the model selection process.

3.9

C onclusions

In the 2x2 cross-over trial, the performance of various treatm en t estim ators (CROS, PAR, TS) has been studied in some detail. In sum m ary CROS should be the preferable tre a tm e n t estim ator, no m atter if carry-over is included or not in the model. T he alternative (TS procedure), where CROS is selected w ith pro b a­ bility p and PA R w ith probability 1 — p, should be avoided, because it has lower power and higher MSE when com pared to CROS. If th e analyst insists in using the two stage procedure th en one can replace th e original scheme w ith a new one in which, th e sizes of the tests for carry-over and tre a tm e n t difference are set so th a t the overall size of th e procedure is 5%. U nfortunately the improved plan does not perform b e tte r when com pared to CROS in term s of power, or MSE. This investigation leads to the conclusion th a t TS procedure should be gradually abandoned by the analyst of the cross-over experim ent.

Based on a representative example of a cross-over trial in asthm a, b o th Bayesian and Frequentist analysis suggest th a t carry-over is very unlikely to be present in a well-planned trial. The use of baselines or covariates hardly affect our con­ clusions ab o u t treatm en t difference, although th eir incorporation m ight increase precision for inferences about carry-over effect. In conclusion m agnitude and s ta n d a rd error of treatm en t difference are affected by th e presence of carry-over term s. T reatm ent effect tends to be statistically u n im p o rtan t when carry-over is incorporated in th e final model, while in the absence of it tre a tm e n t difference is highly significant. The trialist should carefully investigate th e potential for pharm acological carry-over and choose th e appropriate length of the wash-out p eriod for elim inating such an effect. Once this precaution has been taken the analysis model should not include carry-over term s of any kind.

3.10

B U G S and S + code used for th e derivation