Web-based Supplementary Materials
“Continual Reassessment Method for Partial Ordering”
by Nolan A. Wages, Mark R. Conaway, and John O’Quigley
Web Appendix A: Further details for matrix orders
In this section, we provide some further details about the simulation results presented in Section 4.3. Specifically, Table 1 reports the distribution of the percentage of MTD recommendation across each dose combination, whereas Table 2 provides the distribution of allocation across each dose. Through examination of these tables, we can see that the method is treating a large percentage of patients at and around the MTD combinations. In fact, even in cases when POCRM performs less well, the method is recommending doses around the MTD. For instance, in scenario 5, the MTD combinations are only recommended at the conclusion of 38% of the simulated trials. However, doses with true toxicity probabilities between the target, 0.40, and 0.48 are recommended at the end of 86% of the simulated trials. Furthermore, 63% of enrolled patients were treated at doses with true DLT probabilities between 0.40 and 0.48. Similarly, in scenario 6 the percentage of correct MTD recommendation is 45%, yet 88% of the simualted trials resulted in a dose with a true probability between 0.40 and 0.49 being selected. Moreover, 75% of enrolled patients were treated at doses with probabilities between these two values.
Table 1: Percent of MTD recommendation based on 2000 simulated trials. Dose Drug A Level 1 2 3 4 5 1 2 3 4 5 Scenario 1 Scenario 2 4 0.00 0.03 0.00 0.00 0.02 0.05 0.00 0.00 3 0.13 0.28 0.04 0.00 0.13 0.24 0.05 0.00 2 0.00 0.02 0.25 0.20 0.00 0.05 0.24 0.16 1 0.00 0.00 0.03 0.02 0.00 0.00 0.03 0.03 Scenario 3 Scenario 4 Drug B 3 0.03 0.27 0.05 0.00 0.00 0.08 0.00 0.00 0.00 0.00 2 0.00 0.01 0.30 0.01 0.00 0.01 0.54 0.06 0.00 0.00 1 0.00 0.00 0.02 0.29 0.02 0.00 0.01 0.16 0.14 0.00 Scenario 5 Scenario 6 4 0.02 0.00 0.00 0.00 0.00 0.07 0.01 0.00 0.00 0.00 3 0.13 0.02 0.00 0.00 0.00 0.16 0.10 0.01 0.00 0.00 2 0.17 0.18 0.02 0.00 0.00 0.06 0.14 0.14 0.00 0.00 1 0.04 0.21 0.17 0.04 0.00 0.00 0.04 0.15 0.12 0.02
Table 2: Percent of dose allocation based on 2000 simulated trials.
Dose Drug A Level 1 2 3 4 5 1 2 3 4 5 Scenario 1 Scenario 2 4 0.02 0.04 0.01 0.00 0.03 0.05 0.01 0.00 3 0.13 0.19 0.10 0.01 0.13 0.17 0.09 0.02 2 0.01 0.05 0.20 0.14 0.01 0.07 0.17 0.14 1 0.00 0.01 0.04 0.04 0.00 0.01 0.05 0.05 Scenario 3 Scenario 4 Drug B 3 0.05 0.22 0.08 0.00 0.00 0.14 0.04 0.01 0.00 0.00 2 0.00 0.02 0.24 0.04 0.00 0.03 0.31 0.10 0.01 0.00 1 0.00 0.00 0.05 0.23 0.06 0.01 0.02 0.19 0.12 0.01 Scenario 5 Scenario 6 4 0.06 0.01 0.00 0.00 0.00 0.08 0.03 0.00 0.00 0.00 3 0.12 0.05 0.01 0.00 0.00 0.11 0.11 0.03 0.00 0.00 2 0.13 0.12 0.06 0.01 0.00 0.05 0.11 0.12 0.03 0.01 1 0.07 0.14 0.12 0.06 0.01 0.02 0.06 0.11 0.11 0.03
Table 3: Simulation results for POCRM for 5×5 and 6×6 matrix orders Scenario 1 2 3 4 5 6 % recommendation 60.0 59.0 32.0 53.0 50.0 26.0 % of toxicities 31.6 27.5 30.6 27.9 28.7 33.0 % treated at MTD 42.0 45.0 24.0 40.0 38.0 20.0
Web Appendix B: Higher orders
In this section, we present some simulation results similar to those provided in Section 4.3 of the manuscript in order to illustrate the performance of the proposed design for some higher order matrices (i.e. 5×5 and 6×6). Table 6 lists the true probability of toxicity for each of these scenarios, with the target MTD combination indicated in bold type. In scenarios 1 through 3, there are 25 treatment combinations and in scenarios 4 through 6, there are 36 treatment combinations. We again began by choosing the three simple dose-toxicity orders given in Section 3.3 of the manuscript according to the diagonals of the matrix of treatments Figure 1. Under all scenarios, 2000 trials were simulated for a fixed sample size of n = 50 patients. The target toxicity rate is given in bold type in Table 6 and we utilize a uniform prior distribution, p(m), on the ordering. We generated skeletons according to the approach of Lee and Cheung (2009).
Table 3 reports the percentage of correct MTD recommendation, the percentage of observed toxicities and the percentage of patients treated at the MTD for the POCRM. Overall, the simulation results indicate that, in terms of identifying the MTD, the performance of the POCRM is satisfactory for higher orders. The method is recommending and treating patients at the MTD in a large percentage of trials. Furthermore, the simulation results again support the idea that the “correct” ordering does not need to be one of those chosen for investigation in order for the method to perform well.
Table 4: Toxicity scenarios for 5×5 and 6×6 matrix orders Doses 1 2 3 4 5 6 1 2 3 4 5 6 Scenario 1 Scenario 2 5 0.45 0.56 0.64 0.72 0.80 0.19 0.30 0.42 0.50 0.60 4 0.30 0.42 0.52 0.62 0.70 0.10 0.19 0.30 0.41 0.50 3 0.20 0.30 0.40 0.50 0.60 0.09 0.12 0.20 0.30 0.40 2 0.12 0.20 0.30 0.40 0.50 0.08 0.09 0.12 0.20 0.30 1 0.05 0.10 0.20 0.30 0.40 0.06 0.08 0.10 0.12 0.20 Scenario 3 Scenario 4 6 0.10 0.20 0.30 0.40 0.50 0.60 5 0.40 0.50 0.60 0.70 0.80 0.08 0.14 0.20 0.30 0.40 0.60 4 0.30 0.42 0.52 0.62 0.70 0.06 0.12 0.15 0.20 0.30 0.40 3 0.10 0.20 0.30 0.42 0.50 0.04 0.10 0.13 0.16 0.20 0.30 2 0.05 0.12 0.19 0.30 0.40 0.03 0.08 0.12 0.14 0.18 0.20 1 0.04 0.06 0.08 0.12 0.20 0.02 0.06 0.10 0.12 0.15 0.18 Scenario 5 Scenario 6 6 0.10 0.30 0.40 0.50 0.60 0.65 0.72 0.76 0.80 0.83 0.86 0.91 5 0.08 0.14 0.20 0.30 0.41 0.60 0.68 0.72 0.75 0.79 0.83 0.87 4 0.06 0.12 0.15 0.30 0.40 0.60 0.65 0.68 0.70 0.72 0.79 0.83 3 0.04 0.10 0.13 0.16 0.20 0.30 0.30 0.42 0.52 0.62 0.70 0.80 2 0.03 0.08 0.12 0.14 0.18 0.20 0.20 0.30 0.40 0.50 0.60 0.75 1 0.02 0.06 0.10 0.12 0.15 0.18 0.02 0.06 0.12 0.20 0.30 0.50
Table 5: Percent of MTD recommendation based on 2000 simulated trials. Doses 1 2 3 4 5 6 1 2 3 4 5 6 Scenario 1 Scenario 2 5 0.03 0.00 0.00 0.00 0.00 0.07 0.16 0.06 0.00 0.00 4 0.13 0.04 0.00 0.00 0.00 0.00 0.03 0.12 0.04 0.00 3 0.05 0.15 0.06 0.00 0.00 0.00 0.00 0.01 0.14 0.06 2 0.00 0.06 0.17 0.05 0.00 0.00 0.00 0.00 0.03 0.17 1 0.00 0.00 0.07 0.15 0.04 0.00 0.00 0.00 0.00 0.09 Scenario 3 Scenario 4 6 0.00 0.11 0.21 0.07 0.00 0.00 5 0.11 0.02 0.00 0.00 0.00 0.00 0.00 0.02 0.07 0.02 0.00 4 0.07 0.14 0.02 0.00 0.00 0.00 0.00 0.00 0.03 0.06 0.06 3 0.00 0.06 0.13 0.02 0.00 0.00 0.00 0.00 0.00 0.02 0.19 2 0.00 0.00 0.06 0.12 0.03 0.00 0.00 0.00 0.00 0.00 0.11 1 0.00 0.00 0.00 0.06 0.16 0.00 0.00 0.00 0.00 0.00 0.00 Scenario 5 Scenario 6 6 0.01 0.19 0.13 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 5 0.00 0.01 0.04 0.04 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 4 0.00 0.00 0.01 0.05 0.06 0.00 0.05 0.00 0.00 0.00 0.00 0.00 3 0.00 0.00 0.00 0.00 0.04 0.22 0.19 0.08 0.00 0.00 0.00 0.00 2 0.00 0.00 0.00 0.00 0.01 0.15 0.02 0.06 0.14 0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.09 0.38 0.00 0.00
Table 6: Percent of dose allocation based on 2000 simulated trials. Doses 1 2 3 4 5 6 1 2 3 4 5 6 Scenario 1 Scenario 2 5 0.03 0.00 0.00 0.00 0.00 0.08 0.12 0.07 0.01 0.00 4 0.13 0.04 0.00 0.00 0.000 0.01 0.03 0.11 0.04 0.01 3 0.06 0.10 0.06 0.01 0.00 0.00 0.01 0.03 0.10 0.06 2 0.02 0.06 0.10 0.05 0.01 0.00 0.00 0.01 0.04 0.12 1 0.01 0.02 0.06 0.12 0.06 0.00 0.00 0.00 0.03 0.10 Scenario 3 Scenario 4 6 0.01 0.10 0.15 0.08 0.01 0.00 5 0.10 0.03 0.01 0.00 0.00 0.01 0.01 0.02 0.05 0.02 0.01 4 0.06 0.09 0.05 0.01 0.00 0.01 0.00 0.01 0.02 0.05 0.06 3 0.01 0.06 0.09 0.04 0.01 0.00 0.00 0.00 0.01 0.02 0.15 2 0.01 0.01 0.06 0.09 0.04 0.00 0.00 0.00 0.00 0.01 0.12 1 0.00 0.00 0.01 0.08 0.13 0.00 0.00 0.00 0.00 0.01 0.03 Scenario 5 Scenario 6 6 0.02 0.15 0.13 0.04 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 5 0.01 0.02 0.03 0.05 0.00 0.00 0.02 0.00 0.00 0.00 0.00 0.00 4 0.00 0.00 0.01 0.03 0.05 0.03 0.12 0.01 0.00 0.00 0.00 0.00 3 0.00 0.00 0.00 0.02 0.03 0.15 0.12 0.06 0.01 0.00 0.00 0.8000 2 0.00 0.00 0.00 0.00 0.01 0.16 0.04 0.06 0.08 0.00 0.00 0.01 1 0.00 0.00 0.00 0.00 0.01 0.03 0.01 0.03 0.14 0.22 0.02 0.02
Web Appendix C: Alternative Approach
This section investigates an alternative approach that accounts for model uncertainty by con-sidering multiple models (orderings) when recommending a dose. As in the manuscript, the key idea for this method is that each of the possible simple orderings consistent with the partial order can be thought of as a model. We assume that there are M possible orderings of the toxicity probabilities for the dose levels that are all being considered. Instead of selecting the ordering with the greatest posterior density and reducing the problem to a standard phase I trial, this approach estimates the dose to recommend to the next entered patient for each distinct ordering and uses the dose that is most agreed upon across all possible orders. That is, it works with the cumulative probabilities for some MTD given the set of models.
Consider the simple example where there are four possible orderings. Suppose after j
patients have been enrolled,π(1|Ωj) =π(2|Ωj) =π(3|Ωj) = 0.2 recommendsd3andπ(4|Ωj) =
0.4 recommends d4. According to POCRM, dose level d4 will be recommended to the next
patient becauseπ(4|Ωj) is the maximum of all the weights, disregarding the fact that the most
agreed upon dose is d3. This approach measures the cumulative probability and selects the
dose that is most agreed upon across all possible orderings. For the simple example described above, the proposed method will choose d3 instead of d4 as the dose to be given to the next
patient. In the method, the cumulative probability for eachdi is the sum of the posterior model
probabilities across all orderings which indicate that the recommended dose is di. Then, we
choose the dose level that has the highest cumulative probability. This method for finding the maximum tolerated dose takes into account all the possible orderings and does not limit itself to one particular ordering.
To illustrate this alternative approach, consider the example given in Section 2.1 involving
k = 6 discrete treatment combinations, d1, . . . , d6. The partial order associated with this trial
has associated with it five possible simple orders. Under all scenarios, 2000 trials were simulated for a sample of n = 25 patients. The target toxicity rate is θ = 1/5 and we incorporated a
Table 7: Percentage of MTD recommendation %tox R 0.15 0.20 0.38 0.52 0.70 0.80 CRM 0.37 0.48 0.15 0.01 0.00 0.00 0.251 POCRM 0.41 0.41 0.12 0.06 0.00 0.00 0.255 Alternative approach 0.42 0.43 0.09 0.06 0.00 0.00 0.259 R 0.04 0.07 0.20 0.35 0.55 0.70 CRM 0.01 0.17 0.61 0.21 0.01 0.00 0.234 POCRM 0.02 0.22 0.49 0.25 0.02 0.00 0.224 Alternative approach 0.02 0.26 0.46 0.24 0.02 0.00 0.232 R 0.00 0.00 0.02 0.07 0.22 0.41 CRM 0.00 0.00 0.00 0.23 0.65 0.12 0.205 POCRM 0.00 0.00 0.14 0.16 0.58 0.12 0.174 Alternative approach 0.00 0.00 0.16 0.12 0.60 0.12 0.193 R 0.02 0.11 0.20 0.35 0.58 0.70 CRM 0.01 0.23 0.55 0.20 0.01 0.00 0.230 POCRM 0.03 0.27 0.43 0.26 0.01 0.00 0.227 Alternative approach 0.03 0.33 0.40 0.23 0.01 0.00 0.235 R 0.12 0.22 0.30 0.45 0.55 0.70 CRM 0.30 0.42 0.25 0.03 0.00 0.00 0.250 POCRM 0.32 0.39 0.17 0.11 0.01 0.00 0.252 Alternative approach 0.33 0.39 0.15 0.12 0.01 0.00 0.254
uniform prior distribution, p(m). Once again, the probability of dose-limiting toxic response is modeled via the power parameter model with the skeletons generated via the method of Lee and Cheung (2009). Table 7 indicates that the simulation results of the cumulative probability approach are competitive with that of POCRM, making it a practical alternative as a multiple-agent trial design. However, this only gives a snapshot comparison to their method because of the few toxicity scenarios considered.
