Survey Length

There was evidence of an interaction between survey length and the number of response options on the pattern of mean IMSE. Figure 26 displays the mean IMSE for each data generation model and analysis model by survey length for items with 4 response options averaged over sample size. Figure 27 displays the mean IMSE for each data generation model and analysis model by survey length for items with 6 response options averaged over sample size. Responses generated under the IRTree Model to 10 item and 20 item surveys with 4 and 6 responses options fit with the

MNRM resulted in the largest mean IMSE. When responses were generated under the IRTree Model, within model mean IMSE was lower for surveys with 10 items than surveys with 20 items for the IRTree Model, the MNRM, and the MPCM. This pattern of mean IMSE differences was consistent for surveys with items containing 4 response options and surveys with items containing 6 response options.

There was no effect of survey length on the mean IMSE for responses to items with 4 options generated under the MNRM and the MPCM as evidenced by the parallel lines. The mean IMSE was equal for responses generated under the MNRM to surveys with 4 response options fit with the MPCM and responses generated under the MNRM to surveys with 4 response options fit with MNRM. The MPCM analysis method resulted in the lowest mean IMSE for responses to 4- option surveys consisting of 10 and 20 items when responses were generated under the MPCM. The IRTree Model resulted in the highest mean IMSE values for 10 item surveys with 4 response options and 20 item surveys with 4 response options when data was generated under the MNRM and when responses were generated using the MPCM.

Surveys with 20 6-category items fit using the MNRM resulted in the largest mean IMSE within each data generation routine. The mean IMSE was equal for responses generated under the MNRM to 10 6-response option item surveys fit with the MNRM and fit with the MPCM. The MPCM analysis method resulted in the lowest mean IMSE for responses to 10 6-option item surveys generated under the MPCM. The IRTree Model analysis method resulted in the highest mean IMSE for responses to 10 6-response option surveys generated under the MPCM.

Figure 26. Mean IMSE for responses to surveys with 10 and 20 items with 4 response options generated under the IRTree Model, the MNRM, and the MPCM and analyzed by the IRTree Model, the MNRM, and the

MPCM.

Figure 27. Mean IMSE for responses to surveys with 10 and 20 items with 6 response options generated under the IRTree Model, the MNRM, and the MPCM and analyzed by the IRTree Model, the MNRM, and the

4.2.5 Number of Response Options

Figure 28 displays the mean IMSE for each data generation model and analysis model for surveys of 10 items with 4 and 6 response options averaged over sample size. Figure 29 displays the mean IMSE for each data generation model and analysis model for surveys of 20 items with 4 and 6 response options averaged over sample size. Overall, the lines slope up to the right indicating the mean IMSE was greater for items with 6 response options compared to items with 4 response options. The mean IMSE under the MPCM was approximately equal to the mean IMSE using the IRTree Model for responses to 10 item surveys generated under the IRTree Model across the levels of response options. When items responses were simulated under the IRTree Model, the MNRM resulted in a higher mean IMSE for surveys of 10 items with either 4 or 6 response options compared to the other two methods. There was no systematic difference in mean IMSE between analysis performed using the MNRM and analysis performed using the MPCM on MNRM simulated responses to 10 items with 4 and 6 response options. Analysis of MPCM generated responses to 10 items with 4 and 6 response options using the MPCM resulted in the lowest mean IMSE compared to the other two analysis methods. Fitting MNRM and MPCM simulated responses to 10 items with 4 and 6 response options with the IRTree Model led to the largest mean IMSE compared to the same simulated responses analyzed with the MNRM or MPCM.

For 20 items with 6 response options, data generated under the MNRM and data generated under the MPCM fit with the MPCM resulted in the lowest mean IMSE. The mean IMSE was larger when responses to 20 items with 6 categories generated under the MPCM were analyzed with the MNRM compared to the mean IMSE when analysis was performed with the IRTree Model. The mean IMSE was smaller when responses to 20 items with 4 categories generated

under the MPCM were analyzed with the MNRM compared to mean IMSE when analysis was completed with the IRTree Model.

Analysis using the MPCM and analysis using the MNRM of responses to 20 item with 4 response options simulated under the MNRM resulted in the lowest mean IMSE. When 20 items with 6 response options generated under the MNRM were considered, the mean IMSE for the MPCM was the least and the mean IMSE for the MNRM was the greatest. Responses to 20 items with 4 and 6 categories generated under the IRTree Model fit with the IRTree Model resulted in the lowest mean IMSE compared to the same responses fit with the MPCM or the MNRM. For responses to 20 items 4 response options simulated under the IRTree Model, there was no difference in mean IMSE when fit with the MNRM and fit with the MPCM. A mean IMSE difference is evident, however, when responses to 20 items with 6 categories simulated under the IRTree Model were fit using the MNRM and the MPCM. Analysis performed using the MNRM resulted in a higher mean IMSE than analysis performed with the MPCM.

Figure 28. Mean IMSE for responses to 10 items with 4 and 6 response options generated under the IRTree Model, the MNRM, and the MPCM and analyzed by the IRTree Model, the MNRM, and the MPCM.

Figure 29. Mean IMSE for responses to 20 items with 4 and 6 response options generated under the IRTree Model, the MNRM, and the MPCM and analyzed by the IRTree Model, the MNRM, and the MPCM.