Study Limitations

Possible inconsistent coding

Identifying and coding supplements as botanicals and other NVNM in HCHS/SOL was a difficult process. Although guidance was sought from and given by the Office of Dietary Supplements, the author was unable to ensure that coding procedures were entirely consistent with other studies. The investigator chose the Langual-based process, because it was more specific than the NHANES-based supplement coding process. The latter was appropriate for and applied to the categorization of individual supplement ingredients in the dietary supplement data. Because much of the medication-

90

based data could not be coded at the ingredient level, ingredient-level coding was not appropriate for these data. The Langual-based coding scheme was easily adapted to product-level data and enabled the capture of many individual ingredients and the assignment of supplements to both broad and narrow categories.

Every effort was made to ensure consistent coding across the datasets and repeated data checks were performed. However, inconsistencies undoubtedly occurred. The medication-based and dietary-based data were very different in character. The medication-based data encoded much product information in a numeric variable saved as a string, but this information did not include specific ingredients. Where sufficient detail was available for the products, ingredients were obtained from online product labels and saved in an Excel spreadsheet, but only the most common ingredients were encoded, usually as classes of products such as protein, fiber or lipotropic agents. Particularly common individual ingredients were coded separately, e.g., omega-3 and omega-6, glucosamine with related chondroitin, MSM, and collagen, and lipotropic agents. Inconsistencies in coding could have influenced negatively the concordance statistics and resulted in biased estimates. An additional review of the coding of both the medication inventory and dietary supplement interview data would be desirable.

Possible additional measurement error

Lower overall estimates in the Bronx are of some concern. It is unclear whether botanical supplement use is just not as popular in the Bronx, or if there a systematic measurement error affecting those estimates. However, in previous studies, the prevalence of NVNM therapies in the northeast (18%) was lower than that seen in the west (24%) [122].

No standardization for botanical supplement assessments

A standard procedure for botanical supplement capture and assessing prevalence does not exist. Some methods restrict prevalence estimates to supplements that are consumed at least once per week and others count any use. Some studies ask about supplement use in the past week [73, 107] or two weeks [59, 171], others collect data on supplement use in the past year, e.g., [115, 119] or supplement use at any time in the participant’s life, e.g., [5, 183]. Some studies ask about use of

91

supplements for a particular indication. For example, studies request information about the use of supplements for diabetes [102] or arthritis [184]. Others ask about supplement use by individuals with certain conditions (e.g., menopause [37]) or individuals who are taking medications [16]. These differences in study design can make a large difference in prevalence estimates.

No studies have presented methods for correction for measurement error in dietary supplement studies. As noted above, all prior studies have compared duplicate instruments across supplements with kappa and intra-class correlation statistics. Calibration of botanical supplement use is hampered by the lack of a “gold standard”, an instrument with 100% sensitivity and specificity for defining the variable.

In the nutritional epidemiology literature, a literature with similar measurement error

challenges, several methods have been considered, all of which presume continuous exposure variables and utilize linear regression models. The goal of these models is to approximate the “true” value of a dietary value given available measures. They include: 1) validation with an instrument considered error-free (the “gold standard” or “criterion” measure) in a subsample with application of a correction to the entire sample [66, 165]; 2) repeated assessments in the same population with the same

instrument[185]; and/or 3) assessments with one or more additional, but error-prone instruments with utilization of statistical methods to come closer to the “true” measure [186]. The latter approach is hampered by non-identifiability when multiple variables are unknown. Non-identifiability can be solved by either making assumptions about the actual value of one of the variables and examining the sensitivity of the analysis to variations in the value [187] or by utilizing a Bayesian approach, making assumptions about the range and the shape of the distribution of unknown variables, based on prior literature and analyses [186, 188]. Other model-based procedures, e.g. regression calibration, correct measurement error in one variable by regressing it on an outcome, preferably one with little error. In regression calibration, analysts substitute the expected value of a variable, based on one or more values measured with error. The procedure requires setting one measure as the criterion, but uses all available information in setting up the equation.

In the current study, neither of the assessment instruments could be considered a gold standard. The medication inventory, if done in the home, could have been a criterion measure. But an

92

inventory completed at a study visit could miss supplements participants fail to bring to the visit [189]. Alternatively, the dietary supplement interview, following the 24-hour dietary recall as it does, may be a better criterion measure, making the assumption that the dietary supplement interview is more sensitive than the medication inventory for the outcome. In the current study, it may be possible to define a calibration coefficient for adjusting the probability of botanical supplement use as assessed by the medication inventory by regressing it on the probability of use as determined by the dietary

supplement interview including probability of being a botanical user given covariates as assessed in Aim 2. However, attempts to run this calculation resulted in improbably large correction estimates. Additional work in this area is needed.

It could be possible to estimate the “true botanical supplement prevalence” based on the imperfectly measured medication-based and dietary-based estimates using a structural equation modeling approach [190]. Structural equation modeling (SEM) is an attractive method for situations in which observed variables, measured with error, are expressions of a latent variable. In future, consultation with an expert in SEM may make the desired calculation possible.

Missing data

Very little data was missing for the dietary supplement interview or for the medication inventory. Moreover, even though more than 1600 products in the medication inventory could not be coded, because many individuals took multiple supplements, less than 4% of individuals were affected. Sensitivity analyses changed the interpretation of the comparisons between methods very little.

More data was missing for the analysis of characteristics of botanical supplement users. The analysis was restricted to individuals with complete data for all of the variables included in models. Data was missing for the medication inventory or dietary recall (355), education (386), physical activity (246), dietary quality (243), cigarette smoking (93), perceived health (238), birthplace (73), years of US residence (120), insurance status (323), and income (1488), resulting in a reduction in the analysis population from 16,415 to 13,735(16%). Individuals included in the analysis differed from those excluded by: 1) background group (fewer Cubans, more Mexicans); 2) gender (fewer females); 3) education (fewer less educated, more highly educated); 3) physical activity (more active); 4) smoking

93

(fewer smokers); 5) AHEI score (slightly higher in analysis dataset); and 6) acculturation (slightly higher SASH score). Hence the analysis dataset was enriched by individuals more likely to be supplement users. In addition, modeling missing variable status revealed several variables predicted missing value status with regard to percent of poverty. Attempts to correct for missing data with multiple

imputation programs were unsuccessful due to the need to include survey design characteristics in the model.

Lack of generalizability of study findings

Although HCHS/SOL recruits Hispanics/Latinos from across the United States and across background groups, by design, all of the target areas/field centers are in urban areas. Hence, the results of this study will not be generalizable to rural Hispanic/Latino populations, such as immigrants to small towns or migrant farmworkers.