Validity

Paper I & III

A major advantage with the study design of Paper I and III is that exposure information was obtained before the outcome had occurred. In contrast to cross-sectional designed studies, causal inference is possible because one can be certain that the outcome occurred after the exposure in time. And unlike case-control designed studies, recall bias is not a problem.

A limitation in Paper I was the low level of statistical power. There were 50 cases of repeat attempts. Hence, the probability of making a type II error (failure to reject a false null hypothesis) is relatively high.

A problem in both Paper I and III, was that in order to be recorded, the repeated attempt had to lead to treatment in an emergency unit or hospital admittance in Asker & Bærum. However, in Paper III, we used a narrow time interval of six months in order to control for this deficiency. It is doubtful that a substantial proportion of individuals moved to another part of the country within six months after the index attempt. Further, the results were similar for a repeated attempt within both six and 12 months. Hence, it is unlikely that this limitation have biased these two studies.

In Paper III, selection bias is a problem. Both self-selection by patients and selection by health care personnel may have biased the study. Patients were free to choose whether or not to accept the assistance offered by the community suicide prevention team. Furthermore, health care personnel could choose not to offer team assistance when circumstances were considered inappropriate or of no particular concern. This kind of bias can only be avoided by

randomisation. Since patients were not randomised, we know a priori that firm conclusions cannot be drawn.

Paper II

In epidemiology, ecological-level data are often considered as less valid than individual-level data. According to Schwartz [115], ecological studies are conceived as “crude attempts to ascertain individual level correlations”, that is, as substitutes when individual level-data are unavailable. Further, the flaws in ecological models “limit their usefulness to hypothesis generations”. The flaws in ecological models are generally attributed to a problem known as the ‘ecological fallacy’. Ecological fallacy refers to the logical fallacy in cross-level inference, that is, to infer from the group level to the individual level, or from the individual level to the group level. It has been confirmed in several studies “that the correlation coefficient between

two individual-level variables is generally not the same as that between those same variables for aggregates into which the individuals are grouped” [115].

However, this is neither a statistical problem, nor a problem of measuring associations, but rather it is a problem of construct validity. That is, the assumption that because a group has a specified characteristic, the members of that group will have the same characteristic. An illuminating example is given in the paper by Schwartz [115]:

“A hung jury is a jury that is indecisive, it cannot decide whether the accused is guilty or innocent. However, to deduce that members of such a jury are indecisive would be absurd. Members of a hung jury are very decisive, so much that they cannot be persuaded to change their mind.”

Since ecological fallacy is a problem of cross-level inference, it is not a problem inherent in ecological studies. For instance, the problem of ecological fallacy commonly occurs in individual-level studies when proxy measures are used. For instance, assume that one wants to adjust for diet in a study of the association between a given exposure and disease. Data on dietary habits turns out to be unavailable; therefore data on education is used as a proxy measure of diet. The logic is that groups with different educational levels will exhibit different dietary habits. However, doing so will involve an ecological fallacy, because it is assumed that the individual exhibits similar dietary habits as the group the individual is a part of (this example is adapted from Schwarts, 1994) [115]. The result will most likely be measurement errors and a poor estimation of dietary habits.

In Paper II, ecological fallacy is not a problem. If our research aim had been to determine whether successful psychiatric treatment lowered the risk of dying by suicide, then we would have had a problem of ecological fallacy. However, the question of interest was if increased

resources in mental health services and a greater number of individuals being treated would have an annual dose-response impact on the national suicide rate. This question cannot be answered by individual-level data. Hence, our ecological model is not a substitute for a more appropriate individual-level model.

Complex phenomena are explained by mechanisms at a finer tuned level, e.g. depression can be explained by a decrease in serotoninergic neurotransmission. Hence, the reason for investigating a possible association between increased resources in mental health services and suicide mortality is the probability that successful treatment of individuals with psychiatric disorders will reduce the risk of dying by suicide. However, even if this mechanism is empirically true at the individual level, we cannot simply suppose the same relation at the societal level, as shown above.

A substantial advantage with an ecological study is that selection bias is not an issue. In addition, the changes in the exposure variables of interest are substantial over the period 1990-2006. This gives the design a natural experimental character, that is, competing explanatory variables can be assumed to be constant. Hence, the internal validity is high because of reduced problems with confounding.