Maltreatment classifications

Beginning in 2008 MIMR-CDR adopted a sensitive tiered maltreatment definition to expand classification and identification of deaths that were maltreatment-related due to any physical abuse, neglect, and/or gross-negligence. Each death is evaluated for maltreatment, and through consensus the MIMR-CDR panel labels each death as either “Yes”, “Yes probably”, “No”, “Unknown committee

suspicious”, or “Unknown but unlikely”. This definition is not used for official designation or instigation legal investigation, but rather to lead public health prevention efforts.

The Alaska maltreatment classification questions used during this study period are slightly different from the national classification questions. Although different, both allow for a degree of ambiguity by allowing for “possible” maltreatment. Like the National model, Alaska includes a category for

Negligence. (see APPENDIX 6 for the Alaska and National questions). This category was included in Alaska to differentiate between Neglectful acts and seemingly lesser acts or behaviors such as inappropriate supervision that resulted in an adverse outcome. While the definition of omission utilized specifies without regard to intent the Alaska committee concluded that the preventative approaches and interventions are likely somewhat differential and should be differentiated between Neglect and

Negligence.

All forms of maltreatment (Abuse, Neglect, and Negligence) are assessed using the omission and commission definitions and are outlined as:

Abuse: Words or overt actions that cause harm, potential harm, or threat of harm to a child. Acts of Abuse are deliberate and intentional: however, harm to a child may or may not be the intended

consequence. Intentionality only applies to the caregivers’ acts – not the consequences of those acts. For example, a caregiver may intend to hit a child as punishment (i.e., hitting the child is not accidental or unintentional) but not intend to cause the child to have a concussion.

Neglect: The failure to provide for a child’s basic physical, emotional, or educational needs or to

protect a child from harm or potential harm. Like acts of abuse, harm to a child may or may not be the intended consequence.

Gross negligence: Similar to Intentional Neglect which addresses the failure to provide or protect a child from harm or potential harm, but also includes the failure to exercise reasonable care that would be expected of any other person in a similar situation because of the perceived risk of a negative outcome.

4.4.5 Analysis

We assessed the inter-rater reliability between two review panels for the same set of cases using weighted Kappa statistics and percent agreement. We calculated the weighted Kappa as proposed by “Fless-Cohen” that uses a quadratic weighting of the discordant pairs assuming ordinality to emphasize close matches to the diagonal more heavily than distal matches.(205) We constructed 6x6 cross- classification (i.e. confusion matrix) tables for abuse, neglect, and negligence. Overall agreement was derived by combining the three cross-classification tables of abuse, neglect, and negligence. Cell totals were created by averaging the cross-classification summary table cells to preserve the combined marginal distributions and prevent variance inflation. From this combined table we calculated an overall maltreatment percent agreement and weighted Kappa statistic. Table 4.5 presents the combined cross- classification table between panels and documentation describing the calculations to derive the weighted Kappa (each cross-classification table for abuse, neglect, and negligence can be viewed in APPENDIX 7)

We also assessed each classification matrix (Abuse, Neglect, Negligence) for rater bias using a rater bias coefficient and tested for proportional differences using Chi Square test.(206) We calculated the incidence estimates and 95% Confidence Intervals (95% CI) for both panels, intersect between panels, and union of the two panels.

We examined the effect of the observed reliability on cumulative incidence estimates per 1,000 live births. As a working assumption, the “Yes” and Yes probably” categories were combined and

assumed to represent a maltreatment positive surrogate. A lower bound on incidence was assumed to be the deaths classified as “Yes” or Yes probably” by both panels (“panel-intersection”), and a upper bound

on incidence was assumed to be the deaths classified as “Yes” or Yes probably” by either panel (“panel- union”).

Finally, we measured the influence of demographics, missing information, and presence/absence of select factors known to be related with maltreatment to determine if classification consistency or discordance is influenced by the factors.

It should be noted that the assumption of sample independence is violated in the proposed study as the individual reviewers may be the same in the second review as in the initial review. The detection probability of maltreatment for any given case is assumed to be independent of prior classification and only varies by committee membership and heterogeneity of the questions. Under these assumptions the probability of maltreatment classification (the unit of measure) is conditionally dependent on the sampling unit (the child) but independent of the review committee. The null hypothesis assumes that the conditional probability of classification for individual (p) in the initial review will equal the conditional probability of individual (p) in the re-review.

Table 4.5 Overall derived cross-classification of maltreatment between panels one and two with analysis methods described, (n=101) Re-review No Unknown but unlikely Unknown committee suspicious Yes probably Yes Total Init ial Re vie w No 47 4 4 2 1 58

Unknown but unlikely 6 3 2 1 1 13

Unknown committee suspicious 3 1 4 0 2 10 Yes probably 2 0 0 3 1 6 Yes 3 1 2 1 7 14 Total 61 9 12 7 12 101 Fleiss-Cohen weights = 1 − [(𝑟𝑖− 𝑑𝑗) 2 /(𝑀 − 1)2_{], where 𝑟}

𝑖= distance of ith cell from the jth row/column diagonal, setting d = 0, and M = max row/column count.

The weighted Kappa is calculated as (∑ 𝑃𝑟 (𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑)𝑖𝑗

25

1 )−(∑251 𝑃𝑟 (𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑)𝑖𝑗)

(1−∑25₁ 𝑃𝑟 (𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑)_𝑖𝑗) , where 𝑖𝑗= the row/column cell.

For example, in row (i) “Yes probably” and column (j) “No” cell intersect the weight for this cell is calculated as 1-[(3-0)2 / (5-1)2] = 0.4375. The probability overserved [Pr(observed)] = (2/ 101)*0.4375 = 0.01, and the probability expected [Pr(expected)] = [((61*6)/101) / 101]*0.4375 = 0.02.

The weighted Kappa is the sum of the 25 observed probabilities of all cells minus the sum of the expected of all cell probabilities, divided by 1 minus the sum of the probabilities of all 25 expected cells which equals [(0.8962 – 0.7374) / (1 – 0.7374)] = 0.605.

Percent agreement for the full confusion matrix is the sum of the cross-diagonal cells divided by the table total = [(47+3+4+3+7) / 101] = 63.4%. The row/cell percent agreement is calculated by the row/cell diagonal intersect divided by the row/cell total, for example for the No/No intersect the percent agreement = [47/ (47+6+3+2+3+4+4+2+1)] = 65.3%

CHAPTER 5: BIAS IN LONGITUDINAL DATA LINKAGE STUDIES OF CHILD MALTREATMENT