test McNemars By employing essentially the same procedure as for comparing two conditional proportions (p1 and p2), but taking into account b b the dependence between p2+ and p+2, we can develop a chisquare test statistic:
(n12 n21)2 2. n12 + n21 This statistic is known as McNemars test. A version of McNemars test after correction of continuity is: (|n12 n21| 1)2 2. n12 + n21 Sign test. If the sample size is small, or more specically, if n12 + n21 is small, then the McNemar test discussed above is not appropriate. Since the two proportions agree under the null, a subject with discordant X and Y is equally likely to be in either (1,2) or (2,1) cell. So, conditional on such subjects, we are essentially facing a Bernoulli distribution and testing whether the parameter
p = .5. This is known as a sign test (a special case for testing proportions). In SAS, we can ask for McNemars test by including the "AGREE " option in TABLE statement. Exact test may be requested by including the "EXACT MCNEM" option.
4. Sets of 2 2 tables. Sometimes, we may have a set of similar contingency tables. For example, suppose that a new treatment is being tested against a control condition (e.g. placebo) at several hospitals across different cites in a multi-center randomized trial. We are interested in learning if the treatment has (superior) eects over the control condition. In other words, we want to test if the odds ratio is greater than 1 in favor of the new treatment. Since patients from dierent sites may be dierent in terms of their health conditions and varying levels of quality of healthcare services they receive from the dierent hospitals, the new treatment is likely to yield dierential treatment eects across the sites. So, if we pull all patients data into one contingency table regardless of the dierence among patients and hospitals, we may not have the opportunity to study the impact of dierences among patients and hospitals on treatment eect. In some cases, pulling data across tables may have far more serious ramications as the next example illustrates. Example. Suppose that there are two hospitals serving residents in a community. Hospital A (good) is staed with better surgeons than hospital B (bad) for some type of surgeries. The table below compares the success rates of surgery between the two hospitals over a period of time.
Outcome Hospital Success Fail Total A (Good) 50 50 100 B (Bad) 68 32 100 Total 118 82 100 The data seem to suggest that the bad hospital had a higher 68 50 success rate ( 100 vs. 100 ). The odds ratio (or relative risk) in comparing the success rate is less than 1 in favor of the bad hospital. So, we may conclude based on the evidence in the data that the bad hospital actually performed better! What is going on? Is the data lying here? Actually, there is nothing wrong with the data. The problem is that the data in the above table does not tell the whole story about surgeries performed between the two hospitals. If we stratify the data by disease severity before surgery, we obtain the following table: Outcome Severity Hospital Success Fail Total Less severe (1) A (Good) 18 2 20 B (Bad) 64 16 80 More severe (2) A (Good) 32 48 80 B (Bad) 4 16 20
If we compare success rates for each level of severity before surgery, we can see that Hospital A (good) always performed better than Hospital B (bad). However, in comparison to hospital B (bad), hospital A (good) received far more patients with a more severe disease status before surgery. This selection bias in receiving patients is what is causing Hospital A to have a lower overall success rate when disease severity is ignored. This phenomenon is called the Simpsons paradox. Selection bias is one of the most important issues in the eld of statistics and biostatistics. In fact, most cutting-edge topics in statistical research in recent years such as causal inference and longitudinal analysis all address this key issue. In this example, more severe patients selected (either self-select or through referrals) the good hospital and this disproportionality lowered the overall success rate for this hospital. Variables that cause selection bias are called confounding variables, confounders, covariates, etc. in the nomenclature of statistical applications. The above example shows that we cannot simply ignore confounding variables and collapse multiple tables into one. A correct approach is to account for dierences in the individual tables when making inference about the association between the
0
Observation period
T
HIV Infection
AIDS onset Observed
AIDS onset Unobserved
row and column variables. For such multiple, stratied 2 2 tables, we may use the Mantel-Haenszel test. Sometimes, selection bias is dicult to detect. For example, as we discussed in Chapter 1, disease surveillance systems may under report caseloads if the disease of interest has a long latency such as HIV/AIDS. Here, selection bias is the result of our limited observation time and is less obvious than other confounding factors such as disease severity and demographic dierences in treatment and cohort studies. Note: Simpsons paradox illustrates the eect of selection bias by a covariate for categorical outcomes. The same phenomenon occurs for continuous responses as well. The diagram illustrates the eect of selection bias within the same setting of the example, but with a continuous response. Suppose that the lower the response the better the outcome. If we ignore disease severity before treatment, we may again
conclude that hospital B (bad) performed better than hospital A (good). If we account for the confound eect of disease severity in the analysis, we will be able to tease out the eect of hospital from that of disease severity, leading to correct conclusions. Such a procedure is called "control for the eect of covariates" or "control for covariates" or "covariate analysis." When comparing the mean response of a continuous outcome among two or more groups, we often use the analysis of covariance (ANCOVA) model.
Mean response
.
Lower is better
severe
. . .
severe All patients
. .
Bad
All patients not severe
not severe
Good Hospital
Another related type of analysis that is becoming increasingly popular particularly in research in the behavioral and social sciences in recent years is moderation analysis. We will not discuss the concept of moderation in this chapter, but just want
to make you aware that moderation analysis is not the same thing as analysis of covariance (covariate analysis or analysis by controlling for covariates). Mantel-Haenszel test for no row and column association Suppose that there are q number of 2 2 tables, each has the same row and column variables. Again, we are testing if there is association between the row and column variables. Y X 1 1 n1 11 0 n1 21 1 Tot n+1 0 n1 12 n1 22 1 n+2 Tot X 1 q n1 n11 , , 1 1+ q 0 n21 n1 2+ q Tot n+1 n1 Y 0 q n12 q n22 q n+2 Tot q n1+ q n2+ nq
The null hypothesis is that there is no row by column association in any of the tables. Mantel-Haenszel suggested the following statistic:
nP o q h mh ) 2 11 h=1(n11 , Pq h h=1 v11
h h h h h = nhph ph and v h = n1+n+1n2+n+2 are the where m11 1+ +1 11 (nh)2(nh1)
expected value and variance of the count for the cell (1, 1) under the null hypothesis. is It approximately chi-square distributed if the sample size is large. Note: You can use the SAS PROC FREQ to perform the MantelHaenszel test. Output for the statistic and p-value are labeled under "Cochran-Mantel-Haenszel statistic". Testing for homogeneity of odds ratios.
Sometimes, we know a priori that the odds are dierent from 1, but may vary across dierent tables. In this case, we are interested in testing whether the odds ratios are the same across all the dierent tables, i.e., the homogeneity of odds ratios. So, the null hypothesis is dierent from the null in the above setting which posits that there is no row by column dierence or all odds ratios are equal to 1. We can use the Breslow-Day test for testing such a null. Breslow-Day test.
The Breslow-Day test is based on the Breslow-Day statistic which is computed as follows:
0 q X (nh mh )2 11 11 h0 v11 h=1
h0 where mh0 and v11 denote the expected value and variance, 11 respectively, under the null hypothesis of homogeneous odds h0 ratios. Note that mh0 and v11 are calculated based on the 11 null that all odds ratios are equal to 1 and thus in general,
h h h h h0 6= mh = nhph ph and v h0 6= v h = n1+n+1n2+n+2 . m11 11 1+ +1 11 11 (nh)2(nh1)
The Breslow-Day statistic has approximately a chi-square distribution of q 1 degrees of freedom if the sample is large. Note the the loss of 1 degree of freedom is due to the estimation of the common odds ratio across the tables. Unlike the MantelHaenszel statistic, the Breslow-Day test requires a relatively large sample size within each table. 5. General s r table Consider again two categorical outcomes X and Y . Now, suppose that X has s and Y has r levels. The outcomes of the pair (X, Y ) can again be displayed in a contingency table:
X 1 1 n11 2 n21 . . . . s ns1 Total n+1
2 n12 n22 . . ns2 n+2
Y ...
r n1r n2r . . nsr n+r
Total n1+ n2+ . . ns+ n
As in the 2 2 table case, we are again interested in association between X and Y and nature of such association when X and Y are associated. (a). Tests of association i). Pearsons tests for general association Pearsons chi-square statistic is dened for general s r tables and used to test general association between the row and column variables. In this test, both the row and column variables are treated as nominal variables, even if one or both may be ordinal. Pearsons chi-square statistic is dened as follows: QP =
s r X X (nij mij )2
i=1 j=1
mij
,
where mij = i+ +j is the expected value for count in cell (i, j). n The statistic QP has an asymptotic chi-square distribution with (r 1)(s 1) degrees of freedom under the null of no row by column association. The degree of freedom is based on the fact that with xed marginal counts, cell counts in any (r 1)(s1) submatrix will determine the whole table. In the 22 table case, one cell count in any of the four cells such as n11 determines the table. ii). Mean score test for ordinal column variable If the column variable Y is ordinal, and in addition, if its mean score is meaningful, then the mean score of Y for each row of X can be constructed and the association between X and Y can be stated in terms of such mean scores as follows: H0 : the mean scores of Y are the same across the rows of X. To compute the mean score, the levels used to indicate the ordinal column variable (i.e., 1, 2, . . . , r) are frequently used. Sometimes, however, to make the mean scores more meaningful or interpretable for a given application, other numerical scores
n
n
may be assigned to each level of the column variable. For example, if r = 5 and the ve levels of Y represent: strongly disagree, disagree, neutral, agree, strongly agree, we can use the column levels 1, 2, 3, 4, 5 to compute the mean score of Y . Alternatively, if we regard the change from "strongly disagree" ("strongly disagree") to disagree (agree) as representing a bigger jump from the change between disagree (agree) and neutral, we may want to change the equal spacing in 1, 2, 3, 4, 5 to something like: 1, 4, 5, 6, 9 to emphasize the dierential eect when moving across the response categories. Thus, in general, let a = (a1, a2, . . . , ar ) Rr be a column vector of dimension r with aj representing new the score for the j th level of the original response Y . Then, the mean score for the ith row is fi =
Pr j=1 aj nij
ni+
,
1 i s.
By using...