Module 4 Chapter 1: Review and Extension of Descriptive and Group Comparison Analyses

Dr. Audrey Begun

Module 4 Chapter 1: Review and Extension of Descriptive and Group Comparison Analyses

In our previous course you learned to distinguish between univariate and bivariate analyses, and to both conduct and interpret a variety of each type of analysis. Those activities were directed toward evidence for understanding diverse populations, social work problems, and social phenomena. Much of what was learned applies to our current course concerned with intervention and evaluation research for understanding social work interventions. In this chapter, these concepts will be reviewed in the context of intervention and evaluation research.

In this chapter you:

review principles of univariate analysis (mean, median, and standard deviation), this time as applied to data for understanding social work interventions;
review principles of bivariate analysis (single sample t-test, independent samples t-test, analysis of variance, chi-square, and non-parametric analytic approaches), this time as applied to data for understanding social work interventions;
learn basic principles about logistic regression which can answer questions about dichotomous intervention outcomes.

Review of Univariate Statistics Extended to Intervention Research

In our previous course (Module 4, Chapter 2), you learned about analyses that describe data one, single variable at a time—univariate analysis. These descriptive statistics and analyses continue to be relevant in research for understanding social work interventions:

Frequency & proportion data (categorical variables)
Central tendency analysis (numeric/continuous variables)
- mean
- median
- mode
Distribution analysis (numeric/continuous variables)
- range
- variance
- standard deviation
- normal curve
- skew & kurtosis

For example:

In evaluating an intervention to help families who are or at risk of homelessness, you might collect and analyze data concerning the proportion of families who were able to achieve stable housing status for a year or more (a dichotomous, categorical variable).
You might wish to compare the mean number of disciplinary actions that occur in middle school classrooms before and after teachers are trained to employ trauma-informed teaching and classroom management practices.
In evaluating the impact of an intervention on native-speakers of English with its impact on individuals for whom English is a second or third language, you might examine the variance in outcomes for these two groups—little variability in the second group might indicate that the intervention was not being adequately “received” or “processed” by these individuals.
You may need to assess the distribution of values on an outcome variable in planning your statistical analyses in an intervention study—whether or not the values are relatively normally distributed may determine which statistical tests are most appropriate.

Review of Bivariate Analysis Extended to Intervention Research

Recall that bivariate analysis is about assessing the nature and strength of relationships that exist between two variables. In our previous course you learned several statistical tests for answering different types of research questions using different types of variables. You learned about the:

single sample t-test
independent samples t-test
analysis of variance (Anova)
chi-square test
correlation test

In evaluating social work interventions, any of these tests might be applied, depending on the nature of the study questions, study design, and study variables involved. Here we will review all but the correlation test because that it seldom used as a means of evaluating outcomes in intervention research.

Single sample t-test.

In a very simple post-only study design an investigator might wish to compare a group’s mean outcome score to a pre-determined, established standard or norm value. This would be used, for example, to evaluate the impact of an intervention to improve the health of infants born to mothers who smoke cigarettes by reducing or eliminating smoking during pregnancy (and, ideally, after birth, as well). The outcome measure could be the newborn infants’ 5-minute Apgar scores, a measure routinely recorded at birth for all babies born in the United States (and many other countries). An Apgar score of 7, 8, or 9 means an infant is healthy; lower Apgar scores are associated with poorer health outcomes. Apgar scores can only be measured within minutes following birth, so a post-only design might make sense. The null hypothesis would be:

H₀: the post-intervention mean Apgar score (measured at 5-minutes post-birth) is 7 or greater for babies born to mothers who received the intervention.

You can refer to your prior course Excel workbook exercises as a reminder of how to implement the steps involved in a single sample t-test. The selection of 7 as the comparison mean value is a bit arbitrary. Ideally, the investigators have historical data from before the intervention was made available concerning the mean 5-minute Apgar score for mothers who smoke. For now, the investigation uses the “healthy baby” value as the comparison mean.

Sleeping infant

Independent samples t-test.

In Module 3 we considered study designs for answering research questions that involve comparing two intervention conditions. This might be a new intervention being compared to no intervention, a placebo, or a treatment-as-usual (TAU) “control” condition. Once investigators have collected outcome data for study participants who are exposed to these different intervention conditions, it is time to analyze those data to see if the hypothesized differences between the groups are actually observed. This is identical to the use of independent samples t-tests examined in our prior course where they were used to answer questions comparing groups to answer questions about diverse populations, social work problems, or social phenomena. Recall that certain assumptions need to be fulfilled in order to use this test—assumptions related to normal distribution and variance equality, for example. Otherwise, non-parametric analyses would be preferable. For the sake of clarity, let’s work with a specific example using independent samples t-tests to evaluate an intervention.

A Canadian team of investigators (Toneatto, Pillai, & Courtice, 2014) wanted to know if an intervention that enhanced Cognitive Behavioral Therapy (CBT) for problem gambling with Mindfulness Training was better than no intervention (the control group participants were on a wait-list for the program and did, eventually, receive the intervention). Both groups in this pilot study (N=18 participants total, 9 in each group) were measured before and after the intervention. The study design looked like this (note there was no random assignment to the intervention or wait-list control condition):

The Variables.

In this example, the observation (O₂) data reflect the outcome variable. In this case, the outcome variables were expected to be dependent on the intervention condition manipulated by the investigators: Mindfulness Enhanced Cognitive Behavioral Therapy (M-CBT) versus Wait List Control (no intervention). Thus, the outcome variables are the dependent variables in this example. The outcome variables were a set of measures of gambling urges and symptoms; we can demonstrate the important points with just two of these, gambling diagnosis symptoms and gambling urges. Both of these variables are numeric in nature, with the scale on each showing that a higher score means a more serious gambling problem.

The Null Hypothesis.

The research question being asked is if there existed a significant difference between the outcomes for the group receiving the experimental intervention (X₁) and the group on the waiting list (non-treatment control)—was the difference between these two groups meaningfully different from zero (no difference)? In statistics logic, the investigators were testing the null hypothesis that the difference between the two groups was zero.

H₀: there exists no statistically significant difference in symptoms or craving outcomes for the two treatment groups.

If the results of analysis led the investigators to reject the null hypothesis, it means they were reasonably confident that there existed a meaningful difference between the two groups (the difference was NOT zero). If the analysis led the investigators to fail to reject the null hypothesis, it means that no difference was detected (but they cannot conclude that no difference exists).

The Statistical Analysis Approach.

Since exactly two groups were compared, the independent groups t-test was appropriate. (Remember, in our prior course you learned that if three or more groups were being compared, an analysis of variance would be required; if only one group was involved, a single-sample t-test would be appropriate.) The underlying assumptions for this type of analysis are exactly the same as what you learned in in our prior course:

Type of (Outcome) Variable: The scale of measurement for the dependent variable is continuous (interval).
Normal Distribution: The dependent variable is normally distributed in the population, or a sufficiently large sample size was drawn to allow approximation of the normal distribution. Note: the “rule of thumb” is that neither group should be smaller than 6, and ideally has more.
Independent Observations: Individuals within each sample and between the two groups are independent of each other—random selection indicates that the chances of any one “unit” being sampled are independent of the chances for any other being sampled.
Homogeneity of Variance: Variance is the same for the two groups, as indicated by equal standard deviations in the two samples.

The independent groups t-test analysis (sometimes called a student’s t-test) involved dividing a difference score by a variance estimate. This is accomplished by the following steps:

calculating the mean score for each group (2 groups’ means are called M_group1and M_group2);
computing a “difference from the mean” score for each individual participant in each of the 2 groups—this is called the deviation score for each participant—then squaring that deviation score for each participant in each of the 2 groups;
computing the sum of the squared deviation scores for each group;
calculating the estimate for variance as the standard deviation squared for each group (S_group1² and S_group2²) by taking the sum of the squared deviation scores for each group calculated in step 3 and dividing by the number of cases in that group minus 1 (n_group1 – 1 and n_group2 – 1 );
compute the t-value as the difference between the two groups’ means (this is Mgroup1– Mgroup2) divided by the square root of a variance estimate computed as [(S_group1²divided by n_group1) + (S_group2²divided by n_group2);
compare the computed t-value with the criterion value identified using the degrees of freedom for the total N-2 (where N is the total number of participants in the two groups combined) and the α<.05 criterion.

Here is how it worked out in our example about the gambling intervention.

The Analysis Results and Interpretation.

The investigators observed a statistically significant difference in both outcome variables when they compared the enhanced intervention group to the non-treatment (wait listed) control group. Furthermore, the difference was in the expected direction: the intervention group’s mean scores were lower, indicating less severe gambling symptoms, than the control group’s mean scores.

Table 1. Outcome measures by group (adapted from Toneatto et al, 2014).

For both study outcome measures (symptoms and urges), the differences between the two treatment group means were statistically significant (the p<.05). Thus, the investigators rejected the null hypothesis of no difference (that the difference was zero) and concluded that mindfulness enhancement of cognitive behavioral therapy for problem gambling warrants further, more systematic research attention. One caution about over-interpreting these results is that intervention of any type is usually preferable to no treatment; future research should compare the mindfulness enhancement to the treatment-as-usual condition or the cognitive behavioral therapy alone.

Remember, also, that a strong intervention or evaluation research study also includes information concerning effect sizes—not only whether the differences were statistically significant but were they of meaningful magnitude, were the differences clinically significant?

Welcome to Las Vegas sign

Analysis of variance (Anova).

The independent samples t-test is a fine analysis when two groups are compared. But, as you learned in our prior course, analysis of variance (Anova) is preferred when more than two groups are being compared. Again, recall that certain assumptions need to be fulfilled in order to use this test—assumptions related to normal distribution and variance equality, for example. Otherwise, non-parametric analyses would be preferable. Consider the example presented by Walsh and Lord (2004) in their study of client satisfaction with social work services. In one analysis, the authors compared satisfaction scores for three groups of parents referred for social work services in a pediatric hospital setting. The three groups were:

parents referred for practical assistance(X₁)
parents referred for counselling (X₂)
parents referred for both practical assistance and counselling (X₃)

A diagram of this scenario might look like this:

ANOVA Diagram