What in the World is ANOVA? Analysis of Variance
by Jared DiDomenico, CAPS Statistics Tutor
What on Earth is ANOVA?
One of the more advanced topics that undergraduate students may start to encounter is the analysis of variance (ANOVA) statistical procedure. Usually students are pretty apt at understanding the t-tests and their procedures, but ANOVA is a bit different. To put it simply, it’s when you want to compare the means of two samples and see if they differ. But what happens if you have more than two groups? What if we are curious about more than one type of factor (e.g. sex, education, etc.)? Then we use the ANOVA model.
Wikipedia defines ANOVA as, “… a collection of statistical models used to analyze the differences among group means and their associated procedures (such as "variation" among and between groups).” That’s quite a mouthful. Essentially, it means we can examine the means of various groups by comparing the variation that we could expect to find naturally between the groups and within the groups. Let’s think about that.
Source
Variation is something that is always going to be a part of data that consists of random variables. We are all familiar with the normal distribution. Height is an example of something that is normally distributed. This is saying that people in a population will have different heights at various frequencies (i.e. tall people occur less often than people of average height) that can be distributed like a bell curve. If we wanted to examine something like height from an ANOVA point of view it would kind of look like this:
EXAMPLE #1
Source
Are the heights between the various university sports teams different from one another? Let’s examine the heights of all the soccer, basketball, and football players at UNM (for the sake of this example, data will be made up). We found that the average heights of the soccer players were 70 inches, the football players were 72 inches, and the basketball players were 75 inches. Oh look, they aren’t equal therefore they are different! If anyone has had any statistics classes they know that this isn’t necessarily true because there is natural variation that may have just made our samples different. ANOVA is aimed at seeing if the means of heights in these groups are different and this can be understood if the variation that is present within the soccer players, within the football players, and within the basketball players is different than the variation between the all of the players.
Are the Means Different?
Like any statistical test, we have to have a null hypothesis and an alternative hypothesis. The null hypothesis the hypothesis we assume to be true. Null is usually synonymous with nothing, thus the null hypothesis is no effect. An alternative hypothesis is something that is different from null and cannot occur if the null is true. If the null is no effect, then something mutually exclusive is that there is an effect. Both can’t happen at the same time.
Now, specifically for ANOVA, our null hypothesis is all the group means are the same, or no difference.
The alternative is the opposite, or the compliment in probability, that at least one mean differs from the others. The easiest notation for this is:
To calculate whether or not there are any differences you need to know a few things…
Sums of Squares
Source
Sums of Squares are a measure of variability within our data. The total sums of squares is a measure of all the variability that is present within our model. (Note: I am going to start calling this ANOVA a model, because that’s what it is, its predicting a response knowing information from what group the response was in, plus a little error). This is a simple, yet tedious calculation, and is as follows:
SSTO (total sums of squares)
Essentially this is saying take the ith value from jth group, subtract the grand mean of the responses from it, square it, and then add it all up using every point in every group.
If we were do some mathematical proofs, we could figure out how to split up the total variation to show the variation between the groups and the variation within the groups. Essentially they are calculated as follows:
SSTR (treatment sums of squares,sometimes called between)
SSE (sums of squares of error, sometimes called within)
This shows us that, SSTO = SSTR +SSE. So it will be easier to solve two of them then use that information to solve the other. At the end of this I will include an example to illustrate the calculations.
The ANOVA Table
We can organize all the needed information by creating an ANOVA table. The general One-Way ANOVA table is below:
The table above shows how to use the sums of squares and information from the group sizes to be able to calculate an F- statistic called F*.
Once we know F* we can compare it to the F-distribution to find a p-value. First we need a level of confidence, the degrees of freedom for the treatment and the degrees of freedom for the error to find the critical F. Thus,
If F* > F(0.95, degrees of freedom for treatment, degrees of freedom for error), we will reject the null hypothesis and conclude the alternative.
If F* < F(0.95, degrees of freedom for treatment, degrees of freedom for error), we will fail to reject the null hypothesis.
From here you interpret you results. If you rejected the null then you know that there is at least one group or treatment mean that differs from the others. If you fail to reject the null hypothesis then all the group means are equal. Just be sure to tell what that means in context of the problems.
AND THAT’S IT FOR ANOVA.
Usually this test is followed by what are called multiple comparisons, if one rejects the null. If the null hypothesis was not rejected then you are finished. Multiple comparisons will be the subject of future blogs.
FULL EXAMPLE
The following are generated data following sports example mentioned previously:
1) State Your Hypotheses
The null hypothesis is that all the teams have the same mean height:
The alternative is that at least one sports team differs in mean height:
2) Calculation of Sums of Square
SSTO (total sums of squares)
SSTO =(69-72.33)^2+(72-72.33)^2+ ...+ (79-72.33)^2+ (76-72.33)^2=270.7
SSTR (treatment sums of squares, sometimes called between)
SSTR = 10*(70-72.33)^2+(10*(72-72.33))^2+10*(75-72.33)^2=126.7
SSE (sums of squares of error, sometimes called within)
SSE = (69-70)^2+⋯+(69-70)^2+(74-72)^2+⋯+(67-72)^2+ (76-75)^2+⋯+ (76-75)^2=144
Notice -> SSTO = SSTR + SSE = 126.7 + 144 = 270.7
3) Create the ANOVA Table
4) Decision
If F* > F(0.95, 2, 27), we will reject the null hypothesis and conclude the alternative.
If F* < F(0.95, 2, 27), we will fail to reject the null hypothesis.
We need to find F(0.95, 2, 27). Using a table, or program, we find that F(0.95, 2, 27) = 3.354.
Because F* = 11.88 > 3.354 = F(0.95, 2, 27), we reject the null hypothesis that the group means are equal and go with the alternative.
5) Conclusion
Because we rejected the null hypothesis, we conclude the alternative: At least one team has a mean height different than another.
To figure out which one, we must perform multiple comparisons, the subject of a different blog.
Source













