Chapter 18 ANCOVA: Linear models with categorical and continuous explanatory variables
In the previous chapters we looked at linear models with a continuous response variable and a single explanatory variable: either categorical (one-way ANOVA) or continuous (linear regression). In this chapter we will look at linear models where the explanatory variables are both continuous and categorical. You can think of these as a cross between ANOVA and linear regression. These types of models are often called “ANCOVA” or Analysis of Covariance.
In a simple case, you might be interested in a model with a continuous response variable (e.g. height) and a continuous and a categorical explanatory variable (e.g. hand length and sex). The categorical variable may have any number of levels, but the simplest case is two (e.g. sex with male and female levels).
There are several possible outcomes, and some of them are illustrated in Figure 18.1. We might see that neither of the two explanatory variables has a significant effect. We might see that one does but not the other. We might see an interaction effect (where the effect of one variable, e.g. hand length, depends on the other, e.g. sex). We might also see an interaction effect but no main effect.
Figure 18.1: Some potential results of an ANCOVA. There may be a significant effect (or not) of the continuous explanatory variable (cont) and of the categorical explanatory variable (categ), and there may be a significant interaction between them (int) (or not). Y and N in each title indicate yes or no.
18.1 The height ~ hand length example.
In a previous class (linear regression) you explored the relationship between hand length and height. The aim there was (1) to determine if the relationship (i.e. the slope) was significantly different from 0, and (2) to estimate the equation of the relationship so you could make predictions of height from hand length.
Here we will extend that example by asking whether there are differences between males and females. We’ll use the same morphometry.csv dataset of body measurements from a sample of people.
Remember to load the dplyr, magrittr and
ggplot2 packages. The code below reads data from the
CourseData folder, so R’s working directory must be the
folder that contains CourseData. The easiest way is to open
the course’s RStudio Project, which does this for you. If you set the
working directory yourself instead (with setwd() or
Session > Set Working Directory), point it at that same
folder.
We’ll begin by plotting the data (Figure 18.2).
morph <- read.csv("CourseData/morphometry.csv") %>%
mutate(Height = Height / 10, HandLength = HandLength / 10) # mm -> cm
(A <- ggplot(morph, aes(
x = HandLength, y = Height,
colour = Sex
)) +
geom_point() +
geom_smooth(method = "lm", se = FALSE))
Figure 18.2: ANCOVA on hand length vs. height data in males and females
This shows the results of the ANCOVA model before we have even fit it! You can see that our two continuous variables, Height (the response variable) and HandLength (one of the explanatory variables) are associated: there is an overall positive relationship between HandLength and Height. You can also see that Sex (the categorical explanatory variable) is important: for any given hand length, the male line sits above the female one, so males tend to be taller. For example, at a hand length of 20cm a female is ~164cm tall while a male would be about ~171cm tall. The two slopes look fairly similar. These observations leave us with the following questions: (1) at a given hand length, do males and females differ in height (i.e. are the two lines at different heights)? (2) are the slopes for males and females significantly different (or would a model with a single common slope, but different intercepts, be better)?
Now we can fit our model using the lm function. The model formula is Height ~ HandLength + Sex + HandLength:Sex. The HandLength and Sex are the so called main effects while HandLength:Sex represents the interaction between them (i.e. it is used to address the question “does the effect of hand length differ between the sexes?”). R knows that it is fitting an ANCOVA type model rather than a two-way ANOVA because it knows the type of variables that it is dealing with. You can see this if you ask R to tell you what the class of the variables are:
## [1] "character"
## [1] "numeric"
The first step should, as before, be to check out the diagnostic plots:

These look good. No evidence of non-normality in the residuals, no heteroscedasticity and no weird outliers.
18.2 Summarising with anova
Now we can get the anova table of our ANCOVA model (yes, I know that sounds strange).
## Analysis of Variance Table
##
## Response: Height
## Df Sum Sq Mean Sq F value Pr(>F)
## HandLength 1 9535.3 9535.3 574.0622 < 2.2e-16 ***
## Sex 1 880.3 880.3 52.9950 1.711e-11 ***
## HandLength:Sex 1 2.9 2.9 0.1742 0.677
## Residuals 151 2508.1 16.6
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
This type of sequential sum of squares Analysis of Variance table should be getting fairly familiar to you now, but let’s unpack what this means. There are four rows in the summary table - one for each of the terms in the model (HandLength, Sex and HandLength:Sex), and one for the Residuals (the unexplained variation that remains after fitting the model). The table includes degrees of freedom (Df), sum of squares (Sum Sq), mean sum of squares (Mean Sq) and the associated F and p-values (F value and Pr(>F).
You can interpret the sum of squares column in terms of the amount of variation in the response variable (Height) that is explained by each term. (Each term here has 1 degree of freedom, so Sum Sq and Mean Sq are the same.) The table first tells us the amount of variation in Height that is captured by hand length (9535.3). Then it tells us that an additional bit of variation (880.3) is captured if we allow the intercepts to vary with sex. Then it tells us that a small additional amount of variation is explained by allowing the slope to vary between the sexes (the interaction, 2.9). Finally, there is a bit of unexplained variation left over (Residuals, 2508.1).
Be careful not to read this table as a ranking of which variable is “most important”. The table is sequential: each row is the variation explained after the rows above it have been accounted for. Hand length and sex are correlated (men tend to have longer hands), so they share some of the same information, and whichever term comes first in the formula gets the credit for that shared part. You can see this by swapping the order of the terms:
## Analysis of Variance Table
##
## Response: Height
## Df Sum Sq Mean Sq F value Pr(>F)
## Sex 1 8809.5 8809.5 530.3690 <2e-16 ***
## HandLength 1 1606.0 1606.0 96.6881 <2e-16 ***
## Sex:HandLength 1 2.9 2.9 0.1742 0.677
## Residuals 151 2508.1 16.6
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Now sex appears to explain far more than hand length. The F tests for the first two rows change, but the test for the last row (the interaction) is the same in both tables, because it is always tested after both main effects. So the test of the interaction is safe to use here, whichever order you choose.
You would report from this table something like this:
Hand length and sex both explain a significant amount of the variation in height (ANCOVA - HandLength: F = 574.1, d.f. = 1 and 151, p<0.001; Sex: F = 53, d.f. = 1 and 151, p<0.001). The interaction effect was not significant, which means that the slopes of the relationship between hand length and height are not significantly different between the sexes (ANCOVA - F = 0.2, d.f. = 1 and 151, p = 0.68).
It is of course useful to take the interpretation a bit further. You could do this with reference to the plot - e.g. Figure 18.2 shows the clear positive relationship between hand length and height, and shows that males tend to be taller than females at any given hand length.
The interaction is not significant, but that alone is not a reason to remove it: a non-significant result does not show that the two slopes are the same, only that this study cannot clearly tell them apart. So look at the estimate as well. In the summary of mod_A, the HandLength:SexMale coefficient is the difference in slope between males and females: 0.33 cm per cm (95% CI -1.25 to 1.92, from confint(mod_A)). That is small compared with the slope itself (about 3.7 cm per cm), although the interval shows that we cannot rule out a moderate difference. The interaction model also fits no better than the simpler model (Chapter 20 compares the two with anova() and AIC).
Our question is about the effects of hand length and sex, not about whether the slopes differ, so we simplify the model by dropping the interaction, leaving a model with a single common slope but different intercepts for the two sexes (a “parallel lines” model). If the interaction were part of your research question, you would instead keep it and report its estimate and confidence interval, whatever its p-value. Removing terms one at a time because their p-values are above 0.05 (“stepwise” selection) biases the estimates and inflates false positives, and is not recommended.19 We refit like this:
It is this simpler model that we now interpret.
18.3 The summary of coefficients (summary)
To put some quantitative numbers on this description of the pattern we need to get the summary of the simplified (parallel lines) model from R.
##
## Call:
## lm(formula = Height ~ HandLength + Sex, data = morph)
##
## Residuals:
## Min 1Q Median 3Q Max
## -10.7529 -2.6604 -0.3563 2.9424 9.7437
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 85.6144 7.5536 11.33 < 2e-16 ***
## HandLength 3.9115 0.3967 9.86 < 2e-16 ***
## SexMale 7.4154 1.0159 7.30 1.5e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.064 on 152 degrees of freedom
## Multiple R-squared: 0.8057, Adjusted R-squared: 0.8032
## F-statistic: 315.2 on 2 and 152 DF, p-value: < 2.2e-16
This summary table gives the coefficients of the statistical model, their standard errors, and the t-test results of whether the estimate is different from 0. This is the same as the summary tables given for ANOVA and linear regression.
In the ANOVA summary tables, the estimates were given in relation to the reference level – the (Intercept) – and these ANCOVA summary tables are no different. Interpreting is best done with reference to the graph of the data and fitted model outputs (the graph above).
The reference level (the (Intercept)) is the intercept for the line for the first level of the categorical variable (Females, in this case). Here the model estimates that the intercept for Females is at 85.61 (i.e. if you extended the line out to the left it would eventually cross the y-axis at this point). This is the predicted height for a hand length of 0 cm, far outside the data (hand lengths here range from 17.4 to 23.9 cm), so it has no biological meaning on its own: it is just the anchor point for the line. The next coefficient HandLength is the slope of the relationship (3.91) – because we dropped the interaction, this common slope applies to both sexes. Then we have SexMale: this coefficient (7.42) is the difference in intercept between the Female and Male lines – i.e. at any given hand length, males are estimated to be about 7.4cm taller than females.
We can therefore do some simple arithmetic to get the equations (i.e. slopes and intercepts) of the lines for both sexes. Both lines share the same slope (3.91). For females the intercept is 85.61 (the reference level, read directly from the table). For males the intercept is 85.61 + 7.42 = 93.03.
To show how precise these estimates are, we can get 95% confidence intervals for the coefficients with confint:
## 2.5 % 97.5 %
## (Intercept) 70.690784 100.538077
## HandLength 3.127697 4.695242
## SexMale 5.408339 9.422377
So, at the same hand length, males are estimated to be 7.4 cm taller than females (95% CI 5.4 to 9.4 cm), and height increases by 3.9 cm for every extra cm of hand length (95% CI 3.1 to 4.7).
We could add these estimates to our reporting of the results.
Figure 18.2 shows the clear positive relationship between hand length and height, and shows that males are taller than females at a given hand length. The two fitted lines are parallel: both have a slope of 3.91cm of height per cm of hand length, and the male line is offset upwards by 7.4cm (95% CI 5.4 to 9.4 cm).
You could check these by using geom_abline to add the two (parallel) lines to the plot (just as a “sanity check”).
A +
geom_abline(intercept = coef(mod_B)[1], slope = coef(mod_B)[2]) + # female
geom_abline(
intercept = coef(mod_B)[1] + coef(mod_B)[3],
slope = coef(mod_B)[2]
) # maleAt the bottom of the summary output we are given the \(R^2\) values. Because this model has more than one term (i.e. variable) in it we should use the adjusted \(R^2\) value. This has been corrected for the fact that the model has extra explanatory variables. So in this case, we could report that the model explains 80.3% of variation in Height (Adjusted \(R^2\) = 0.80) - not bad!
So, to describe this summary table more generally - the coefficients can be slopes, intercepts, differences between slopes, and differences between intercepts. They are slopes and intercepts for the first level of the categorical variable, and for the subsequent levels they are differences. Piecing these together can be hard to figure out without reference to the plot of the data and model fits - another good reason to plot your data!
Try it: Hidden Variable Lab (open the lab)
Explore what leaving out a variable does to a slope.
- Why does the hand-length slope drop from about 6.1 to 3.9 cm per cm when sex is added to the model?
- In the Simpson’s paradox tab, how far apart must the sites be before the overall slope changes sign?
- In the ANOVA/ANCOVA Interaction Lab (ANCOVA tab), set the group intercept difference to 0. What happens to the group term?
Whittingham, M. J., Stephens, P. A., Bradbury, R. B. & Freckleton, R. P. (2006). Why do we still use stepwise modelling in ecology and behaviour? Journal of Animal Ecology 75: 1182–1189. https://doi.org/10.1111/j.1365-2656.2006.01141.x↩︎