Computing power for an interaction in linear regression (created 2010-06-03).
This page is moving to a new website.
I was asked to help with a power calculation involving an interaction of two continuous variables in a linear regression model. I was terrified at the prospect, but I agreed to help. It turns out that the calculation is a lot easier than you might think.
First it helps to understand what an interaction between two continuous variables in a linear regression model really means. In this case, it was believed that one variable was an effect modifier. That means that the relationship between two primary variables changes depending on what the value of the effect modifier would be. Here's a simple example.
The upper left panel of this plot shows all of the data. There is a moderately strong positive association between the two variables. The upper right panel looks at a subset of the data where the "effect modifier" is low. The two lower panels look at a subset of the data where the "effect modifier is moderate or high. Although the data may shift around from side to side, notice that the same moderately strong positive association persists within each subgroup. This is an example where the "effect modifier" is not really an effect modifier (which is why I put that term in quotes). Contrast the above with a different example.
Here the nature of the relationship between X and Y changes, depending on the value of the effect modifier. There is a strong linear relationship between X and Y when the effect modifier is low and a weak linear relationship when the effect modifier is high. You can construct more extreme examples than this, such as there being a positive relationship between X and Y for one value of the effect modifier and a negative relationship between X and Y for a different value of the effect modifier.
I don't mean to imply that interactions and effect modifiers are interchangeable terms. An interaction actually can represent something much broader in scope than an effect modifier.
So how do you fit an interaction model in linear regression? The answer is multiplication. A two variable model without an interaction is defined as
where β0 represents the intercept, β1 represents the slope for the first independent variable, and β2 represents the slope for the second independent variable. A two variable model with an interaction is defined as
where β3 represents the interaction effect of the two independent variables. It is easier to understand the interaction term if you center the data (subtract away the means of the independent variables) before you multiply them together.
Focus your attention solely on this product term
This product will produce positive values if both terms are positive or both terms are negative. So the product term is positive if both independent variables are simultaneously above average or if both independent variables are simultaneously below average. Think of these as concordant pairs of independent variables. Likewise, the product term is negative if one independent variables is above average and the other one is below average. Think of these as discordant pairs of independent variables.
If the coefficient associated with the interaction term is positive (a positive interaction) that indicates an additional boost in the regression model for the concordant pairs of independent variables above and beyond the effect of each variable separately. This can sometimes be thought of as synergy or the whole being greater than the sum of the parts. This interpretation, however, may be overly simplistic in some contexts, so be cautious and think before you automatically claim that a postive interaction is evidence of synergy.
A negative interaction term (a negative interaction) indicates an additional boost in the regression model for the discordant pairs of independent variables above and beyond the effect of each variable separately. This can sometimes be thought of as antagonism. One variable works well (has a positive effect) in the absence of the other variable, and vice versa, but when both variables are large, they compete with one another and largely cancel each other out. Again, this is a possible interpretation, but it may not fit in all contexts.
Here is a geometrical perspective on interactions. First, examine the surface shown below, which corresponds to no interaction.
The equation for this surface is
The equation tells you to start at 40. Add one for each unit that X1 is above average or subtract one for each unit that X1 is below average. Do a comparable addition/subtraction for X2. The left corner of the surface, corresponding to X1 = 5 (5 units below average) and X2 = 3 (2 units below average) produces a value of 40 - 5 - 2 = 33. The corner of the surface all the way in the rear, corresponding to X1 = 5 and X2 = 7 produces a value of 40 - 5 + 2 = 37. The corner of the surface sticking out in the front, corresponding to X1 = 15 and X2 = 3 produces a value of 40 + 5 - 2 = 43. Finally, the right corner, corresponding to X1 = 15 and X2 = 7 produces a value of 40 + 5 + 2 = 47.
The surface shown above is called a plane. It can be thought of as a series of parallel lines in the X1 direction and also as a series of parallel lines in the X2 direction. For example, when you travel from the left corner to the front corner, you see an increase from 33 to 43 and when you travel from the back corner to the front corner you see a similar shift from 37 to 47.
The surface shown above is comparable to the first set of plots shown at the very beginning of this page. In that data set, there was the same moerately strong positive association for low, medium, and high values of the effect modifier.
Now let's examine a surface with a positive interaction.
The equation for this surface is
The product term tells you to calculate the extent to which X1 and X2 are above/below average, multiply those extents and add half of that amount to Y. So the left corner, corresponding to X1 = 5 (5 units below average) and X2 = 3 (2 units below average) produces a product of (-5)(-2) = 10. Add half of this (5) to Y in addition to the amount you would add/subtract based on the individual values of X1 and X2. Thus, the left corner is now 38 instead of 33. Similarly the right corner is now 52 instead of 47. The discordant corners: the front and back corners corresponding to X1 above average, X2 below average and vice versa, see a decline of 5. These values are now 38 instead of 43 and 32 instead of 37.
Imagine a flat sheet of paper hovering at an angle in the air. Now push the two concordant corners upward and flex the two discordant corners downward at the same time. It curves the surface into a shape somewhat evocative of a saddle. It may not be too apparent in this particular figure because of the orientation and the relatively small magnitude for the interaction term.
With this surface, you can no longer see a series of parallel lines in the X1 or X2 direction. The relationship between X1 and Y when X2 = 3, corresponding to the line connecting the left and front corners is a perfectly flat relationship since the values at those two corners are both 38. The relationship between X1 and Y when X2 = 7, corresponding to the line connecting the back and right corners, in contrast is very steep, the values increase from 32 at the back corner to 52 at the right corner.
Now take a look at a surface corresponding to a negative interaction.
The equation for this surface is
This formula produces the opposite effect. The concordant corners (the left and right corners) are pushed down by 5 units and the discordant corners are pushed up by 5 units. The values for the left, right, front, and back corners are 28, 42, 48, and 42, respectively.
Again with this surface, you can no longer see a series of parallel lines in the X1 or X2 direction, but the pattern is different. The relationship between X1 and Y when X2 = 3 is steep since the values change from 28 at the left corner to 48 at the front corner. The relationship between X1 and Y when X2 = 7, corresponding to the line connecting the back and right corners, in the one that is flat now, since the value is 42 at both the back and right corners.
Go back and look at the data for the second example. The relationship between X and Y was strongest when the effect modifier was low and weakest when the effect modifier was high. This suggests antagonism, or a negative interaction similar to the one shown just above.