Multiple regression
- Read it, not yetReach the end of the page
- Practice it, not yetAnswer a few questions
- Master it, not yet80% on a drill of 8 or more
Several x's. Each coefficient is one x's effect holding the others fixed. The F-test checks the whole model, each t-test one x. Multicollinearity: two x's saying the same thing.
Why it matters
Multiple regression asks the fair question: what is x worth with everything else held fixed?
It separates effects that arrive tangled together, like experience and education in pay.
- PayThe pay gap left after accounting for role, level and tenure.
- Real estatePrice from size, age and location at once.
- ForecastingSales from price, season and promotions together.
Everyday example
Home prices on square feet and rooms: each predicts price alone, but together the rooms coefficient can turn negative and insignificant, because bigger homes have more rooms. A correlation of 0.95 means VIF ≈ 10.3: the model can't tell which is working.
Check yourself
Price = 42 + 0.118 SqFt + 6.5 Garage, price in $1,000s. Interpret 0.118.
Each extra square foot adds about $118, holding garage spaces fixedOnly inside the range of homes in the data.F = 18.2 vs F(0.05, 2, 12) = 3.89. Decision?
Reject H₀. At least one x has predictive value.SqFt t = 3.40, p = 0.005. Meaning?
Square footage adds predictive value to a model that already has garage spaces
The trap
The mistake people make most on this one.
Holding other variables fixed is about the math, not the world. If size and rooms move together in reality, you can't hold one fixed.
Formulas
6 formulas and a way to remember them.
- Model
- ŷ = b₀ + b₁x₁ + … + bₖxₖ
- F test
- F = MSR / MSE, df (k, n − k − 1)
- t test
- t = bⱼ / SE(bⱼ), df = n − k − 1
- Adjusted R²
- 1 − [SSE / (n − k − 1)] / [SST / (n − 1)]
- VIF
- VIFⱼ = 1 / (1 − Rⱼ²); trouble above 10
- Dummies
- k categories need k − 1 indicator columns
- Remember it
- A recipe with several ingredients. F asks if the recipe works; each t, if one ingredient earns its place. VIF flags duplicates.
Watch and go deeper
3 free videos and pages.
- StatQuest: Multiple Regression, Clearly ExplainedHow the ideas of simple regression carry over to several predictors, ending with an F test of whether the multiple regression beats the simple one. About 5 minutes.
- Penn State STAT 501, Lesson 8: Categorical PredictorsRead 8.2 to 8.5 on coding a category as 0/1 dummy variables and reading their coefficients. About 20 minutes.
- Penn State STAT 501, Lesson 12: MulticollinearityRead 12.1 to 12.4: what happens when predictors move together and how variance inflation factors flag it. About 20 minutes.