In R, the model.matrix function is used to create the design matrix for regression. In particular, it is used to expand factor variables into dummy variables (also known as “one-hot encoding“).
Let’s see this in action on the iris dataset:
data(iris) str(iris) # 'data.frame': 150 obs. of 5 variables: # $ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ... # $ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ... # $ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ... # $ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ... # $ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ... x <- model.matrix(Sepal.Length ~ Species, iris) head(x) # (Intercept) Speciesversicolor Speciesvirginica # 1 1 0 0 # 2 1 0 0 # 3 1 0 0 # 4 1 0 0 # 5 1 0 0 # 6 1 0 0
model.matrix returns a column of ones labeled (Intercept) by default. Also note that while the Species factor has 3 levels (“setosa”, “versicolor” and “virginica”), the return value of model.matrix only has dummy variables for the latter two levels. For a factor variable, model.matrix treats the first level it encounters as the “baseline” level and will not produce a dummy variable for it. This is to avoid the problem of multi-collinearity.
However, there are situations where we might want dummy variables to be produced for all levels including the baseline level. (For example, when we do regularized regression, since multi-collinearity is no longer implies unidentifiability of the model.) We can induce this behavior by passing a specific value to the contrasts.arg argument:
x <- model.matrix( Sepal.Length ~ Species, data = iris, contrasts.arg = list(Species = contrasts(iris$Species, contrasts = FALSE))) head(x) # (Intercept) Speciessetosa Speciesversicolor Speciesvirginica # 1 1 1 0 0 # 2 1 1 0 0 # 3 1 1 0 0 # 4 1 1 0 0 # 5 1 1 0 0 # 6 1 1 0 0
Let’s have a closer look at what we passed as the value of Species in the list:
contrasts(iris$Species, contrasts = FALSE) # setosa versicolor virginica # setosa 1 0 0 # versicolor 0 1 0 # virginica 0 0 1
Notice that there are 3 columns, one for each level. If we didn’t pass this special value in, the default would have had just 2 columns, one for each of the levels we see in the output:
contrasts(iris$Species, contrasts = TRUE) # versicolor virginica # setosa 0 0 # versicolor 1 0 # virginica 0 1
It’s easy to modify the code above to include the baseline level for a different factor variable in another data frame. The code below is an example of how you can include the baseline level for all factor variables in the data frame.
df <- data.frame(x = factor(rep(c("a", "b", "c"), times = 3)),
y = factor(rep(c("d", "e", "f"), times = 3)),
z = 1:9)
str(df)
# 'data.frame': 9 obs. of 3 variables:
# $ x: Factor w/ 3 levels "a","b","c": 1 2 3 1 2 3 1 2 3
# $ y: Factor w/ 3 levels "d","e","f": 1 2 3 1 2 3 1 2 3
# $ z: int 1 2 3 4 5 6 7 8 9
# default: no dummy variable for baseline level
x <- model.matrix(~ ., data = df)
head(x)
# (Intercept) xb xc ye yf z
# 1 1 0 0 0 0 1
# 2 1 1 0 1 0 2
# 3 1 0 1 0 1 3
# 4 1 0 0 0 0 4
# 5 1 1 0 1 0 5
# 6 1 0 1 0 1 6
# dummy variables for baseline levels included
x <- model.matrix(
~ .,
data = df,
contrasts.arg = lapply(df[, sapply(df, is.factor), drop = FALSE],
contrasts, contrasts = FALSE))
head(x)
# (Intercept) xa xb xc yd ye yf z
# 1 1 1 0 0 1 0 0 1
# 2 1 0 1 0 0 1 0 2
# 3 1 0 0 1 0 0 1 3
# 4 1 1 0 0 1 0 0 4
# 5 1 0 1 0 0 1 0 5
# 6 1 0 0 1 0 0 1 6
References:
- StackOverflow. All Levels of a Factor in a Model Matrix in R.
Statistical Odds & Ends 
Pingback: Changing the column names for model.matrix output | Statistical Odds & Ends