Fixed Effects & Random Effects

Whenever we do any linear regression or apply an ANOVA test we can have two different kinds of models – fixed effect models and random effect models. If our model consists of a mixture of both effects then such a model is called a mixed model.

Fixed Effect Model

As the name suggests, a fixed effect model is a model in which the number of treatments which can affect the dependent variable are fixed and known. Hence we can consider all of the treatments in our linear model. As an example, suppose we want to study the effect of four different diet regimens on the weight loss of individuals using the ANOVA procedure. Then we can construct our linear model and denote the effect due to the i^th treatment (diet regimen) as α_i. These α_i’s will be fixed constants in our model.

These kinds of models are used when the number of different treatments is less. If there are too many treatments that it is better to go for random effect models. So if in our above example, there were 100 kinds of different diet regimens we would construct a random effect model to test if there was any significant difference between them.

Random Effect Model

 If there are too many treatments that affect the dependent variable then it becomes tedious and impractical to consider all of them in our model. In that case we choose a suitably small random sample from our treatments. This small sample is used to check if there is any significant effect due to treatment. Here the effect due to the i^th treatment (denoted as α_i) will be treated as random variables and not fixed constants in our model.

As an example of a random effect model we may want to test the hypothesis that the performance of students in a standardised test is affected by the school which they attend. Since there are too many schools to include all of them in the model would be impractical. In this case we would use the random effect model by selecting a suitably small number of schools randomly.

In summary the main differences between fixed and random effect models are as follows:

1. In fixed effect models we have less number of treatments whereas in random effect models we have large number of treatments.
2. In fixed effect model the treatment effects (denoted as α_i) are treated as fixed constants in our model whereas in random effect model they are treated as random variables.
3. In a fixed effect models we can make conclusions only about the treatments chosen whereas in random effect models we can make conclusions about the entire population of treatments from which the sample has been chosen.

Mixed effect models:

As the name suggests, mixed effect models are a mixture of fixed and random effect models. These kinds of models are considered when there are many treatments but we are interested in a few of them in particular. Those treatments in whom we are interested are considered as fixed effects in our model and the remaining treatments are treated as random effects.