Fixed effects vs Random Effects (with Examples)

Whenever we do any linear regression or apply an ANOVA test we can have two different kinds of models – fixed effect models and random effect models. If our model consists of a mixture of both effects then such a model is called a mixed model.

Fixed Effect Model

As the name suggests, a fixed effect model is a model in which the number of treatments that can affect the dependent variable is fixed and known. Hence we can consider all of the treatments in our linear model.

As an example, suppose we want to study the effect of four different diet regimens on the weight loss of individuals using the ANOVA procedure.

Then we can construct our linear model and denote the effect due to the i^th treatment (diet regimen) as α_i. These α_i’s will be fixed constants in our model.

These kinds of models are used when the number of different treatments is less. If there are too many treatments that it is better to go for random effect models.

So if in our above example, there were 100 kinds of different diet regimens we would construct a random effect model to test if there was any significant difference between them.

Random Effect Model

If there are too many treatments that affect the dependent variable then it becomes tedious and impractical to consider all of them in our model. In that case, we choose a suitably small random sample from our treatments.

This small sample is used to check if there is any significant effect due to treatment. Here the effect due to the i^th treatment (denoted as α_i) will be treated as random variables and not fixed constants in our model.

As an example of a random effect model, we may want to test the hypothesis that the performance of students in a standardized test is affected by the school in which they attend.

Since there are too many schools including all of them in the model would be impractical. In this case, we would use the random effect model by selecting a suitably small number of schools randomly.

In summary, the main differences between fixed and random effect models are as follows:

1. In fixed-effect models, we have less number of treatments whereas in random-effect models we have a large number of treatments.
2. In the fixed effect model, the treatment effects (denoted as αi) are treated as fixed constants in our model whereas in the random effect model, they are treated as random variables.
3. In fixed-effect models, we can make conclusions only about the treatments chosen whereas in random-effect models we can make conclusions about the entire population of treatments from which the sample has been chosen.

Mixed effect models:

As the name suggests, mixed effect models are a mixture of fixed and random effect models. These kinds of models are considered when there are many treatments but we are interested in a few of them in particular.

Those treatments in which we are interested are considered fixed effects in our model and the remaining treatments are treated as random effects.