Errors vs Residuals: What’s the Difference?

Question

Sooo...Are residuals the same as errors?

The terms seem to be used interchangeably in some questions/answers that I have come across and I think I am getting confused about one or two things on error terms, residuals, correlation etc, especially when it comes to autoregressive models.

So if anyone has a great explanation or couple of key points that candidates should remember for questions about 'errors in residuals' or 'correlation in residuals or error terms' to help make sure their knowledge has a good foundation on this topic, that would be much appreciated :)

Zee Tan · Accepted Answer

Residuals are very closely related to errors but they are not the same.

When e.g. calculating variance and standard deviation, residuals are the deviations each observed value from the observed sample mean.

Errors are when you're stating deviations of observed values from the true population mean.

Error vs residual example

Say you're rolling a fair six-sided die 5 times, with the results:

Roll 1: 3
 	Roll 2: 6
 	Roll 3: 2
 	Roll 4: 4
 	Roll 5: 1

Observed sample mean: ( 3+ 6+ 2+ 4+ 1 ) / 5 = 3.2

True mean: 3.5, which we know from a fair die is (1 + 2 + 3 + 4 + 5 + 6) / 6

Error vs residual example for roll 1:

Roll 1 residual = 3 - 3.2 = -0.2
 	Roll 1 error = 3 - 3.5 = -0.5

In this example the true mean is observable but in most cases it may not be (such as the mean height of the entire UK population).

cfyay · Answer

Excellent, thanks!

Zee Tan · Answer

Great - please accept my earlier answer by clicking on the green "✔︎ Accept" button so that the thread can be updated!