Identifying Outliers

We could guess at outliers by looking at a graph of the scatter plot and best fit-line. However, we would like some guideline as to how far away a point needs to be in order to be considered an outlier. As a rough rule of thumb, we can flag any point that is located further than two standard deviations above or below the best-fit line as an outlier. The standard deviation used is the standard deviation of the residuals or errors.

We can do this visually in the scatter plot by drawing an extra pair of lines that are two standard deviations above and below the best-fit line. Any data points that are outside this extra pair of lines are flagged as potential outliers. Or we can do this numerically by calculating each residual and comparing it to twice the standard deviation. The graphical procedure is shown first, followed by the numerical calculations. You would generally need to use only one of these methods.

Graphical Identification of Outliers

Let's see the same third exam versus final exam example.

Note: If there is an outlier, as an exercise, delete it and fit the remaining data to a new line. For this example, the new line ought to fit the remaining data better. This means the SSE should be smaller and the correlation coefficient ought to be closer to 1 or –1.

Numerical Identification of Outliers

In the third/final exam example in Section 10.2, the first two columns are the third-exam and final-exam data. The third column shows the predicted [latex]\hat{y}[/latex] values calculated from the line of best fit: [latex]\hat{y} = –173.5 + 4.83x[/latex]. The residuals, or errors, have been calculated in the fourth column of the table: [latex]\text{observed } y \text{ value} − \text{predicted } y \text{ value} = y − \hat{y}[/latex].

[latex]s[/latex] is the standard deviation of all the [latex]y − \hat{y} = \epsilon[/latex] values where [latex]n = \text{the total number of data points}[/latex]. If each residual is calculated and squared, and the results are added, we get the SSE. The standard deviation of the residuals is calculated from the SSE as [latex]s=\sqrt{\frac{\text{SSE}}{n-2}}[/latex]

Numerical Identification of Outliers: Calculating s and Finding Outliers Manually

You can calculate the outlier in the first example by doing the following.

First, square each [latex]| y – \hat{y} |[/latex].

The squares are [latex]35^2; 17^2; 16^2; 6^2; 19^2; 9^2; 3^2; 1^2; 10^2; 9^2; 1^2[/latex]

Then, add (sum) all the [latex]| y – \hat{y} | ^2[/latex] terms using the formula

[latex]\stackrel{11}{\underset{i = 1}{\Sigma }}{\left(|{y}_{i}-{\hat{y}}_{i}|\right)}^{2}=\stackrel{11}{\underset{i = 1}{\Sigma }}{\epsilon_i}^{2}[/latex] (Recall that [latex]y_i – \hat{y}_i = \epsilon_i[/latex].)

[latex]= 35^2 + 17^2 + 16^2 + 6^2 + 19^2 + 9^2 + 3^2 + 1^2 + 10^2 + 9^2 + 1^2 = 2440 = \text{SSE}[/latex]

The result, SSE, is the Sum of Squared Errors.

Next, calculate [latex]s[/latex], the standard deviation of all the [latex]y – \hat{y} = \epsilon[/latex] values where [latex]n = \text{the total number of data points}[/latex].

The calculation is [latex]s=\sqrt{\frac{\text{SSE}}{n–2}}[/latex].

For the third exam/final exam problem, [latex]s=\sqrt{\frac{2440}{11–2}}=16.47[/latex].

Next, multiply [latex]s[/latex] by 2:

[latex](2)(16.47) = 32.94[/latex]

32.94 is 2 standard deviations away from the mean of the [latex]y – \hat{y}[/latex] values.

If we were to measure the vertical distance from any data point to the corresponding point on the line of best fit and that distance is at least [latex]2s[/latex], then we would consider the data point to be "too far" from the line of best fit. We call that point a potential outlier.

For the example, if any of the [latex]|y - \hat{y}|[/latex] values are at least 32.94, the corresponding [latex](x, y)[/latex] data point is a potential outlier.

For the third exam/final exam problem, all the [latex]|y – \hat{y}|[/latex]'s are less than 31.29 except for the first one which is 35.

[latex]35 > 31.29[/latex] That is, [latex]|y – \hat{y}| \ge (2)(s)[/latex]

The point which corresponds to [latex]|y - \hat{y}| = 35[/latex] is [latex](65, 175)[/latex]. Therefore, the data point [latex](65, 175)[/latex] is a potential outlier. For this example, we will delete it. (Remember, we do not always delete an outlier.)

The next step is to compute a new best-fit line using the ten remaining points. The new line of best fit and the correlation coefficient are [latex]\hat{y} = –355.19 + 7.39x[/latex] and [latex]r = 0.9121[/latex].

How Does the Outlier Affect the Best Fit Line?

In the third exam/final exam example, numerically and graphically, we have identified the point (65, 175) as an outlier. We should re-examine the data for this point to see if there are any problems with the data. If there is an error, we should fix the error if possible, or delete the data. If the data is correct, we would leave it in the data set. For this problem, we will suppose that we examined the data and found that this outlier data was an error. Therefore, we will continue on and delete the outlier, so that we can explore how it affects the results, as a learning experience. If you compute a new best-fit line and correlation coefficient using the ten remaining points, you get the new line of best fit and the correlation coefficient are [latex]\hat{y} = –355.19 + 7.39x[/latex] and [latex]r = 0.9121[/latex].

The new line with [latex]r = 0.9121[/latex] is a stronger correlation than the original ([latex]r = 0.6631[/latex]) because [latex]r = 0.9121[/latex] is closer to one. This means that the new line is a better fit to the ten remaining data values. The line can better predict the final exam score given the third exam score.