Residuals
The residual eᵢ for the ith observation compares the observed value of y with the predicted value of y.
eᵢ = observed value of y – predicted value of y = yᵢ - ŷᵢ
The residual for any value of X reflects the error of the prediction.
A positive residual indicates that the observation falls above the regression line. A negative residual indicates that the observation falls below the regression line.
Extrapolation Mathematically, there is no problem with trying to predict a value of Y given a value of X that is outside of the range of the data. However, there is a statistical problem.
The process of predicting a value of Y for a value of X outside of the range of data is called extrapolation and should be avoided.
Outliers The regression line is strongly affected by outliers, especially outliers in the x-direction. Additionally, outliers also affect the coefficient of determination r².
With outliers included in the scatterplot, the regression line is less accurate in describing the relationship.
The following are three types of outliers:
The blue point is an outlier n the y-direction. It generally has little effect on the regression line.
The green point is a bivariate outlier that is neither in the x- or y-directions. It falls generally outside of the pattern of points and generally has little effect on the regression line.
The red point is an outlier in the x-direction. It has a strong effect on the regression line.
Influential Observations An observation is influential if removing it from the data would dramatically change the position of the regression line and the value of r².
Generally, outliers in the x-direction are influential observations.