Chapter 9
Linear Regression and Correlation Prepared by: Fariba Mirzaei, Reza Ahmadi, Romana Dolati
Revision 9.1 LINEAR RELATIONSHIP One of the simplest relationship between two variables occurs when high values on one measure are associated with high values on another and low values on one are associated with low values on the other. Such a relationship can be drawn on a graph and it approximates a straight line. A linear relationship is an association between two variables that may be accurately represented on a graph by a straight line.
Revision Linear Function The formula y = α + βx expresses observations on y Linear Function of observations on x. The formula has a straight line graph with slop β and y-intercept α.
Revision •
•
•
If β is positive, then y increase as x increase- the straight line goes upward. So when a relationship between two variables follows a straight line with β>0, the relationship is said to be positive. If β negative, then y decrease as x increase. The straight line then goes downward, and the relationship is said to be negative. A model is simple approximation for the relationship between variables in the population. The linear function is the simplest mathematical function.
9.2 Least Squares Prediction Equation Using sample data, we can estimate the linear model. The process treats α and β in the equation y = α + βx as unknown parameters and estimates them. The estimated linear function then provides predicted y-values at fixed values for x.
Outline: Scatter plot
Prediction Equation
Prediction Errors (Residuals)
Least Squares Estimates
Ø
Scatter Plot
The first step of model fitting is to plot the data, to reveal whether a model with a straight line trend makes sense. The data values (x,y) for any one subject form a point relative to the x- and y-axes. A plot of the n observations as n points is called a scatter plot.
Ø
Prediction Equation
When the scatter plot suggests that the model y = α + βx is realistic, we use the data to estimate this line. The notation
ŷ= a + bx represents a sample equation that estimates the linear model. In the sample equation, the y-intercept (a) estimates the y-intercept α of the model and the slope (b) estimates the slope β. Substituting a particular x-value into a + bx provides a value, denoted by ŷ, that predicts y at that value of x. The sample equation ŷ= a + bx is called the prediction equation, because it provides a prediction ŷ for the response variable at any value of x. .
. The prediction equation is the best straight line, falling closet to the points in the scatter plot, in a sense discussed later in this section The formulates for a and b in the prediction equation
ŷ= a + bx are:
Example 1: The following table has shown the statistics marks and the average of students. (a) Find prediction equation. (b) Predict the average for a student who got 20 in statistics.
Statistics Mark
GPA
10
2
15
3
8
2
5
1.5
18
3.5
12
2.5
6
2
16
3
17
4
12
3
Answer (a) ŷ= a + bx ŷ= 0.77+ 0.158x (b) For x=20 ŷ= 0.77+ 0.158x=o.77+0.158×20 ŷ=3.93
Prediction Errors (Residuals)
Ø
For an observation, the differences between an observed value and the predicted value of the response variable, y-ŷ, is residual. A positive residual results when the observed value y is larger than the predicted value ŷ, so y-ŷ >0 . A negative residual results when the observed value is smaller than the predicted value . The smaller the absolute value of the residual, the better is prediction, since the predicted value is closer to the observed value .
Example 2: Find the residual and interpret for x=15 ,y=3 and x=6 ,y=2 in Example1.
Answer ŷ= a + bx ŷ= 0.77+ 0.158x For x=15 ŷ= 0.77+ 0.158x=o.77+0.158×15 ŷ=3.14 y=3 ,ŷ=3.14 → y-ŷ=3-3.14=-0.14<0
Answer(cont) ŷ= a + bx ŷ= 0.77+ 0.158x For x=6 ŷ= 0.77+ 0.158x=o.77+0.158×6 ŷ=1.718 y=2 ,ŷ=1.718 → y-ŷ=2-1.718=0.282>0
Least Squares Estimates
Ø
•
The least squares estimates a and b are the values that provide the prediction equation ŷ= a + bx for which the residual sum of squares
•
•
, is a minimum.
The symbol SSE is an observation for sum of squared errors. The prediction line ŷ=a + bx is called least squares line , because it is the one with the smallest sum of squarer residual.
Example 3: (a) Find SSE for the data in Example 1. (b) Show scatter plot and prediction line. Answer(a):
Scatter Plot
Prediction Line
4.00
GPA
3.50
Observed
4.00
Linear
3.50
3.00
A P G
3.00
2.50
2.50
2.00
2.00
1.50 1.50 5.00
7.50
10.00
Marks
12.50
15.00
17.50
5.00
7.50
10.00
12.50
Marks
15.00
17.50
References 1.
2.
Agresti, A. & Finlay, B. (2007).Statistical Method for The Social Science, Pearson Education, Inc. Freund, J. E & Perles, B. M. (2004) Statistics: A First Course: Fifth Edition. Pearson: Prentice Hall, New Jersey.
Thank You for Your Attention
