{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# chap5 - Chapter 5 Linear Stochastic Models 5.1 Least...

This preview shows pages 1–3. Sign up to view the full content.

Chapter 5 Linear Stochastic Models 5.1 Least Squares Suppose that we observe some dependent variable (e.g. number of red blood cells) as a function of some independent variable (e.g. dosage of a drug), based on n experiments. The empirical data consists of ( x 1 ,y 1 ) ,... ( x n ,y n ), where x i is the value of the independent variable used for the i ’th experiment, and y i is the corresponding value of the dependent variable. We wish to find a simple mathematical model that summarizes the relationship of the dependent variable to the independent variable. The simplest such model is a linear model of the form y ( x ) = ax + b If n = 2 with x 1 negationslash = x 2 , we can find a and b by solving a linear system of two equations in two unknowns. If n > 2, the system is overdetermined, and we can apply the method of “least squares”. Specifically, let vector y = ( y 1 ,...,y n ) T , vectorx = ( x 1 ,...,x n ) T , vectore = (1 , 1 ,..., 1) T . A reasonable means of finding the “best” values of a and b is to select a and b as to minimize some measure of distance between vector y and avectorx + bvectore . One notion of distance that leads to a particularly nice system of determining equations for a and b is to measure the distance beween vector z 1 , vector z 2 IR n via bardbl vector z 1 vector z 2 bardbl , where bardbl vectorw bardbl 2 = vectorw T Λ vectorw. Here, Λ is a given n × n symmetric positive definite matrix. The minimizers a , b of the sum of squares min a,b bardbl vector y avectorx bvectore bardbl 2 satisfy the linear system parenleftbigg vectorx T Λ vectorx vectore T Λ vectorx vectorx T Λ vectore vectore T Λ vectore parenrightbiggparenleftbigg a b parenrightbigg = parenleftbigg vector y T Λ vectorx vector y T Λ vectore parenrightbigg . Our choice of a quadratic form as our (squared) notion of distance is what leads to a linear system for the minimizer. Other notions of distance would lead to more complex optimization problem. The case in which Λ = I is called “ordinary least squares”, whereas Λ negationslash = I is called“weighted least squares”. This approach to fitting a linear model to observed data leaves open several questions: 1. How should one choose the matrix Λ? 2. How can one assign “error bars” to our slope and intercept values a and b ? 77

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3. Is there any way to objectively test the linear model against the even simpler model in which a = 0 (the “constant” model)? A statistical formulation of this linear modeling problem will permit us to address these issues. 5.2 Linear Regression Models with Gaussian Residuals We turn to the statistical view of how to build a linear model. We now view the dependent data values as random variables. In particular, we assume that Y i is a rv corresponding to the measured response of the dependent variable (e.g. blood pressure) as a function of the independent variable (e.g. drug dosage).
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern