{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

chap5 - Chapter 5 Linear Stochastic Models 5.1 Least...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Chapter 5 Linear Stochastic Models 5.1 Least Squares Suppose that we observe some dependent variable (e.g. number of red blood cells) as a function of some independent variable (e.g. dosage of a drug), based on n experiments. The empirical data consists of ( x 1 ,y 1 ) ,... ( x n ,y n ), where x i is the value of the independent variable used for the i ’th experiment, and y i is the corresponding value of the dependent variable. We wish to find a simple mathematical model that summarizes the relationship of the dependent variable to the independent variable. The simplest such model is a linear model of the form y ( x ) = ax + b If n = 2 with x 1 negationslash = x 2 , we can find a and b by solving a linear system of two equations in two unknowns. If n > 2, the system is overdetermined, and we can apply the method of “least squares”. Specifically, let vector y = ( y 1 ,...,y n ) T , vectorx = ( x 1 ,...,x n ) T , vectore = (1 , 1 ,..., 1) T . A reasonable means of finding the “best” values of a and b is to select a and b as to minimize some measure of distance between vector y and avectorx + bvectore . One notion of distance that leads to a particularly nice system of determining equations for a and b is to measure the distance beween vector z 1 , vector z 2 IR n via bardbl vector z 1 vector z 2 bardbl , where bardbl vectorw bardbl 2 = vectorw T Λ vectorw. Here, Λ is a given n × n symmetric positive definite matrix. The minimizers a , b of the sum of squares min a,b bardbl vector y avectorx bvectore bardbl 2 satisfy the linear system parenleftbigg vectorx T Λ vectorx vectore T Λ vectorx vectorx T Λ vectore vectore T Λ vectore parenrightbiggparenleftbigg a b parenrightbigg = parenleftbigg vector y T Λ vectorx vector y T Λ vectore parenrightbigg . Our choice of a quadratic form as our (squared) notion of distance is what leads to a linear system for the minimizer. Other notions of distance would lead to more complex optimization problem. The case in which Λ = I is called “ordinary least squares”, whereas Λ negationslash = I is called“weighted least squares”. This approach to fitting a linear model to observed data leaves open several questions: 1. How should one choose the matrix Λ? 2. How can one assign “error bars” to our slope and intercept values a and b ? 77
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
3. Is there any way to objectively test the linear model against the even simpler model in which a = 0 (the “constant” model)? A statistical formulation of this linear modeling problem will permit us to address these issues. 5.2 Linear Regression Models with Gaussian Residuals We turn to the statistical view of how to build a linear model. We now view the dependent data values as random variables. In particular, we assume that Y i is a rv corresponding to the measured response of the dependent variable (e.g. blood pressure) as a function of the independent variable (e.g. drug dosage).
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern