This preview shows page 1. Sign up to view the full content.
Unformatted text preview: SE 10 Research Design
Lecture 2 Defining and Describing Data 1 Measurement Measurement is a way of representing concepts in a numerical form We cannot use concepts in social science research unless we can measure or quantify them in some way 2 Variable are used in different ways Can't take the average of variables like gender or favorite color Giving the most common test score might not be as useful as the average Different types of variables describe data in different ways and are used in different types of analyses based on how they are measured and recorded 3 Levels of measurement Nominal: categories, no logical numerical value, measures type or quality Ordinal: categories that can be logically ordered but are not inherently numerical Interval: numbers are meaningful, equal distance between each rank Ratio: similar to interval, but must have a meaningful "0" value
4 Dichotomous: only two categories Nominal level measures No numerical value associated with categories Categories not in a logical order Examples No halves or averages Can report frequency or percent Gender Race City of residence 5 Ordinal level measures No numbers Categories have logical order Examples Military rank Education Very dissatisfied, dissatisfied, neutral, satisfied, very satisfied No halves or averages No measurable distances between categories or not the same distance between categories
6 Interval level measures Numbers associated with categories Categories are logically ordered Measurable and equal distance between categories Examples Averages possible Temperature Age: 09, 1019, 2029, 3039, 4049, 5059, etc. 7 Ratio level measures Numerical values Values are in order The "0" values means the absence of the concept being studied Examples Scales or index variables usually not ratio level Averages and ratios possible
8 Prison sentence length Weight Level of measurement not inherent to the variable The researcher can select the level of measurement to be used in some cases Age Desired level of measurement depends on research question and statistical needs
9 Juvenile, Adult 02, 36, 712, 1317, 1825, 2635, etc. 09, 1019, 2029, 3039, 4049, 5059, ect. Number of days old Discrete & Continuous variables Discrete variables: a finite number of possible outcomes Nominal and Ordinal levels of measurement
Class rank, number of children Continuous variables: an infinite number of possible outcomes Interval and Ratio levels of measurement Miles commuted, percent body fat 10 How do we summarize the information we get from measuring Frequency histograms and graphs "Central tendency" "Dispersion" Mean Median Mode Skew Range Interquartile range Standard deviation 11 Frequency histogram
8 Favorite color 20 respondents 7 Yellow: 2 Red: 4 Green:4 Blue: 7 Purple: 3 6 5 4 3 2 1 0 12 Frequency distribution normal curve Used for ordered, numerical data interval or ratio
13 Central tendency Mean
Prison Sentence lengths 1 5 6 3 2 3 7 9 2 2 Mean the "average" Sum of the scores divided by number of scores 1+5+6+3+2+3+7+9+2+2= 40 N = 10 40/10 = 4
14 Central tendency Median
Sentence lengths 1 5 6 3 2 3 7 9 2 2 Median the middle value Order the values 1 2 2 2 3 3 5 6 7 9 The median is the number that leaves the same number of scores on either side For even numbers of values, take the mean of the two middle values
15 Central tendency Mode
Sentence lengths 1 5 6 3 2 3 7 9 2 2 The mode is the value that occurs most frequently 1 5 6 3 2 3 7 9 2 2 2 occurs most frequently If there is no most frequent value there is NO mode A distribution can have more than one mode
16 Different results for each measure of central tendency Mean: 4 Median: 3 Mode: 2 Which you pick depends on your purpose 17 How are measures of central tendency affected by outliers?
Outlier: a value that is very extreme compare to the rest of the values Sentence lengths 1 2 2 2 3 3 5 6 7 9 150 Mean: 190/11 = 17.27 Median: 3 Mode: 2 18 Dispersion Range Ages 18 18 22 23 27 29 30 30 30 32 35 39 The range is the numerical distance between the smallest and largest score Very sensitive to outliers The range is between 18 and 39 39 18 = 21 19 Dispersion: interquartile range 18 18 22 23 27 29 30 30 30 32 35
1st quartile 25th percentile Ages Median 50th percentile 3rd quartile 75th percentile The interquartile range is the range of the 1st and 3rd quartiles Quartiles can be found by adding 1 to the number of observations and dividing by 4 (11 + 1)/4 = 3 Count 3 values from lowest and highest 30 22 = 8 50% of the values are between the 1st and 3rd quartiles 20 Dispersion standard deviation
X 18 18 20 22 23 27 29 30 30 33 Sum Avg. 250 25 X Avg. 1825 1825 2025 2225 2325 2725 2925 3025 3025 3325 Deviation Dev. Squared 7 7 5 3 2 2 4 5 5 8 0 49 49 25 9 4 4 16 25 25 64 270 27 The standard deviation is the square root of the mean of the sum of the squared deviations from the mean Deviations always sum to 0 Mean of sum of squares here is 27 Square root of 27 is 5.2 Standard deviation = 5.2 SD describes the average variation about the mean 21 The normal curve and standard deviations 22 Skew of distributions
Positive Skew "tail" goes towards larger numbers, bulk of observations smaller Mode Negative Skew "tail" goes towards smaller numbers, bulk of observations larger Mode Media Mean n Mean Median
23 Transition from describing to critiquing We know how to measure and present data On Monday we talked about defining variables so we can measure them Now, how do we know if we've measured them well? 24 Measurement includes: True score: what the score would be without any error the real value Error: any form of error that means the true score is not recorded Observed Score: the score that is recorded Researcher error Equipment calibration Bad questions Subject error 25 Error All measurement has the potential for error Error can be random or nonrandom (bias) Statistics can easily handle random error We do not know the size of the error for each observation 26 Observed Scores
X = T + E X = Observed Score T = True Score E = Error 27 Observed Scores
Test Scores 100 Mean = 75 True Score Error 0 Observed Score Observed Score Error True Score 28 Two types of error Random error: Nonrandom error Cannot be predicted Mean error approximately 0 in large samples Not a problem in research Systematic can predict error for individual observations Mean error is not 0 Bad! Use good research methods to minimize
29 Measurement reliability and validity reduce error Reliability: consistency of the measurement Validity: accuracy of the measurement Would you get the same result if you measured again? Are you measuring what you say you're measuring? 30 Reliability Consistency doesn't have to be accurate, just replicable Physical measures: length, weight, etc. Subject measurement: test scores, survey answers Observation: ratings of behavior, subjective scoring 31 How to measure and improve reliability Testretest reliability Parallel tests Interrater reliability Standardize procedures Helpful to have clear definition of variables to be measured 32 TestRetest Reliability Give same test on two or more occasions What time should elapse between measures? Physical measures: can be done right away Measurement of people
Too short may remember answers Too long true score may have changed 33 Parallel Tests Reliability Give similar tests on two or more occasions "Multiple Measures" Remembered answers problem disappears tests can be closer together New problem: must be sure tests are measuring the same thing
34 Interrater Reliability Used for subjective observation type scores Two raters score independently and then compare Statistical tests for degree of agreement between raters Problems easily caught and dealt with during comparison
35 Standardizing Procedures Reduce subjectivity give detailed guidelines for scoring Reduce number of observers Uniform method for all participants Frequent calibration of equipment 36 What do reliability tests tell us about error? Not that much yet Cannot tell if and how true score and observed score differ Can determine, if you are accurately measuring the concept, how likely recorded score is to be accurate 37 Validity Accuracy is the concept being tested well represented by the measure being recorded? Multiple types of validity and several ways to assess the validity of a measurement To be valid, you need to have reliability 38 How to visualize validity
Our research question is about concepts Exogenous Construct Endogenous Construct We can only measure variables
Independent Variable Dependant Variable Are the variables measuring the construct accurately?
39 Types of validity Criterion validity how does the measure compare to other standard and accepted measures of the same thing? Face/Content validity does the measure make sense? Construct validity does the performance of the measure fit with existing theories?
40 Criterion validity Can formally test the relationship between a new measure and an old measure of the same concept Only works when there is a previously existing accepted measure Types: Predictive, concurrent, and postdictive validity
41 Face or Content validity Subjective assessment of how well the measure captures the concept Should always be considered, but especially when no criterion to use Panel of experts, logic, focus groups, previous research used to determine content
42 Construct validity Asks how well the measure reflects the concept by looking at relationship with other variables Is it positively associated with concepts as indicated by theory: convergent validity Is it not associated with concepts not indicated by theory: discriminant validity
43 Relationship of reliability and validity Reliability is a prerequisite for validity Measure can be reliable but not valid Measure CANNOT be valid without being reliable 44 Not Valid Reliable, not valid Reliable Valid Reliable and valid Ideal Not reliable or valid Not Reliable Not reliable, but "valid" Does this accurately represent the concept? Not really valid.
45 The full story about error
X = T + B + W + Er X = Observed Score T = True Score B = Bias W = Wrong Construct Er = Random Error Types of Error E = B + W + Er
46 Bias Variation from the true score in a consistent and predictable direction Affects accuracy of score, not necessarily consistency Problem with validity Low reliability with nonrandom error also creates bias
47 Low reliability that does NOT cause bias Low reliability that does cause bias Random error NonRandom error 48 Wrong construct error General term for measurement validity problem Variable is not measuring what you think it is Nonmeasured factors affect the score 49 Reducing error with MMMT Multiple Methods, Multiple Traits (MMMT) Methods: observation, survey, etc Traits: indicators of a construct Use two or more methods to measure a similar variable Measure two or more indicators of the same construct
50 ...
View
Full
Document
This note was uploaded on 09/14/2008 for the course SOCECOL 10 taught by Professor Pazzaniraitt during the Summer '08 term at UC Irvine.
 Summer '08
 PazzaniRaitt

Click to edit the document details