while, Chin (2002) found that full professors and associate professors each published 26 percent of the articles between 1984 and 1999, very similar to our findings. Baker (1985) did not report these data. Between 2000 and 2009, 88 percent of articles were published without the support of grants, which is a decrease from Chin’s (2002) finding that 95 percent of articles published from 1984 to 1999 were published without the support of grants but similar to Baker’s (1985) finding (85 percent). Across all time periods, we see most studies were not supported by external grants. What Gets Published in Teaching Sociology Similar to Baker (1985) and Chin (2002), we found that case studies, or what we term here evidence-based research studies, were the most common types of publications. Here, we do not distinguish type of publication (article or note) or mode of innovation (e.g., classroom device, total course design) as earlier studies have done because these distinctions are not central to our analysis, which is concerned with type of evaluation, regard- less of classroom mode or publication type. 3 Over time, we expected to see articles exhibit- ing more sophisticated evaluation measures, reflecting greater awareness of and commitment to SoTL. Table 4 shows that there is a general trend toward more rigorous evaluation measures. The category of articles containing no evaluation data fell from 29 percent of all articles in Baker’s (1985) sample (1973–1983) to 19 percent in Chin’s (2002) sample (1984–1999) to 4 percent from 2000 to 2009, an overall decrease of 25 percent from 1973 to 2009. The category of articles con- taining casual evaluation (i.e., impressionistic data, unsolicited comments from students but not systematic comments such as student evaluations) went from 48 percent of all articles from 1973 to 1983 to 51 percent from 1984 to 1999 to 10 per- cent from 2000 to 2009, an overall decrease of 38 percent. The category of articles with a single system of evaluation increased from 10 percent from 1973 to 1983 to 18 percent from 1984 to 1999 to 65 percent from 2000 to 2009, an overall increase of 55 percent. The category of articles using a systematic comparison design remained reasonably stable—from 13 percent in the first period, 1973 to 1983, to 12 percent from 1984 to 1999—increasing to 20 percent from 2000 to 2009; this is an overall increase of 7 percent. Recall Baker’s (1985) lament that single sys- tems of evaluation often included a measure of teacher satisfaction or an attitudinal measure (e.g., “I enjoyed this course”) rather than a highly valid Table 4. Type of Evaluation Used for Evidence-based Studies, Articles, and Notes Published in Teaching Sociology, 1973–1983, 1984–1999, 2000–2009 1973–1983 1984–1999 2000–2009 No evaluation 48 (29) 58 (19) 7 (4) Casual data 80 (48) 80 (51) 21 (10) Single system 16 (10) 54 (18) 133 (65) Systematic comparison 21 (13) 35 (12) 41 (20) Total 165 (100) 299 (100) 204 (100) Source: Chin (2002:Table 1) and Baker (1985:Table 2). Note: Values in parentheses are percentages.
Paino et al. 101 measure of learning. Hence, we delved more deeply into the types of evaluation used. In par- ticular, we were interested in whether researchers relied on student reports of satisfaction, which are generally considered poor indicators of learning
You've reached the end of your free preview.
Want to read all 14 pages?