While chin 2002 found that full professors and

while, Chin (2002) found that full professors and
associate professors each published 26 percent of
the articles between 1984 and 1999, very similar to
our findings. Baker (1985) did not report these data.
Between 2000 and 2009, 88 percent of articles
were published without the support of grants,
which is a decrease from Chin’s (2002) finding
that 95 percent of articles published from 1984 to
1999 were published without the support of grants
but similar to Baker’s (1985) finding (85 percent).
Across all time periods, we see most studies were
not supported by external grants.
What Gets Published in Teaching
Sociology
Similar to Baker (1985) and Chin (2002), we
found that case studies, or what we term here
evidence-based research studies, were the most
common types of publications. Here, we do not
distinguish type of publication (article or note) or
mode of innovation (e.g., classroom device, total
course design) as earlier studies have done because
these distinctions are not central to our analysis,
which is concerned with type of evaluation, regard-
less of classroom mode or publication type.
3
Over time, we expected to see articles exhibit-
ing more sophisticated evaluation measures,
reflecting greater awareness of and commitment to
SoTL. Table 4 shows that there is a general trend
toward more rigorous evaluation measures. The
category of articles containing no evaluation data
fell from 29 percent of all articles in Baker’s
(1985) sample (1973–1983) to 19 percent in Chin’s
(2002) sample (1984–1999) to 4 percent from
2000 to 2009, an overall decrease of 25 percent
from 1973 to 2009. The category of articles con-
taining casual evaluation (i.e., impressionistic
data, unsolicited comments from students but not
systematic comments such as student evaluations)
went from 48 percent of all articles from 1973 to
1983 to 51 percent from 1984 to 1999 to 10 per-
cent from 2000 to 2009, an overall decrease of
38 percent. The category of articles with a single
system of evaluation increased from 10 percent
from 1973 to 1983 to 18 percent from 1984 to
1999 to 65 percent from 2000 to 2009, an overall
increase of 55 percent. The category of articles
using a systematic comparison design remained
reasonably stable—from 13 percent in the first
period, 1973 to 1983, to 12 percent from 1984 to
1999—increasing to 20 percent from 2000 to
2009; this is an overall increase of 7 percent.
Recall Baker’s (1985) lament that single sys-
tems of evaluation often included a measure of
teacher satisfaction or an attitudinal measure (e.g.,
“I enjoyed this course”) rather than a highly valid
Table 4. Type of Evaluation Used for Evidence-based Studies, Articles, and Notes Published in Teaching
Sociology, 1973–1983, 1984–1999, 2000–2009
1973–1983 1984–1999 2000–2009
No evaluation 48 (29) 58 (19) 7 (4)
Casual data 80 (48) 80 (51) 21 (10)
Single system 16 (10) 54 (18) 133 (65)
Systematic comparison 21 (13) 35 (12) 41 (20)
Total 165 (100) 299 (100) 204 (100)
Source: Chin (2002:Table 1) and Baker (1985:Table 2).
Note: Values in parentheses are percentages.

Paino et al. 101
measure of learning. Hence, we delved more
deeply into the types of evaluation used. In par-
ticular, we were interested in whether researchers
relied on student reports of satisfaction, which are
generally considered poor indicators of learning