2009, MEMORY, 17 (4), 471479
Metacognitive strategies in student learning: Do students practise retrieval when they study on their own?
Jeffrey D. Karpicke Purdue University, West Lafayette, IN, USA Andrew C. Butler and Henry L. Roediger III Washington University in St. Louis, MO, USA
Basic research on human learning and memory has shown that practising retrieval of information (by testing the information) has powerful effects on learning and long-term retention. Repeated testing enhances learning more than repeated reading, which often confers limited benefit beyond that gained from the initial reading of the material. Laboratory research also suggests that students lack metacognitive awareness of the mnemonic benefits of testing. The implication is that in real-world educational settings students may not engage in retrieval practise to enhance learning. To investigate students real-world study behaviours, we surveyed 177 college students and asked them (1) to list strategies they used when studying (an open-ended, free report question) and (2) to choose whether they would reread or practise recall after studying a textbook chapter (a forced report question). The results of both questions point to the same conclusion: A majority of students repeatedly read their notes or textbook (despite the limited benefits of this strategy), but relatively few engage in self-testing or retrieval practise while studying. We propose that many students experience illusions of competence while studying and that these illusions have significant consequences for the strategies students select when they monitor and regulate their own learning.
Keywords: Testing effect; Retrieval; Metacognition; Strategies.
A powerful way to enhance student learning is by testing information. When students have been tested on material they remember more in the long term than if they had repeatedly studied it. This phenomenon is known as the testing effect and shows that the act of retrieving information from memory has a potent effect on learning, enhancing long-term retention of the tested information (for review, see Roediger & Karpicke, 2006a). The testing effect is especially striking in light of current findings showing
limited benefits of repeated reading for student learning (see Callender & McDaniel, 2009; McDaniel & Callender, 2008). Our recent research has generalised the testing effect to educational materials (Butler & Roediger, 2007; Karpicke & Roediger, 2007, 2008; Roediger & Karpicke, 2006b) and real-world classroom environments (see McDaniel, Roediger, & McDermott, 2007). Testing enhances learning not only if instructors give tests and quizzes in the classroom but also if students practise recall while they study
Address correspondence to: Jeffrey D. Karpicke, Department of Psychological Sciences, Purdue University, 703 Third Street, West Lafayette, IN 47907-2081, USA. E-mail: [e-mail] We thank Stephanie Karpicke and Julie Evans for collecting and scoring the questionnaire data. This research was supported by a Collaborative Activity Grant from the James S. McDonnell Foundation and a grant from the Institute of Education Science.
# 2009 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business http://www.psypress.com/memory DOI:10.1080/09658210802647009
472
KARPICKE, BUTLER, ROEDIGER
on their own. If students were to practise retrieval of information while studying this strategy would have the potential to greatly improve academic performance. However, we do not know the extent to which students practise recall while they study in real-world educational settings (relative to other less-effective strategies like repeated reading) or whether students who practise recall do so because they are aware of the mnemonic benefits. These are important and practically relevant research questions but few studies have been aimed at answering them (see, e.g., Kornell & Bjork, 2007). The objective of this research was to determine the extent to which students practise recall relative to other study strategies in real-world educational settings. In addition we wanted to examine whether students who choose to engage in retrieval practice do so because they know that testing promotes long-term retention. Another reason students may use testing during studying is to determine what information is known and what is not known so that future study time can be allocated to the unknown material (see Dunlosky, Hertzog, Kennedy, & Thiede, 2005; Dunlosky, Rawson, & McDonald, 2002). This is a fine justification for testing but it differs from using testing as a learning device in its own right. To accomplish these goals we created a new study strategies questionnaire and surveyed a large sample of undergraduate students. Although there are a variety of study strategy inventories in the education literature (see Entwistle & McCune, 2004; Pintrich, Smith, Garcia, & McKeachie, 1993; Weinstein, Schulte, & Palmer, 1987) these and other inventories do not specifically assess whether students practise retrieval while studying. Our survey included a free report question asking students to list the strategies they use while studying and a forced report question that asked them to choose between repeated reading or repeated testing. The purpose of including both forced and free report question formats was to gain converging evidence aimed at the target issue and to circumvent possible response biases created by using either format alone (see Schuman & Presser, 1996; Schwarz, 1999). We predicted that relatively few students would report self-testing as a study strategy and that the majority of students would report choosing to reread or engage in some other non-testing activity when forced to choose a study strategy. We also predicted that most students who
selected self-testing would be unaware of the mnemonic benefits of testing. In the first section of this paper we provide a brief overview of relevant research on repeated reading, repeated testing, and students metacognitive awareness of the testing effect. Next we present the results of our survey of study strategies. In the final section we interpret the survey results in light of current theories of metacognition and self-regulated learning and then discuss the practical and educational implications of our findings.
MOTIVATION FOR THE SURVEY: PRIOR RESEARCH ON REPEATED READING VS REPEATED TESTING
The testing effect refers to the finding that taking a test enhances long-term retention more than spending an equivalent amount of time repeatedly studying. There are clear and direct implications of the testing effect for student learning. One way for students to enhance their learning would be to practise recalling information while studying. However, research on the testing effect has also shown that when students are asked to assess their own learning they sometimes fail to predict that testing enhances learning more than repeated reading (e.g., Karpicke & Roediger, 2008). In short, there is a rapidly growing body of research (briefly reviewed below) indicating that testing has powerful effects on learning but students lack metacognitive awareness of the testing effect. Students often report that they repeatedly read their notes or textbook while studying (Carrier, 2003; Pressley, Van Etten, Yokoi, Freebern, & Van Meter, 1998; Van Etten, Freebern, & Pressley, 1997). Yet there are several reasons to question the effectiveness of repetitive reading beyond reading a single time. Basic research on memory has shown that spending extra time maintaining or holding items in memory does not by itself promote learning (Craik & Watkins, 1973) and students may spend large amounts of additional time studying despite no gain in later memory for the items, a phenomenon called labour-in-vain during learning (Nelson & Leonesio, 1988). Recent research with educationally relevant materials has shown that repeatedly reading prose passages produces limited benefits beyond a single reading (Amlund, Kardash, & Kulhavy, 1986; Callender & McDaniel, 2009).
METACOGNITIVE STRATEGIES IN LEARNING
473
This is especially true when repeated readings are massed together in a single learning session, although spaced rereading tends to produce positive effects (Rawson & Kintsch, 2005). In short, memory research has shown many times that repetitive reading by itself is not an effective strategy for promoting learning and long-term retention (for review, see McDaniel & Callender, 2008). In contrast, several studies have shown that repeated testing is a potent method for producing robust learning. In one of our studies (Karpicke & Roediger, 2008) we had students learn a set of Swahili vocabulary words across alternating study and test periods. In study periods students studied a Swahili word and its English translation (mashua boat) and in test periods they saw the Swahili words as cues to recall the English words (mashua ?). The students learned the words in one of four conditions and students in all conditions took a final test 1 week after initial learning. In two learning conditions, once a word was correctly recalled it was dropped from further test periods. The students who recalled each word only once in these two conditions recalled just 35% of the items on the final test a week later. In the other two conditions students continued to repeatedly recall words even after they had recalled them once. Students who repeatedly recalled the words during learning recalled about 80% of the items on the final test. Repeated retrieval practice*even after students were able to successfully recall items in the learning phase* produced large positive effects on long-term retention. Were students aware of the effect of repeated testing on long-term retention? At the end of the initial learning phase we asked students to predict how many pairs they would recall on the final test a week later. There was no difference in average predictions across the four conditions: All groups predicted they would recall about 50% of the items. Despite the large effect of repeated retrieval on retention, students were not aware of the mnemonic benefit of testing. Similar findings have occurred in other experiments examining the testing effect and students judgements of learning (e.g., Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008; Karpicke, McCabe, & Roediger, 2006; Roediger & Karpicke, 2006b). In sum, basic laboratory research on human learning and memory has shown that (1) repeated reading by itself is a questionable and often ineffective study strategy, (2) repeated retrieval
practice produces robust learning and long-term retention, but (3) students appear to lack metacognitive awareness of the testing effect. The implication of this basic research is that students may not practise retrieval when they study in realworld educational settings. Instead they may spend their time repeatedly reading material when they study. The objective of our survey was to examine the prevalence of retrieval practice, relative to other study strategies, in students real-world study behaviours and students metacognitive awareness of the benefits of self-testing.
A SURVEY OF STUDENTS LEARNING STRATEGIES
One reaction we have encountered when we present our research on the testing effect goes something like this: This is completely obvious. Of course testing enhances learning. We already knew this. None of this is new or surprising. Perhaps the testing effect is obvious to some instructors*but is it obvious to students? If so we would expect students to report that they frequently practise recall while studying. But our basic laboratory research has consistently shown that students lack metacognitive awareness of the testing effect. In fact students sometimes predict that repeated reading will produce better longterm retention than repeated testing (Roediger & Karpicke, 2006b). The intent of our survey was to determine whether students self-reported study behaviours would converge with our laboratory findings.
Method
We surveyed 177 undergraduate students at Washington University in St. Louis about the strategies they use to study for exams. The students were participants in various learning and memory experiments in our laboratory and they completed the survey at the very end of their experimental session. Washington University students are a select group with average SAT scores greater than 1400 (Verbal' Quantitative). Our survey included two questions aimed at identifying how often the students practised recalling information while studying. Question 1 was an open-ended free report question in which students listed the strategies they used when
474
KARPICKE, BUTLER, ROEDIGER
studying and then rank ordered the strategies in terms of how frequently they used them. All 177 students answered Question 1. Question 2 was a forced report question that asked students to imagine they were studying a textbook chapter for an exam and to choose one of three alternatives: (1) repeated reading of the chapter, (2) practising recall of material from the chapter (with or without the opportunity to reread the chapter, in different versions of this question), or (3) engaging in some other study activity. A total of 101 students answered Version 1 of Question 2 (testing without restudy) and the other 76 students answered Version 2 (testing with restudy). Students completed the entire questionnaire in about 5 to 10 minutes. Our goals were to identify students typical study strategies and to assess how frequently they repeatedly read material or engaged in retrieval practice, and our analysis focused on the frequency with which students reported these particular strategies.
Results
Question 1: Students free report of study strategies. The first question on the survey asked: What kind of strategies do you use when you are studying? List as many strategies as you use and rank-order them from strategies you use most often to strategies you use least often. We initially reviewed all responses from all students. Based on our initial assessment we identified 11 strategies that occurred relatively frequently (more than once across all student responses). Two independent raters then categorised all responses. There was close to 100% agreement between the two raters and the first author resolved any scoring discrepancies. Figure 1 shows the frequency distribution of the number of strategies listed by students in response to Question 1. The figure shows that most students listed and described three strategies (M 0 2.9). Table 1 shows the 11 strategies and the percent of students who listed each strategy. The table also shows the percent of students who ranked each strategy as their number one strategy and the mean rank of each strategy. Repeated reading was by far the most frequently listed strategy with 84% of students reporting it. Not only did students indicate that they repeatedly read while studying but they also indicated that rereading was a favoured strategy*55% of students reported that rereading was the number one strategy they used when studying. Table 1 also
Figure 1. Frequency distribution showing the number of strategies listed by students.
shows another key finding: Only 11% of students (19 of 177) reported that they practised retrieval while studying. These students unambiguously indicated in their list of strategies that they practised testing themselves by recalling information while they studied. Only 1% (2 of 177 students) identified practising recall as their number one strategy. The results in Table 1 clearly show that a large majority of students repeatedly read their textbook or notes but relatively few students engage in self-testing by practicing recall while studying. Table 1 also shows that students reported other strategies that could be interpreted as forms of self-testing. For example, 43% of students indicated that they answer practice problems while studying and 40% reported using flashcards. Each activity could be interpreted as a type of selftesting, but of course there are ways students might use these study methods without engaging in retrieval practice. For example, students may read practice questions and then look up and copy answers from the text. This would qualify as answering practice problems but students who do this would not be practising or even attempting recall of the answers. Likewise, students may write facts on flashcards and repeatedly read them rather than practising recall. A clear limitation of the free response question is that our procedure did not prompt each student to
METACOGNITIVE STRATEGIES IN LEARNING TABLE 1 Results of Question 1 Strategy 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Rereading notes or textbook Do practice problems Flashcards Rewrite notes Study with a group of students Memorise Mnemonics (acronyms, rhymes, etc) Make outlines or review sheets Practise recall (self-testing) Highlight (in notes or book) Think of real life examples Percent who list strategy 83.6 42.9 40.1 29.9 26.5 18.6 13.5 12.9 10.7 6.2 4.5 (148) (76) (71) (53) (47) (33) (24) (23) (19) (11) (8) Percent who rank as #1 strategy 54.8 12.4 6.2 12.4 0.5 5.6 2.8 3.9 1.1 1.6 0.5 (97) (22) (11) (22) (1) (10) (5) (7) (2) (3) (1) Mean rank 1.5 2.1 2.6 1.8 2.9 2.0 2.4 2.1 2.5 2.3 2.8
475
Percent of students listing different learning strategies, percent who ranked strategies as their #1 strategy, and mean rankings of strategies. Raw numbers of students are in parentheses. Mean number of strategies listed was 2.9 (SD00.96). Percentages of students indicating their #1 strategy do not add to 100% because some students merged multiple strategies when reporting their #1 strategy (e.g., indicating that rereading and rewriting notes were their #1 strategy).
elaborate on potentially ambiguous answers (cf. the ethnographic interviewing technique of Pressley and colleagues; Pressley et al., 1998; Van Etten et al., 1997). Nevertheless, even if we considered the 40% of students who use flashcards or the 43% who answer practice problems as students engaging in forms of self-testing, these percentages are dwarfed by the 84% of students who repeatedly read while studying. The results of Question 1 indicate that repeated reading is the most popular study strategy among college students (see too Carrier, 2003), far more popular than practising retrieval, even though retrieval practice is a more effective study strategy. Students listed a variety of study strategies but indicated that they use these alternative study far strategies less frequently than repeated reading. Question 2 asked students to choose repeated reading or self-testing and prompted them to explain the reasoning behind their choice. By including a second question in forced report format we hoped to find converging evidence and to resolve ambiguities inherent in our first openended free report question. Question 2: Forced report questions about repeated studying vs testing. Question 2 was a forced report question about repeated studying versus repeated testing. There were two versions of the question. Version 1 asked students to consider testing without going back and restudying, and Version 2 involved testing followed by restudying (to get feedback after attempting recall). The first version was given to 101 students
and the second version was given to 76 students. Version 1 of Question 2 was as follows: Imagine you are reading a textbook chapter for an upcoming exam. After you have read the chapter one time, would you rather: A. Go back and restudy either the entire chapter or certain parts of the chapter. B. Try to recall material from the chapter (without the possibility of restudying the material). C. Use some other study technique. The students were asked to select one alternative and write a brief explanation for their choice. The scenario described in the question was based directly on our research showing that taking a recall test, even without feedback, enhances long-term retention more than spending the same amount of time restudying (Roediger & Karpicke, 2006b). Table 2 shows the percentage of students who chose to restudy, self-test, or do something else after reading a textbook chapter. Most students unambiguously selected an alternative and explained their choice, but four students gave ambiguous responses that could not be scored. The table shows that 57% of students chose to restudy (option A) and 21% indicated that they would use some other study technique (option C). Thus 78% of students indicated they would not want to test themselves after reading a textbook chapter. Only 18% of the students indicated that they would self-test after studying (option B). To examine students metacognitive awareness of the
476
KARPICKE, BUTLER, ROEDIGER TABLE 2 Version 1 of Question 2
Imagine you are reading a textbook chapter for an upcoming exam. After you have read the chapter one time, would you rather: A. Go back and restudy either the entire chapter or certain parts of the chapter B. Try to recall material from the chapter (without the possibility of restudying the material) C. Use some other study technique 57.4
Overall (58)
Test for feedback
Test to practise recall
17.8 20.7
(18) (21)
9.9
(10)
7.9
(8)
Percent of students who chose to restudy, self-test (without restudying), or do something else after reading a textbook chapter. Raw numbers of students are in parentheses (N 0101). We were unable to score ambiguous responses given by four students.
mnemonic benefits of testing we separated students responses based on their explanation for why they chose self-testing. This analysis showed that 10% of all students (or more than half of those who chose self-testing) reported they would self-test to generate feedback and guide their future studying (even though Version 1 of this question stated that students could not restudy after testing). Only 8% of all students indicated that they would test themselves because practising retrieval would help them do well on the upcoming exam. This pattern of responding suggests that most students were unaware of the mnemonic benefits of self-testing. The results of Version 1 of this forced report question provide converging evidence with our first free report question. Relatively few students reported that they would test themselves after studying a textbook chapter and even fewer indicated they would test themselves because they knew the act of practising recall was valuable for learning. In Version 2 of Question 2 the scenario and alternatives were identical to Version 1 except that option B read Try to recall material from the chapter (with the possibility of restudying afterward). We imagined this would increase the number of students choosing testing perhaps to levels near ceiling if students recognised that testing followed by rereading would produce far superior learning to rereading without testing. Table 3 shows the percent of students who chose each option. The percentage of students choosing self-testing increased when students could reread after the test (42% in Question 2 vs 18% in Question 1) and the percentage was about equal to the percentage of students choosing repeated reading (42% vs 41%). Students explanations of their choices indicated that the increased like-
lihood of choosing testing was due to the possibility of restudying after the test. Of the 32 students who chose self-testing, 25 provided unambiguous explanations that we categorised as testing for feedback or testing to practise recall. A total of 23 students (30%) indicated that they would test themselves to generate feedback they could use when restudying whereas only two students (3%) chose testing because they believed the act of practising recall would help them remember in the future. The results of Version 2 of Question 2 expand on the results of Version 1 by showing that students were more likely to select self-testing when they could restudy after testing but that very few students are aware that the act of practising recall itself enhances learning. What is perhaps most striking about the data in Table 3 is that even when students had the option of rereading after selftesting, the majority of students (58%) continued to indicate that they would not test themselves.
DISCUSSION
The objective of this research was to collect benchmark data on college students real-world study behaviours to assess how often students use retrieval practice relative to other strategies and whether they know about the mnemonic benefits of self-testing. Our basic laboratory studies suggested that students are not aware of the testing effect, leading us to predict that they may not practise retrieval while studying in real-world settings. The results of our survey support this prediction. The majority of students indicated that they repeatedly read their notes or textbook while studying. Relatively few reported that they
METACOGNITIVE STRATEGIES IN LEARNING TABLE 3 Version 2 of Question 2 Imagine you are reading a textbook chapter for an upcoming exam. After you have read the chapter one time, would you rather: A. Go back and restudy either the entire chapter or certain parts of the chapter B. Try to recall material from the chapter (with the possibility of restudying afterward) C. Use some other study technique 40.8
477
Overall (31)
Test for feedback
Test to practise recall
42.1 17.1
(32) (13)
30.3
(23)
2.6
(2)
Percent of students who chose to restudy, self-test followed by restudying, or do something else after reading a textbook chapter. Raw numbers of students are in parentheses (N 076).
tested themselves and of those who engaged in self-testing only a handful reported doing so because they believed the act of practising retrieval would improve their learning. Our survey results point to the conclusion that many students do not view retrieval practice as a strategy that promotes learning. If students do practise recall or test themselves while studying they do it to generate feedback or knowledge about the status of their own learning, not because they believe practising recall itself enhances learning. Our results agree with laboratory experiments showing that students lack metacognitive awareness of the testing effect when they monitor their own learning. A growing body of research has shown that students sometimes predict that practising retrieval will produce no effect on retention (Karpicke & Roediger, 2008) or that they will remember more in the long term if they repeatedly study material rather than test it (Agarwal et al., 2008; Karpicke et al., 2006; Roediger & Karpicke, 2006b). If we assume that metacognitive monitoring processes guide students decisions to choose different learning strategies*an assumption at the core of the influential monitoring-and-control framework of metacognition (Nelson & Narens, 1990)*then the implication of these laboratory results is that students may not choose to test themselves when they regulate their own learning in real-world educational settings. Our survey data confirm that this lack of awareness of the testing effect has consequences for students real-world study behaviours. In addition to agreeing with basic laboratory findings our survey results also agree to some extent with a recent survey by Kornell and Bjork (2007). They surveyed college students about their study behaviours and asked the students,
If you quiz yourself while you study . . . why do you do so? The students selected one of four alternatives: 18% selected I learn more that way than I would through rereading; 68% selected To figure out how well I have learned the information Im studying; 4% indicated I find quizzing more enjoyable than rereading; and 9% said I usually do not quiz myself. Kornell and Bjorks data indicate that the majority of students (91%) do quiz themselves while studying but few do so because they view the act of quizzing itself as a method of enhancing learning (Kornell and Bjork reasoned that the 18% of students who selected I learn more that way than I would through rereading believed that quizzing produced a direct mnemonic benefit; cf. Roediger & Karpicke, 2006a). Likewise, our survey data indicate that few students view practising recall as an activity that enhances learning. However, far more students indicated that they tested themselves in the Kornell and Bjork survey than in our study, and this may be due to a difference in survey procedures. Whereas we used a combination of free and forced report questions to gauge how often students practise retrieval, Kornell and Bjork used one question focused on why students might quiz themselves and the framing of this question may have influenced students responses (see Schuman & Presser, 1996; Schwarz, 1999). It is well known that a single question can be framed in different ways and alter the choices and decisions people make (Tversky & Kahneman, 1981). Nevertheless our results generally agree with those of Kornell and Bjork in showing that few students view retrieval practice as a method of enhancing learning. Further, the differences between the two sets of results highlight potentially important differences between free and forced report methods of questioning.
478
KARPICKE, BUTLER, ROEDIGER
Our results fit with the broad theoretical notion that students experience illusions of competence when monitoring their own learning (Bjork, 1999; Jacoby, Bjork, & Kelley, 1994; Koriat & Bjork, 2005). Koriat and Bjork (2005) argued that illusions of competence tend to occur when students judgements of learning are biased by information available during study but not available during testing (see also Jacoby et al., 1994). Several experimental findings are consistent with this view. For example, students judgements of learning are less accurate when made in study trials than in test trials (Dunlosky & Nelson, 1992). Students are less accurate at judging the difficulty of anagrams when the solution is present than when it is not (Kelley & Jacoby, 1996). We believe repeated reading produces a similar illusion of competence. Specifically, repeatedly reading material like text passages increases the fluency or ease with which students process the text. Students may base their assessments of their learning and comprehension on fluency even though their current processing fluency with the text right in front of them, is not diagnostic of their future retention. Our survey results show that the illusions students experience during learning may have important consequences and implications for the decisions they make and the strategies they choose when studying on their own. Students generally exhibit little awareness of the fact that practising retrieval enhances learning. A clear practical implication is that instructors should inform students about the benefits of self-testing and explain why testing enhances learning. When students rely purely on their subjective experience while they study (e.g., their fluency of processing during rereading) they may fall prey to illusions of competence and believe they know the material better than they actually do. A challenge for instructional practice is to encourage students to base their study strategies on theories about why a particular strategy*like practising repeated retrieval*promotes learning and long-term retention.
Manuscript received 5 October 2008 Manuscript accepted 20 November 2008
REFERENCES
Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., & McDermott, K. B. (2008). Examining the testing effect with open- and
closed-book tests. Applied Cognitive Psychology, 22, 861876. Amlund, J. T., Kardash, C. A. M., & Kulhavy, R. W. (1986). Repetitive reading and recall of expository text. Reading Research Quarterly, 21, 4958. Bjork, R. A. (1999). Assessing our own competence: Heuristics and illusions. In D. Gopher & A. Koriat (Eds.), Attention and peformance XVII. Cognitive regulation of performance: Interaction of theory and application (pp. 435459). Cambridge, MA: MIT Press. Butler, A. C., & Roediger, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19, 514527. Callender, A. A., & McDaniel, M. A. (2009). The limited benets of rereading educational texts. Contemporary Educational Psychology, 34, 3041. Carrier, L. M. (2003). College students choices of study strategies. Perceptual and Motor Skills, 96, 5456. Craik, F. I. M., & Watkins, M. J. (1973). The role of rehearsal in short-term memory. Journal of Verbal Learning and Verbal Behavior, 12, 599607. Dunlosky, J., Hertzog, C., Kennedy, M. R. F., & Thiede, K. W. (2005). The self-monitoring approach for effective learning. Cognitive Technology, 10, 411. Dunlosky, J., & Nelson, T. O. (1992). Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20, 374 380. Dunlosky, J., Rawson, K. A., & McDonald, S. L. (2002). Inuence of practice test on the accuracy of predicting memory performance for paired associates, sentences, and text material. In T. J. Perfect & B. L. Schwartz (Eds.), Applied metacognition (pp. 6892). Cambridge, UK: Cambridge University Press. Entwistle, N., & McCune, V. (2004). The conceptual bases of study strategy inventories. Educational Psychology Review, 16, 325345. Jacoby, L. L., Bjork, R. A., & Kelley, C. M. (1994). Illusions of comprehension, competence, and remembering. In D. Druckman & R. A. Bjork (Eds.), Learning, remembering, believing: Enhancing human performance (pp. 5780). Washington, DC: National Academy Press. Karpicke, J. D., McCabe, D. P., & Roediger, H. L. (2006). Testing enhances recollection: Process dissociation estimates and metamemory judgments. Poster presented at the 47th Annual Meeting of the Psychonomic Society, Houston, TX. Karpicke, J. D., & Roediger, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57, 151162. Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319, 966968. Kelley, C. M., & Jacoby, L. L. (1996). Adult egocentrism: Subjective experience versus analytic bases for judgment. Journal of Memory and Language, 35, 157175. Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring ones knowledge during study.
METACOGNITIVE STRATEGIES IN LEARNING
479
Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 187194. Kornell, N., & Bjork, R. A. (2007). The promise and perils of self-regulated study. Psychonomic Bulletin & Review, 14, 219224. McDaniel, M. A., & Callender, A. A. (2008). Cognition, memory, and education. In H. L. Roediger (Ed.), Cognitive psychology of memory, Vol. 2 of Learning and memory: A comprehensive reference (pp. 819843). Oxford, UK: Elsevier. McDaniel, M. A., Roediger, H. L., & McDermott, K. B. (2007). Generalizing test-enhanced learning from the laboratory to the classroom. Psychonomic Bulletin & Review, 14, 200206. Nelson, T. O., & Leonesio, R. J. (1988). Allocation of self-paced study time and the labor-in-vain effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 676686. Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new ndings. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 26, (pp. 125141)). New York: Academic Press. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801813. Pressley, M., Van Etten, S., Yokoi, L., Freebern, G., & Van Meter, P. (1998). The metacognition of college studentship: A grounded theory approach. In D. J.
Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 347366). Mahwah, NJ: Lawrence Erlbaum Associates Inc. Rawson, K. A., & Kintsch, W. (2005). Rereading effects depend on time of test. Journal of Educational Psychology, 97, 7080. Roediger, H. L., & Karpicke, J. D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181210. Roediger, H. L., & Karpicke, J. D. (2006b). Test enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249 255. Schuman, H., & Presser, S. (1996). Questions and answers in attitude surveys. Thousand Oaks, CA: Sage. Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93 105. Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453458. Van Etten, S., Freebern, G., & Pressley, M. (1997). College students beliefs about exam preparation. Contemporary Educational Psychology, 22, 192212. Weinstein, C. E., Schulte, A. C., & Palmer, D. R. (1987). Learning and Study Strategies Inventory (LASSI). Clearwater, FL: H&H Publishing.