Experiment 3
0.20
-0.12
Experiment 4
0.36
0.00
37
Figure 1. Predicted accuracy on the final memory test as a function of JOL and practice condition. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4
Pr
e
di
ct
e
d
Ac
cur
ac
y
JOL Quartile
Restudy Retrieval
38
Figure 2. Average JOL within each JOL quartile during the study phase. 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4
A
ve
ra
ge
J
O
L
JOL Quartile
39
Figure 3. Accuracy for the practice phase (A) and the final memory test (B), by JOL quartile and practice condition in Experiment 1.
40
Figure 4. Accuracy on the final test by practice condition. Dotted lines show accuracy if all items were either restudied or tested. The solid line shows a criterion-based mix of both practice types. Across all JOL scores, adding more retrieval practice would be predicted to increase recall accuracy. 0 0.1 0.2 0.3 0.4 0 20 40 60 80 100
Ac
cur
ac
y
Criterion (Only Test items if equal or above that JOL)
Restudy All Test All Mix
41
Figure 5. Average JOLs within each JOL quartile during the study phase. 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4
A
ve
ra
ge
J
O
L
JOL Quartile
List 1 List 2 List 3
42
Figure 6. Accuracy for the practice phase and the final memory test, by JOL quartile and practice condition in Experiment 2.
43
Figure 7. Accuracy on the final test by practice condition. Across all JOLs, allowing more retrieval practice predicted greater recall accuracy.
0 0.1 0.2 0.3 0.4 0.5 0.6 0 20 40 60 80 100
Ac
cur
ac
y
Criterion (Only Test items if equal or above that JOL)
Restudy All Test All Mix
44
Figure 8. Accuracy for the practice phase (A) and the final memory test (B), by JOL quartile and practice condition for Experiment 3.
45
Figure 9. Accuracy on the final test by practice condition. Across all JOLs, allowing more retrieval practice predicted lower recall accuracy.
0 0.1 0.2 0.3 0.4 0.5 0.6 0 20 40 60 80 100
Ac
cur
ac
y
Criterion (Only Test items if equal or above that JOL)
Restudy All Test All Mix
46
Figure 10. Accuracy for the practice phase (A) and the final memory test (B), by JOL quartile and practice condition for Experiment 4.
47
Figure 11. Accuracy on the final test by practice condition. Greater recall accuracy was predicted by a mix of retrieval practice and restudy.
0.2 0.3 0.4 0.5 0 20 40 60 80 100
Ac
cur
ac
y
Criterion (Only Test items if equal or above that JOL)
Restudy All Test All Mix
48
Figure 12. Accuracy for the practice phase (A) and the final memory test (B), by JOL quartile and practice condition for Experiment 5.
49
Figure 13. Accuracy on the final memory test for honored vs. dishonored items, both overall (A) and split by the dishonor method (B).
50
Figure 14. Accuracy for honored and dishonored items, split by both restudy choice and how whether the item was in the top half of JOLs or the bottom half.
51
Figure 15. Memory for high and low value items, across practice conditions (A) or split by them (B).
52
Figure 16. Accuracy for the practice phase (A) and the final memory test (B), by JOL quartile and practice condition for Experiment 6.
53
Figure 17. The effects of honoring or dishonoring study choices, by study choice (B) or across choice (A).
54
55
56
Figure 20. Cued recall accuracy during the final memory test averaged across all six experiments.
57
REFERENCES
Benjamin, A. S., & Tullis, J. (2010). What makes distributed practice effective? Cognitive Psychology, 61(3), 228-247.
Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing, (pp. 185-205). Cambridge, MA: MIT Press.
Carpenter, S. K., Pashler, H., Wixted, J. T., & Vul, E. (2008). The effects of tests on learning and forgetting. Memory & Cognition, 36(2), 438-448.
De Jonge, M. O., Tabbers, H. K., & Rikers, R. M. J. P. (2015). The effect of testing on the retention of coherent and incoherent text material. Educational Psychology Review. 27(2), 305-315.
Fiechter, J. L., & Benjamin, A. S. (2017). Diminishing-cues retrieval practice: a memory- enhancing technique that works when regular testing doesn’t. Psychonomic bulletin & review, 1-9.
Fiechter, J. L., & Benjamin, A. S. (under review). Techniques for scaffolding retrieval practice: The costs and benefits of adaptive versus diminishing cues.
Finley, J. R., Benjamin, A. S., Hays, M. J., Bjork, R. A., & Kornell, N. (2011). Benefits of
accumulating versus diminishing cues in recall. Journal of Memory and Language, 64(4), 289-298.
Finn, B., & Metcalfe, J. (2007). The role of memory for past test in the underconfidence with practice effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(1), 238-244.
58
Hall, K. G., Domingues, D. A., & Cavazos, R. (1994). Contextual interference effects with skilled baseball players. Perceptual and Motor Skills, 78(3), 835-841.
Hennessee, J. P., Knowlton, B. J., & Castel, A. D. (2018). The effects of value on context-item associative memory in younger and older adults. Psychology and aging, 33(1), 46-56. Hogan, R. M., & Kintsch, W. (1971). Differential effects of study and test trials on long-term
recognition and recall. Journal of Verbal Learning and Verbal Behavior, 10(5), 562-567. Goode, S., & Magill, R.A. (1986). Contextual interference effects in learning three badminton
serves. Research Quarterly for Exercise and Sport, 57, 308-314.
Izawa, C. (1967). Function of test trials in paired-associate learning. Journal of Experimental Psychology, 75, 194–209.
Jang, Y., Wixted, J. T., Pecher, D., Zeelenberg, R., & Huber, D. E. (2012). Decomposing the interaction between retention interval and study/test practice: The role of retrievability. Quarterly Journal of Experimental Psychology, 65(5), 962-975.
Karpicke, J. D. (2009). Metacognitive control and strategy selection: deciding to practice
retrieval during learning. Journal of Experimental Psychology: General, 138(4), 469-486. Karpicke, J. D., & Roediger, H. L., III. (2007). Expanding retrieval practice promotes short-term
retention, but equally spaced retrieval enhances long-term retention. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 704-719.
Karpicke, J. D., & Roediger, H. L., III. (2010). Is expanding retrieval a superior method for learning text materials? Memory & Cognition, 38, 116-124.
Kimball, D. R., & Metcalfe, J. (2003). Delaying judgments of learning affects memory, not metamemory. Memory & Cognition, 31(6), 918-929.
59
Kimball, D. R., Smith, T. A., & Muntean, W. J. (2012). Does delaying judgments of learning really improve the efficacy of study decisions? Not so much. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(4), 923-954.
Koriat, A., Bjork, R. A., Sheffer, L., & Bar, S. K. (2004). Predicting one's own forgetting: the role of experience-based and theory-based processes. Journal of Experimental
Psychology: General, 133(4), 643-656.
Koriat, A., Nussinson, R., & Ackerman, R. (2014). Judgments of learning depend on how learners interpret study effort. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(6), 1624-1637.
Koriat, A., Sheffer, L., & Ma'ayan, H. (2002). Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice. Journal of Experimental Psychology: General, 131(2), 147-162.
Kornell, N., & Bjork, R. A. (2007). The promise and perils of self-regulated study. Psychonomic Bulletin & Review, 14(2), 219-224.
Kornell, N., & Bjork, R. A. (2008). Optimising self-regulated study: The benefits—and costs— of dropping flashcards. Memory, 16(2), 125-136.
Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 989-998.
Kornell, N., & Metcalfe, J. (2006). Study efficacy and the region of proximal learning framework. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(3), 609-622.
60
Kornell, N., & Son, L. K. (2009). Learners’ choices and beliefs about self-testing. Memory, 17(5), 493-501.
Landauer, T. K., & Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory (pp. 625- 632). London: Academic Press.
Leahy, W., Hanham, J., & Sweller, J. (2015). High element interactivity information during problem solving may lead to failure to obtain the testing effect. Educational Psychology Review, 27(2), 291-304.
Lee, T.D., & Magill, R.A. (1983). The locus of contextual interference in motor-skill acquisition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9, 730-746. Lee, T.D., & Magill, R.A. (1985). Can forgetting facilitate skill acquisition? In D. Goodman, R.
B. Wilberg, & I. M. Franks (Eds.), Differing perspectives in motor learning, memory, and control. Amsterdam: Elsevier. Pp. 3-22.
Magill, R. A., & Hall, K. G. (1990). A review of the contextual interference effect in motor skill acquisition. Human Movement Science, 9, 241-289.
Mitchum, A. L., Kelley, C. M., & Fox, M. C. (2016). When asking the question changes the ultimate answer: Metamemory judgments change memory. Journal of Experimental Psychology: General, 145(2), 200-219.
Roediger, H. L. & Karpicke, J. D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210. Roediger, H. L. & Karpicke, J. D. (2006b). Test-enhanced learning: Taking memory tests
61
Rowland, C. A. (2014). The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin, 140(6), 1432-1463.
Runquist, W. N. (1986). The effect of testing on the forgetting of related and unrelated
associates. Canadian Journal of Psychology/Revue canadienne de psychologie, 40(1), 65- 76.
Siler, J., & Benjamin, A.S. (under review). Long-Term Inference and Memory Following Retrieval Practice.
Simon, D.A., Lee, T.D., & Cullen, J. (2008). Win-Shift, Lose-Stay: Contingent Switching and Contextual Interference in Motor Learning. Perceptual and Motor Skills, 107, 407–418. Spellman, B. A., & Bjork, R. A. (1992). When predictions create reality: Judgments of learning
may alter what they are intended to assess. Psychological Science, 3(5), 315-317. Tauber, S. K. U., Dunlosky, J., & Rawson, K. A. (2015). The influence of retrieval practice
versus delayed judgments of learning on memory: Resolving a memory-metamemory paradox. Experimental psychology, 62(4), 254-263.
Tullis, J. G., Finley, J. R., & Benjamin, A. S. (2013). Metacognition of the testing effect: Guiding learners to predict the benefits of retrieval. Memory & Cognition, 41, 429-442. Tullis, J. G., Fiechter, J. L., & Benjamin, A. S. (2017). The Efficacy of Learners’ Testing
Choices. Journal of Experimental Psychology: Learning, Memory, and Cognition. Advance online publication.
van Gog, T., Kester, L., Dirkx, K., Hoogerheide, V., Boerboom, J., & Verkoeijen, P. P. J. L. (2015). Testing after worked example study does not enhance delayed problem-solving performance compared to restudy. Educational Psychology Review. 27(2), 265-289.
62
Vaughn, K. E., & Rawson, K. A. (2012). When is guessing incorrectly better than studying for enhancing memory? Psychonomic Bulletin & Review, 19(5), 899-905.
Wissman, K. T., Rawson, K. A., & Pyc, M. A. (2012). How and when do students use flashcards? Memory, 20(6), 568-579.
Wood, C. A., & Ging, C. A. (1991). The role of interference and task similarity on the acquisition, retention, and transfer of simple motor skills. Research Quarterly for Exercise and Sport, 62(1), 18-26.