Having completed the work for this project, we are now able to look at was has been done, and how to proceed from there.
Starting on the topic of readability assessment, there are a number of items that could do with further examination or exploration. This research focussed specifically on a single readability assessment method, the Cloze-test. However, as examined in Section 2.1, this is not the only readability assessment method. Other options, like the C-task, have not been used in this project as a potential digital assessment method.
So far all readability assessment methods that have been covered in this report are traditional (paper) methods which are currently in use. While this does provide an ideal basis to work off, a method specifically researched, tested and developed to make use of current advances in Artificial Intelligence (AI), machine learning and processing power could potentially result in an incredibly powerful and accurate sys- tem without the need for it to be based on (old-school) readability assessment meth- ods like Cloze- or C-tests.
One area which was heavily reviewed and researched before starting this project was the area of gamification. Gamification has only been applied sparingly in places such as the user-test in Chapter 3. This is unfortunate since gamification would most likely be the deciding factor in the eventual success or failure of large scale read- ability assessment system based on child feedback via a gamified solution. Looking at the research done by people like Louis von Ahn (see Section 2.2), there are numerous options and methods already out there which have successfully been im- plemented. It would be very interesting to know how these gamification elements could be combined with the underlying principle of readability assessment into a fi- nal system which provide a fun and replayable experience to its users, and useful (accurate) readability data to us. This gamification element would eventually need to be the layer on top of the automated readability assessment system(s).
On the topic of child computer interaction (see Section 2.3), there are a few items left to consider when talking about the implementation of a final working system. One of the goals remained to be a readability assessment method/tool which can be used by children at different levels of reading proficiency, from young to old(er).
How do you design a platform in such a way that it correctly assesses readability, all the while being fun and enjoyable for multiple age-groups and levels of (computer) proficiency? And how can modern technology help in this aspect?
Finally, there is one important topic left to discuss that is essential to the success of the final implementation of this system, which is the topic of scalability. For collect- ing a large amount of feedback through interactions with a system, scalability has to be accounted for. Now, the groundwork for this has been done in Chapter 4 by researching and developing automated solutions for creating Cloze-tests. However, this was only the first of many steps needed to deliver a robust and complete solu- tion that would need to be developed in order to reach the ultimate end-goal of this project, a fully fledged working large scale online gamified readability assessment system.
Bibliography
[1] (2015) Internet users (per 100 people). The World Bank. [Online]. Available: http://data.worldbank.org/indicator/IT.NET.USER.P2/countries/1W? display=graph
[2] R. Pijpers and J. de Haan,Contact!: kinderen en nieuwe media. Bohn Stafleu van Loghum, 2010.
[3] R. Appel and A. Vermeer, “Tweede-taalverwerving en tweede-taalonderwijs,” 1994.
[4] K. Raaijmakers, “”laat mij het zelf beoordelen”. een onderzoek naar de betrouw- baarheid en validiteit van moderne methodieken om tekstbegrip te voorspellen en te meten bij kinderen in de groepen 5 t/m 8 van de basisschool.” University of Utrecht Master Thesis, 2015.
[5] D. Qian, “Assessing the roles of depth and breadth of vocabulary knowledge in reading comprehension,”Canadian modern language review, vol. 56, no. 2, pp. 282–308, 1999.
[6] L. Verhoeven and A. Vermeer, “Woordenschat van leerlingen in het basis-en mlk-onderwijs,” Pedagogische studi ¨en, vol. 69, no. 3, pp. 218–234, 1992. [7] K. E. Stanovich, “Matthew effects in reading: Some consequences of individual
differences in the acquisition of literacy,” Reading research quarterly, pp. 360– 407, 1986.
[8] A. Biemiller, “Vocabulary: Needed if more children are to read well,” Reading Psychology, vol. 24, no. 3-4, pp. 323–335, 2003.
[9] K. Cain and J. Oakhill, “Assessment matters: Issues in the measurement of reading comprehension,” British Journal of Educational Psychology, vol. 76, no. 4, pp. 697–708, 2006.
[10] W. L. Taylor, “Cloze procedure: a new tool for measuring readability.”Journalism quarterly, 1953.
[11] R. G. Abraham and C. A. Chapelle, “The meaning of cloze test scores: An item difficulty perspective,” The Modern Language Journal, vol. 76, no. 4, pp. 468–479, 1992.
[12] M. Hathcock, “Cloze procedure,”esl4teachers.pbworks.com, 2013.
[13] R. Kraf, L. Lentz, and H. Pander Maat, “Drie nederlandse instrumenten voor het automatisch voorspellen van begrijpelijkheid-een klein consumentenonder- zoek,”Tijdschrift voor Taalbeheersing, vol. 33, no. 3, pp. 249–265, 2011.
[14] P. Gibbons, P. E. T. Associationet al.,Learning to learn in a second language. Heinemann Portsmouth, NH, 1991.
[15] P. Wonghiransombat, “The cloze test and the c-test,” Thammasat University Journal, vol. 31, no. 2, 2013.
[16] A. S. Gellert and C. Elbro, “Cloze tests may be quick, but are they dirty? devel- opment and preliminary validation of a cloze test of reading comprehension,” Journal of Psychoeducational Assessment, 2012.
[17] J. McGonigal,Reality is broken: Why games make us better and how they can change the world. Penguin, 2011.
[18] S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From game design ele- ments to gamefulness: defining gamification,” in Proceedings of the 15th In- ternational Academic MindTrek Conference: Envisioning Future Media Envi- ronments. ACM, 2011, pp. 9–15.
[19] E. Halter, From Sun Tzu to xbox: War and video games. Thunder’s Mouth Press, 2006.
[20] L. Von Ahn and L. Dabbish, “Designing games with a purpose,” Communica- tions of the ACM, vol. 51, no. 8, pp. 58–67, 2008.
[21] T. W. Malone, “Heuristics for designing enjoyable user interfaces: Lessons from computer games,” inProceedings of the 1982 conference on Human factors in computing systems. ACM, 1982, pp. 63–68.
[22] E. A. Locke and G. P. Latham, A theory of goal setting & task performance. Prentice-Hall, Inc, 1990.
[23] T. Frattesi, D. Griesbach, J. Leith, T. Shaffer, and J. DeWinter, “Replayability of video games,”IQP, Worcester Polytechnic Institute, Worcester, 2011.
BIBLIOGRAPHY 74
[24] P. Markopoulos and M. Bekker, “Interaction design and children,” Interacting with computers, vol. 15, no. 2, pp. 141–149, 2003.
[25] D. S. Acuff and H. Robert, “The psychology of marketing to kids: what kids buy and why,”New York: The Free Press, 1997.
[26] J. P. Hourcade, “Interaction design and children,” Foundations and Trends in Human-Computer Interaction, vol. 1, no. 4, pp. 277–392, 2008.
[27] D. J. Cech and S. T. Martin,Functional movement development across the life span. Elsevier Health Sciences, 2002.
[28] R. Kail, “Developmental change in speed of processing during childhood and adolescence.”Psychological bulletin, vol. 109, no. 3, p. 490, 1991.
[29] B. Shneiderman, “Promoting universal usability with multi-layer interface de- sign,” in ACM SIGCAPH Computers and the Physically Handicapped, no. 73- 74. ACM, 2003, pp. 1–8.
[30] B. Shneiderman and C. Plaisant, “Designing the user interface: Strategies for effective human-computer interaction,” 1987.
[31] H. E. Jochmann-Mannak, T. W. C. Huibers, and T. J. M. Sanders, “Children’s in- formation retrieval: beyond examining search strategies and interfaces,” in The 2nd BCS-IRSG Symposium: Future Directions in Information Access, London, ser. eWic Series. London: British Computer Society, September 2008, pp. 64–72.
[32] C. L. Borgman, V. A. Walter, and S. G. Hirsh, “The science library catalog: A springboard for information literacy,” School Library Media Quarterly, vol. 24, no. 2, pp. 105–110, 1996.
[33] A. Druin, “The role of children in the design of new technology,”Behaviour and information technology, vol. 21, no. 1, pp. 1–25, 2002.
[34] A. van den Bosch, P. Nauts, and N. Eckhardt, “A kids’ open mind common sense.” inAAAI Fall Symposium: Commonsense Knowledge, 2010.
[35] P. Ajideh and S. Mozaffarzadeh, “C-test vs. multiple-choice cloze test as tests of reading comprehension in iranian efl context: Learners’ perspective,”English Language Teaching, vol. 5, no. 11, p. p143, 2012.
[36] J. C. Alderson, “Testing reading comprehension skills (part two). getting stu- dents to talk about taking a reading test.(a pilot study),” Reading in a foreign language, vol. 7, no. 1, pp. 465–503, 1990.
Comparison of School Year
Equivalents (US/UK/NL/FR)
This appendix entry includes a table comparing various different international school systems based on year equivalence.
Comparison of School Year Equivalents
United States, Great Britain, The Netherlands and France Source: The American School of The Hague (https://www.ash.nl/)
Entry age
United States Grade Level British System
Grade Level
The Netherlands Grade Level French Lycée Grade level 3 Elementary School Preschool Junior School Basisschool Maternelle Petite 4 Transition
Kindergarten Reception Groep 1 Moyenne
5 Kindergarten Year 1 Groep 2 Grande
6 Grade 1 Year 2 Groep 3 CP
7 Grade 2 Year 3 Groep 4
Élémentarie
CE1
8 Grade 3 Year 4 Groep 5 CE 2
9 Grade 4 Year 5 Groep 6 CM 1
10 Middle School Grade 5 Year 6 Groep 7 CM 2
11 Grade 6 Senior School Year 7 Groep 8 Collège 6 ème 12 Grade 7 Year 8 Middelbare School (VWO) Brugklas 5 ème
13 Grade 8 Year 9 2e Jaar 4 ème
14 High School Grade 9
GCSE Year 10 3e Jaar 3 ème
15 Grade 10 Year 11 4e Jaar
Lycée
2 nde
16 Grade 11
A Levels Year 12 5e Jaar 1 ère
User-Test Cloze Forms
The included documents in this appendix entry contain all the 10 stories in Cloze-test form which were used in the user-test of stage A, see Chapter 3. These documents were also used for the offline portion of Cloze-test minus a few changes made for inclusion in this report. These changes were as follows:
An identification letter for each story was added next to the tile in brackets and every correct multiple choice answer is highlighted.