testing with quiztalker.py

quiztalker.py is a PythonACT-R model that both presents and speaks a multiple choice test - item by item. A this point it is pedogically useful for showing strategies for taking quizes by explaining the spoken part as a demonstration of inner speech. The teaching point being that when a student rushes in reading the question stem and does not pause before reading the answers then the answers/foils can sometimes change the meaning of the question and induce errors. The mechanism is spreading activation where the meanings of the foils spread to the meaning of the questions stem.

The quiztalker can be used with different voices in the windows SAPI interface. The quiz items can be added from a test development resource http://edutools.ca/xamtoolbox/links2tools.htm
that I developed for a Moodle project by using a little python script http://a.edutools.ca/actr/makequiztalker.py that converts text item texts into PythonACT-R models with a little cut and past of the reformated items. The demonstration example is a set of review questions from introductory psychology http://a.edutools.ca/actr/quiztalker.py

If the quiz items were created from glossary items using the xamtoolbox tools then it would be more feasible to develop a real PythonACT-R model to model the answering of the questions. This would require some access to a Latent Semantic Analysis engine to get the scaled similarities between the question stems and each of the item answers including the foils. The research by Landauer et al has found that using the semantic distances between the stem and the choices can enable reasonable quiz performance (getting a C on test created for introductory psychology material).

The quiztalker.py model provides a simple way to make the discussion of cognitive processing involved in answering multiple choice items more tangible and focused on what I think are the critical aspects: getting the meaning of the stem as a network of spreading activation and then selecting the answer that best fits with the question stem. By relating the cogntive processing involved with personal quiz performance with the xamtoolbox test development process of finding the better items based on analysis of group performance the modeling style of research can be contrasted with the correlational style of research on the same issues. The discussion can be extended into an assignment of completing an online self-test produced with http://edutools.ca/xamtoolbox/xt26test2selftest.htm where a multiple choice test is converted into a fillin the blank style quiz that encourages more active learning than traditional multiple choice testing (at least in my opinion) because the answer much be generated and simple strategies like "pick option C" do not work at all. (In one of my previous courses there was a r=.6 correlation between the number of self-tests completed and final course mark which encourages me to continue to to provide fillin self-test for students to "practice" with).