Saturday, October 14, 2006

Personal evaluation of ACT-R models

One of the ways to engage students to think more about ACT-R is to provide a way for them to see how well the theory of the mind corresponds to their personal experience. My approach to this is to start with something really basic like reaction time. Gunzelmann et al (2005) developed a well explained ACT-R model of Psychomotor Vigilance Task (PVT) and worked out the parameters to fit the their model to data from 88 hours of total sleep deprivation. With Terry Stewart's efforts this the ACT-R model was translated into PythonACT-R and so is available for students.

I have extended the model to include age and non-verbal IQ based on a brief review of the literature where both of these variables seem to operate through the speed of the production processing cycle of about 50 miliseconds. Age after 20 seems to slow the cycle and non-verbal IQ above 100 is associated with quicker reaction times and by inference faster processing cycles.

So the plan is to have students adjust the PVTdemoAGEPIQ.py model to make it specific to their individual situation and then run some simulations to see what the model predicts for situations like there own. To evaluate the prediction what is needed is a way for them to do the PVT and generate some comparable data. It is possible to use the environment part of PythonACT-R to collect some data but that requires a working ccmsuite simulation environment. As a backup to that situation (which is not possible with the standard ACT-R 6.0 simulator - it will run the task but not collect data as I understand it) I have modified the online reaction time tester by Jim Allen to be pretty close to the standard laboratory PVT equipment. This can be use to demonstrate the PVT situation with only ten trials (redemo10.html) or for actually collecting some comparison data with the 100 trial version (rtdemo100.html). Both situations: the PythonACT-R and the web page version produce a mean and standard deviation for the reaction times so comparisons can be straight forward. The difficulty of using an unstandardized version of the PVT is that at this point it may need to be calibrated but that raises the issue of experimental errors in replication which is a good discussion topic for students.

The teaching plan is that this "compare yourself to the PythonACT-R model" will provide both some interest and some challenges to students as they try to understand why the results came out the way they did. One interesting issue that I do not know how to address in a model is the consistant finding the women have slower reaction times by about 40 milliseconds and make fewer false alarms. I am hopeful that this will replicate in the class results and that it will lead to an interesting discussion of just what women are doing differently in the PVT situation (researchers have speculated that the difference is due to men an women using different "cognitive strategies" on the same task with the same instuctions -- "as fast as possible").

Thursday, October 12, 2006

Using a model of reaction time as the starting point

Another article from the ACT-R collection that provides a nice ACT-R introduction to reaction-time and sleep deprivation is:

Gunzelmann, G., Gluck, K. A., Van Dongen, H. P. A., O'Conner, R. M., & Dinges, D. F. (2005). A neurobehaviorally inspired ACT-R model of sleep deprivation: Decreased performance in psychomotor vigilance. In B. G. Bara, L. Barsalou, and M. Bucciarelli (Eds.), Proceedings of the Twenty-Seventh Annual Meeting of the Cognitive Science Society, (pp. 857-862). Mahwah, NJ: Lawrence Erlbaum Associates.
[info and link to pdf] http://act-r.psy.cmu.edu/publications/pubinfo.php?id=596

For my teaching purposes this provides the bridge into the PythonACT-R models because the model in this article was translated into Python ACT-R by Terry Stewart with a little of my help.
http://brucelandon.douglas.bc.ca/2360w2007/demo/PVTdemo.py

The basic idea in building a theory is to start small and make it simple in the beginning.
Here the idea was to start with the simplest task - psychomotor vigilance task - "when the light goes on press the button as fast as you can" and work with simple reaction times as the data to model. It turns out that there is a very large literature in psychology about reaction time and its relation to many psychological variables: age, IQ, gender, personality, etc. Consequently the model of reaction-time and be a bridge from a simple PythonACT-R model with 3 productions to more complex models that explore intersting variables for students to think about.

ACT-R Teaching Materials from the ACT-R site

Some of the best ACT-R Teaching materials are on the ACT-R site and for my students the most popular one was the one page overview of the theory with diagrams at url:
http://act-r.psy.cmu.edu/about/

There are also 7 Tutorials about how the LISP ACT-R works (in html doc or pdf)
(with LISP code examples that run in ACT-R 6.0 environment)
# Unit1: Understanding Production Systems
# Unit2: Perception and Motor Actions in ACT-R
# Unit3: Attention
# Unit4: Base-level Learning and Accuracy
# Unit5: Activation and Context
# Unit6: production Utilities
# Unit7: Production Rule Learning

The best recent explanation of ACT-R theory by John Anderson that I have used in teaching is a complex article that introduced an extension of the theory with the imaginal buffer:
Anderson, J. R. (2005) Human symbol manipulation within an integrated cognitive architecture. Cognitive Science, 29(3), 313-341.
[info and link to pdf] at http://act-r.psy.cmu.edu/publications/pubinfo.php?id=580

This article was difficult for my students to read but well received. To make it more likely that they would process all of it I made a text-to-speech version on MP3 files for them and got positive feedback that this addion audio version helped.

Wednesday, October 11, 2006

PythonACT-R benefits for teaching

Because python is a very popular scripting language many people have contributed code solutions to many kind of problems. Today I spent some time getting one of the extras bundled with PythonACT-R working. The extra is a text-to-speech processor so that now the demonstration model on sequential memory has a talking experimenter and the participant speaks to recite what has been learned. I am hoping that all of this make a provocative in class demonstration next term.

The side benefit is that my little elizabot.py one page introduction to python programming now can have speech output as well. The responsive speaking program makes a nice introduction to the Artificial Intelligence bots and the Turing Test competition (Lobner prize). This in turn sets the stage for the discussion of the differences between AI models and ACT-R models. (code for all of the python course models will be available via my home page http://brucelandon.douglas.bc.ca

Saturday, October 07, 2006

ACT-R Teaching Materials

The approach is to make working with ACT-R theory of the mind have a low enough threshold so that undergraduates can come to appreciate computer simulation of human cognition. PythonACT-R developed by Terry Stewart enables both the theorizing by scripting mental processes to be easier as well as the evaluation of models by simulations and equivalence testing.
[Terry Stewart' Lecture Notes on PythonACT-R]