Frank E. Ritter
21 Oct 1998
Our aims are to understand more clearly how learning occurs. This is done by creating and testing cognitive models that learn, and, as necessary step, to make model creation, explanation, evaluation, and improvement more routine.
Better theories of learning will help design better computer interfaces and instructional materials, amoung a host of other applications.
Since January 1993, we have been busy working on creating cognitive models, and working on ways to create them more routinely. Progress has been made on several fronts, which we will note here. Interestingly, the web itself has played a substantial role.
The name Cognitive Modeling unit (or CMu) is used because we are "too small to be a group", yet we have had some cohesion. The unit has gotten larger, more a trio now, having had full-time RAs and usually one or more post-graduate students. We also have hosted a lot of visitors, over 15 so far, who stay from a day or a semester. Sam Marshall was the first RA. Gordon Baxter is the current and has been here longest. Gary Jones has recently graduated with a PhD (May, 1999). Two students have graduated with a MSc in Intelligent Knowledge-based Systems. The latest visitor is Moritz Baumann, from the Technical University of Berlin, which we have several informal ties.
We have started to work with Fernand Gobet, but our work comparing different models that learn has not reached a point where we have a report available.
The work can be broken into several components that are taken up in turn. They include:
These topics are taken up in turn below, orgainized into a single file for convenience of printing and reading. A list of informal notes too small or too numerous to get listed here is also available.
To create models that learn, in a variety of ways and in a variety of tasks.
We've created several models using these tools and processes. Several of these models can be seen directly as models of decision making, some them explicitly so. In most cases, we tend to test them using sequential data. We have used these models to understand the data and to provide theories of behavior that are not always visible without a a theory of how the task is preformed sufficient to perform the task.
Pete Bibby and Sam Marshall created a model in Soar that diagnosed faults in an electrical circuit. Along with Shara Lochun, we tested its sequential predictions with protocol data because the model matches aggregate data fairly well. A paper is available showing that while the model now learns at about the same rate as subjects do, comparison with the protocol data shows that the rate of learning within a trial is not the same. This has allowed us to understand where and how learning occurs on a finer level.
With a variety of people we've been working on creating models that interact with their environment and learn from doing so. The best example of this is a model that learns by interacting with a very simple ATC simulation to land a plane. The model moves an eye (fovea) and hand (mouse) around a screen, and after the plan crashes, learns through reflection how to avoid this. This work has shown that interaction is a crucial aspect of problem solving, and that future work in this area should pay heed to reflection and interaction.
A summary of some of the lessons we have learned was presented at the ACT-R workshop.
We've worked on several miscellaneous models. These include a model that solves job shop scheduling tasks, a description of how and where emotions could be included in cognitive models, and a reimplementation and generalization of a model of physics problem solving. Theoretically, these models should provide a basis for creating further models. Practically, there remain problems in doing so.
Nerb, J., Krems, J., & Ritter, F. E. (1993). Rule learning and the power law: A computational model and empirical results. Using a computer model to examine learning and the power law. In Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society ,765-770. Hillsdale, NJ: LEA.
Ritter, F. E. (1993). Three types of emotional effects that will occur in cognitive architectures. Workshop on architectures underlying motivation and emotion, The University of Birmingham 11-12 August. Also presented as a colloquium at the MRC-APU in Cambridge, October, 1993. and at EuroSoar7 and at North American Soar 14.
Ritter, F. E., & Baxter, G. D. (1996). Able, III: Learning in a more visibly principled way (Tech. Report No. 40. 8 pages.). ESRC CREDIT, Dept. of Psychology, U. of Nottingham. In the Proceedings of the 1st European Cognitive Modeling Workshop, November, 1996. Includes new Soar interface and two models.
We created a model that not only solves the Tower of Nottingham, but also learns and shows appropriate behavior at several levels of development. This work should help distangle whether development is best understood as changes to the cognitive architecture, or as the result of learning. This work was first started with Josef Nerb, but is now the PhD work of Gary Jones. At the start of this work, he was funded to visit major cognitive modeling sites in the US. A review of this trip has been reported in an online ONR newsletter. At this point in time, the model matches adult data fairly well, after being modified, it matches children data almost as well, and it shows that several theories of what develops (such as Piaget's) are either incomplete or wrong.
Jones, G., & Ritter, F. E. (1998). Initial explorations of modifying architectures to simulate cognitive and perceptual development. In Proceedings of the Second European Conference on Cognitive Modelling. 44-51. Nottingham: Nottingham University Press.
Ritter, F. E., Nerb, J., Kindsmüller, M. (1994). Steps towards a series of models for a developmental task. Overheads included in the Proceedings of the EuroSoar 8 Workshop. 95-99. Graduate School of Experimental Psychology, U. of Leiden.
I have started to create a model of learning in two person games using Soar to compare with Dieter Wallach's ACT-R model. Currently, his model fits better. We've learned a bit about each other's architectures and about architectures in general by doing this.
We have worked on several tools for creating and testing models. These can be organized under several subheadings: tools to tie models to simulated worlds, and tools for creating models.
We have been working for several years on how to tie models, particularly Soar and Act-R type models to simulations. This is necessary in order to understand how perception influences behavior, and to provide a more complex set of stimuli to our models. Currently, we are having the eye and hand implemented in a commercially available graphical rapid prototyping language (SLGMS). In 1998 we expect to use it to allow a model to see and interact with an interface. We have learned that perception influences cognition, not only slowing it down in a general way, but also, for example, by restricting its behavior by precluding knowing everything at once.
Ritter, F. E., & Major, N. P. (1995). Useful mechanisms for developing simulations for cognitive models. AISB Quarterly, 91(Spring), 7-18.
Ong, R., (1994). Mechanisms for routinely tying cognitive models to interactive simulations. MSc thesis.
Ong, R., & Ritter, F. E. (1995). Mechanisms for routinely tying cognitive models to interactive simulations. In HCI International '95. Osaka, Japan: July 1995.
Baxter, G. D., & Ritter, F. E. (1997). Model-computer interaction: Implementing the action-perception loop for cognitive models. In: The 1st International Conference on Engineering Psychology and Cognitive Ergonomics, October 1996, Stratford-upon-Avon. 215-222. Ashgate.
Rassouli, J. (1995). Steps towards a process model of mouse-based interaction. MSc. Thesis, U. of Nottingham.
Baxter, G. D., & Ritter, F. E. (1996). A simple set of generic perceptual and motor capabilities for use by cognitive models in SLGMS (Tech. Report No. 36). ESRC Credit, Psychology, U. of Nottingham.
Perhaps the simplest way to create a model is to embed it in a tool. With Sarah Nichols, we have created an extension to an existing, free spreadsheet (called Dismal) for creating keystroke-level models. These can be useful models, but they are simple enough that 2nd year undergraduates can learn to apply them within a 5 week practical class. The spreadsheet, including the keystroke model extension are available on the web in my public FTP area. This work shows that simple enough models can be made available to help interface designers in their work. This tool has been used to create alias sets for two releases of a software package.
A group of people, including Randy Jones (at the U. of Michigan), Tony Kalus (at the U. of Portsmouth/U. of Michigan), Gordon Baxter and I, along with input from people like Richard Young, have been working on an improved interface for Soar. Last year we used an early version in a cognitive modeling class at Nottingham. An preliminary version was presented at a workshop. When this is released, it will be called the Tcl/Tk Soar Interface. This work has shown that it is possible to make model creation, understanding, and debugging easier.
Ritter, F. E., & Baxter, G. D. (1996). Able, III: Learning in a more visibly principled way (Tech. Report No. 40. 8 pages.). ESRC CREDIT, Dept. of Psychology, U. of Nottingham. In the 1st European Cognitive Modeling Workshop, November, 1996. Includes new Soar interface and two models.
A revised and extended version is available as Ritter, F. E., Jones, R. M., & Baxter, G. D. (1999). Reusable models and graphical interfaces: Realising the potential of a unified theory of cognition. In U. Schmid, J. Krems, & F. Wysotzki (Eds.), Mind modeling -- A cognitive science approach to reasoning, learning and discovery. 83-109. Lengerich: Pabst Scientific Publishing.
After a model has been created, the next step is to test how well its predictions match human behavior. This is often difficult and time consuming, so automatic and semi-automatic tools are desirable.
As part of my PhD thesis, I've started to codify the steps in testing the sequential predictions of process models. This was written up as a more accessible journal article, and is a slightly longer tech report version is available for those looking for more details.
Once you have a cognitive model in hand, you may wish to test it. The first step in this process is often to gather protocol data.
Dismal was created as a spreadsheet for manipulating psychology data, including protocol data. Work on dismal has also generated some general utilities for GNU-Emacs. A more complete explanation is available as well http://www.psychology.nottingham.ac.uk/staff/ritter/papers/dismal/dismal.html.
Ritter, F. E., Lochun, S., Bibby, P. A., & Marshall, S. (1994). Dismal: A free spreadsheet for sequential data analysis and HCI experimentation. In A. Trapp & N. Hammond (Eds.), Computers in Psychology '94, 62-63. York (UK): CTI Centre for Psychology, U. of York. Reprinted in Psychology Software News, Vol. 5 No. 2 (November 1994), Computers in Teaching Initiative Centre for Psychology, U. of York. pp. 57-58.
MacShapa is another program for editing protocol data. While it is not programmable like Dismal, it is a broader program, it can be hooked up to code video data more directly, and is is easier to use for most tasks. We recommend it in a review we did, and an initial analysis of some Air traffic control data was done using it.
CREDIT has set up a MacShapa lab for analyzing sequential data. Either for its own sake or as a precursor to building a process model. It includes the ability to transcribe directly from video tape to computer files, and enough memory to do this fairly easily. If you would like more details or to use it locally, please contact me.
Arnold, M., Ritter, F. E., Kuk, G. (1995). Prioritizing attention in air traffic control. Part of the MacShapa web pages.
An essential aspect of cognitive modeling is understanding the modeling language.
To this end, with Richard Young we have created instructional materials called "The Psychological Soar Tutorial", or PST. This tutorial has been presented several times with Richard Young to external audiences: EuroSoar 7,8,9,10 workshops, 1993-1996 (UK, Wales, Netherlands, Germany); AISB Spring symposium series, 1994, 1996, 1997; HCI '94; HCI International 1995 (Japan). An extended version of it was presented at the Autumn School on Cognitive Science '97 run by the Graduiertenkolleg (graduate college) in Intelligence in Humans and Machines at the University of Freiburg (Germany). A review of it has been published in AI and Simulation of Behavior Quarterly, and this review has been updated once.
A short report on it was presented at the AISB'99 Workshop on teaching cognitive science to undergraduates, and a paper is in their proceedings.
A report on this is in preparation (a summary was published in CTI Psychology Software News, and is likely to be on their web server), and I can add you to the future distribution list if you wish. We will report that the web is not a panacea for developing course ware. The document we created I hoped would replace lectures and the lecturers. Except for remote students who don't have a choice, this was not the case. The web-based document did provide help with students' revision, and it made the material seem more real to them, such that they took it more seriously. This work showed that the next serious step for making modeling easier is an interface for programming and understanding models, and then higher level modeling tools such as a higher level language for creating models.
It is used as a case study in the U. of Notitnhgam's new lecterure's course.
Ritter, F. E. (1997). WWW presentation of overheads & exercises CTI Psychology Software News, 7(2), p 46.
In addition to creating full process models, it will often be useful (and more practical) to create more abstract models. These should be easier to use and apply than process models, but will often be based on models that do the task of interest.
With George Kuk, we have looked at how to create summaries of problem sub-task transitions in the area of air traffic control.
A preliminary version of the analysis is available as well, and is directly based on Martin Arnold's third year project based on data he collected at DRA Malvern.
The rate at which performance on task improves as the task is repeated is fairly similar across tasks. The improvement generally follows what is called a power-law. Based on an idea in my master's thesis, we looked at the power-law of learning in detail by separating the data into different strategies, where we found that each strategy improves with a power-law rate. This is a step forward in understanding how learning occurs.
Cognitive modeling requires infrastructure. We've worked on setting up some infrastructure, locally and on a larger time and geographic scale.
Lehman, J., Newell, P., Altmann, E., Ritter, F., McGinnis, T. (1994). The Soar introduction video (11 min.). The Soar Group, School of Computer Science, Carnegie-Mellon University.
Organizer and editor of proceedings, EuroSoar 7 Workshop, U. of Nottingham, November 1993.
Support for this work has come from the DRA, the Enterprise in Higher Education, ONR/Europe, and from the UK Economic and Social Research Council through The Credit Centre.
School of Psychology .
University of Nottingham
Nottingham, NG7 2RD UK
+44 (115) 951-5292
+44 (115) 951-5324