25 January 2013
Original Diag-Soar model, Frank Ritter (frank.ritter at psu.edu) and Peter Bibby (peter.bibby at nottingham.ac.uk) (Ritter & Bibby, 2008). the task is based on a task developed by Peter for his thesis.
Diag is a Soar model that shows how learning and transfer arise for a simple fault-finding task. The model shows high correlation with performance and learning across a range of sets of 20 faults for 8 out of 10 subjects.
Maik Friedrich (2008; Friedrich & Ritter, 2008) gathered additional data on a wider selection of strategies. He implemented these strategies in models using Herbal [archive online], and shows nearly equally good fits, and how some of these strategies differ. He also has put up the whole stimuli set.
The accompanying model and traces of it exist in acs.ist.psu.edu/projects/diag, as well as a version of Soar 6.2.5 to run it in as .zip and binary files. If MacSoar will not run on your Mac, email me and I'll get you a CD with MacSoar on it. It runs under Classic mode from MacOS 7 (I think) through to classic under MacOS 10.4.10. There are also files containing the data and the model, and traces of the model. The model might run on a PC version of Soar 6, but we have not tried this.
[Dismal (Ritter & Wood, 2005 ; GNU distribution) is a spreadsheet created for working with sequential data. It works as an extension to the GNU Emacs editor. It was used to manipulate the protocol data.]
diag16t.soar6 is the model. Start up MacSoar 6.2.5 and type "load diag16t.soar6" to load it. This action requires having Soar6 in the same directory, but if you download all the files into one directory, it should work directly. You can also see the traces (noted below) for further examples on how to load it. You will, however, have to load a fault file to have it find a fault. After loading the model and fault file, type "run <CR>" to get it to run. (ps. <CR> means a carriage return or enter).
faultstate-*.soar6 loads a fault. For example, faultstate-eb1.soar6 loads the EB1 fault.
faultstate-lbma.soar6 (LB fault with path through MA)
faultstate-lbsa1.soar6 (LB fault with path through SA1, this is the default)
faultstate-lbsa2.soar6 (LB fault with path through SA2)
eb1-trace-bottomup.txt displays the trace for the model solving the eb1 fault with bottomup (human-like learning) set in Soar. This is the primary model.
eb1-trace-all.txt displays the trace for the model solving the eb1 fault with all-levels learning (AI-like learning) set in Soar.
all-chunks5-04.txt is a listing of several model runs that lead to all the learning that can happen with bottomup learning, and a listing of the chunks (learned rules) that result.
trace-config4.s6 is a file to modify the trace produced by Soar. It was used in the protocol alignment runs.
trace1.txt is a bonus trace. It shows the time to solve each fault, with learning on, without prior learning.
op-augmentation-addition2.soar6 and op-augmentation-result.soar6 are the operator augmentation chunks learned that augment operators and that put the results on operators, respectively. op-augmentation-addition2.soar6 are the operator creation chunks. state-augmentation-chunks2c.soar6 are the declarative type learning.
The diag-data directory holds the human data as a spreadsheet (tasktimes), the protocol data, and the traces of the model's performance with various types of learning added. The traces might not be clear to a non-Soar programmer.
task-times aug07.xl holds the times for the 10 subjects; how long they took on each problem in their series of 20 problems, what problems they saw, and the model's times for that series of problems.
The folder protocol-subject contains printouts (.dp files) from the Dismal spreadsheets for the one protocol subject that were used to align the model and data. There are five files corresponding to the five protoocol episodes. Note, that episode 4 was not aligned because the subject had an incorrect answer.
The subfolder stimuli holds template spreadsheets to hold your data, and has the stimuli as pdf files for the series of problems for subjects 1, 2, and 8. (you can create other series using Acrobat Professional or your own tools.) There is also an instruction file (intro2.doc) for how to set up and run this experiment. The set of materials mention a class as they were also developed for use in a class as a lab.
Ritter, F. E., & Wood, A. B. (2005). Dismal: A spreadsheet for sequential data analysis and HCI experimentation. Behavior Research Methods, Instruments, and Computers. 37(1), 71-81.
Ritter, F. E., & Bibby, P. (2001). Modeling how and when learning happens in a simple fault-finding task. In Proceedings of ICCM - 2001 - Fourth International Conference on Cognitive Modeling. 187-192. Mahwah, NJ: Erlbaum.