TABLE OF CONTENTS 1. Course
Overview |
Please note, this is a live document. Changes announced in class and on the list server will be incorporated from time to time. Announcements in class and their mirror here are the definitive version.
An emerging technology, oddly enough, is the technology of simulating human behaviour. This and related technologies have been called expert systems and artificial intellegence. They are currently being used as opponents in computer games and have been started to be used to test interfaces as surrogate users. They are an interesting intersection between technology and human psychology. The National Resarch Council (Pew & Mavor, 1998, 2007) and research councils in the US, UK, and Australia expect this area to become increasingly important and they invest in it. There are a wide range of possible applications including video games, military simulations, knowledge-based computer-based tutors, and assistive agents [Norling on agents and why they need to be more human: ABC Radio [mp4] [mp3]].
To explore this new technology and its science base, this class has three components. The first, short, component examines the application of models of human performance firstly as scientific theories, but also their use in training environments, in operations research, in design, and as opponents in computer games. This section is based on reading and discussion.
The second component examines how to create such models. Students will study how to create such models using an existing cognitive and AI architecture, Soar. Students will be introduced to the ideas of using published descriptions of human behavior (i.e., journal articles and handbooks), as well as how data is gathered when published materials are not available. Tools for analysing the data to help create models will also be covered. This component results in a model created by the student or a team within a somewhat constrained task, dTank. A prototypical final project is to build a model of your own play in this game, but this project can be student-interest driven.
The final component examines how such models in general and the student's model in particular can be tested, validated, and thus can be believed and improved. If the dTank environment is used, a competition between models will be staged, both as models of intelligence (do they win?), and as models of humans (do they act like humans?). Thus questions related to the philosophy of science are also discussed.
This course requires an Instructor technology room on a Monday afternoon and a student technology room on a Wed afternoon. The student room will have Mac/PCs. You should also be able to install the software on your home machine, and many students will choose to do this.
The official pre-requisites are IST 210 and IST 220 (or such). You should also have an interest in modeling human behaviour, and then secondarily, some knowledge of human behaviour or some knowledge of the technology to model it (or both). Knowledge of human behaviour can come from IST 331 or a psychology course. Knowledge of the technology means production systems, Lisp, or JAVA, or expert systems or agent architectures. Students will be paired where possible to create pairs with complementary knowledge. If you have questions about prerequisites, please see me.
This course should change the way you think about human behaviour, providing a theory of cognition that can be extended to help predict human behaviour for use in applications such as video games and interface testing.
This course provides a balance between theory and practice, which are tightly intertwined in this area. Basic and more advanced readings will introduce the student to current thinking about facts, theories, and ways to model data. A small group project, drawing on the different backgrounds students bring to the program, will support integrating these various pieces of knowledge and applying them. The course includes working in small groups, speaking, and other non-traditional processes and outputs, but the processes and outputs used are those that researchers and developers use in this area.
At the conclusion of this course, students will be able to:
3.1 The IST 402 Web Site. This course has an active web page that contains the syllabus, assignments, links to useful sites, and other valuable material (such as how to correctly prepare assignments, citation templates, and other academic and recreational information). We will post late-breaking information and updates to the web page. This page can currently be found at uniform resource locator (URL) acs.ist.psu.edu/ist402, and later will be available through links from the IST home page via course listings. You should bookmark it.
3.2 The IST 402 Listserv. This course has a mandatory listserv that we will use to post course and class information, conduct on-line discussions, and share information. You are encouraged to use your PSU account, and not a hotmail or yahoo account that cannot receive attachments.
If you are in Section 2 (Ritter) your PSU account should be automatically subscribed. If you want to use a different account, you will need to subscribe to the class email list by sending email to Changkun.
Once you have subscribed, you can then send mail to the class at L-ist402-FA11-S1@lists.psu.edu
If you send mail to me about this course, please include "IST402" in the subject, as this will help a filter bring it to my attention more quickly.
Stuff on Soar (read, in order, until you know Soar) [passwords available from TA and Ritter]
The Psychological Soar Tutorial, available online
Ritter, F. E. (2003). Soar. In L. Nadel (Ed.), Encyclopedia of cognitive science. vol. 4, 60-65. London: Nature Publishing Group. [A006.pdf]
Laird's online tutorial notes, chapter 1 [from the Soar installation]
Lehman, J. F., Laird, J. E., & Rosenbloom, P. S. (1996). A gentle introduction to Soar, an architecture for human cognition. In S. Sternberg & D. Scarborough (Eds.), Invitation to cognitive science, vol. 4 Cambridge, MA: MIT Press.
Newell, A. (1992). Précis of "Unified theories of cognition". Behavioral and Brain Sciences, 15, 425-492. Responses Introduction to Newell
Stuff on the software (you will need all of these, and all are free)
Herbal [a Description of Ontologies, to help with understanding] Herbal tutorial
Those who like SoarceForge, can also find Soar there
You may find the Soar FAQ helpful.
dTank-Soar game, available locally
Stuff on psychology and writing [one copy of each per group]
(ABCS) The ABCS of HCI. Ritter, F. E, Baxter, G. D., & Churchill, E. 2011. Available from LuLu.com at cost, for about $11.00 plus shipping. approximately 400 pages.
Publication Manual of the APA (available at the PSU Bookstore) as a guide to referencing, citing, and the formatting of papers and manuscripts in general. Each pair should have access to a copy of this, or to Strunk and White's "The elements of style." Abebooks.com is another place to get this book.
3.4.1 The importance and applications of models (read two)
Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition and the LISP tutor. Cognitive Science, 13(4), 467-505.
Laird, J., & van Lent, M. (2001). Interactive Computer Games: Human-level AI's Killer Application, ai.eecs.umich.edu/people/laird/papers/AI-games.pdf.
Hudlicka, E., & McNeese, M. D. (2002). User affective and belief states: Assessment and user interface adaptation. Journal of User Modeling and User Adapted Interaction, 12, 1-47.
Pew, R. W., & Mavor, A. S. (Eds.). (2007). Human-system integration in the system development process: A new look. Washington, DC: National Academy Press. http://books.nap.edu/catalog.php?record_id=11893.
Pew, R. W., & Mavor, A. S. (Eds.). (1998). Modeling human and organizational behavior: Application to military simulations. Washington, DC: National Academy Press. nap.edu/catalog/6173.html (click on the cover of the book on the left to read it online.)
Ritter, F. E., Shadbolt, N. R., Elliman, D., Young, R., Gobet, F., & Baxter, G. D. (2003). Techniques for modeling human performance in synthetic environments: A supplementary review. Wright-Patterson Air Force Base, OH: Human Systems Information Analysis Center (HSIAC), formerly known as the Crew System Ergonomics Information Analysis Center (CSERIAC). [local cleaner copy]
** Or, paper of your choice, approx. 20 pages, or a whole article, whichever is shorter. **
3.4.2 Testing and validating models (read all)
Ritter, F. E., & Larkin, J. H. (1994). Using process models to summarize sequences of human actions. Human-Computer Interaction, 9(3), 345-383.
Ritter, F. E. (2004). Choosing and getting started with a cognitive architecture to test and use human-machine interfaces. MMI-Interaktiv-Journal's special issue on Modeling and Simulation in Human-Machine Systems. 7. 17-37. [in English, abstract in German] [local copy]
Grant, D. A. (1962). Testing the null hypothesis and the strategy and tactics of investigating theoretical models. Psychological Review, 69(1), 54-61.
3.4.3 Creating and building models (read all)
Ritter, F. E., Lehtinen, E., & Nerb, J. (2007). Putting things in order: Collecting and analysing data on learning. In F. E. Ritter, J. Nerb, T. O'Shea, & E. Lehtinen (Eds.), In order to Learn: How ordering effects in machine learning illuminates human learning and vice versa
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: The MIT Press. Their appendix on collecting protocols. Their 1980 paper as Precis of book [Read appendix, rest is optional]
Nerb, J., Ritter, F. E., & Langley, P. (2007). Rules of order: Process models of human learning. In F. E. Ritter, J. Nerb, T. O'Shea, & E. Lehtinen (Eds.), In order to learn: How ordering effects in machine learning illuminates human learning and vice versa Kidlington, UK: Pergamon.
Nuxoll, A., & Laird, J. (2003). Soar Dogma. (how to write better, safer, faster Soar code.)
The Soar Frequently Asked Questions list (acs.ist.psu.edu/soar-faq) and the ACT-R FAQ (acs.ist.psu.edu/actr-faq).
How to write an abstract by Mary-Claire Van Leunen
Paschler on testing (simple) models
Sanderson, P. M., McNeese, M. D., & Zaff, B. S. (1994). Handling complex real-world data with two cognitive engineering tools: COGENT and MacSHAPA. Behavior Research Methods, Instruments, & Computers, 26(2), 117-124. [email me for macshapa software for Mac]
You earn your grade but it will be assigned by me. The criteria for each assignment will be discussed in detail, as will the grading scheme. Each written assignment will be evaluated on how well it addresses the questions posed, the clarity of thinking, the organization and presentation of the material, the quality of writing, and its timeliness.
Your grade will be based on 100 possible points. You earn points with each assignment (see below). As a maximum scale (i.e., cutoffs may be lowered): A: 100-74, A-: 73-70, B+ 69- 67, B: 66- 64, B-: 63- 60, C+: 59- 57, C: 56- 50, D: 49- 40, F: 39- 0. (The cutoffs for each grade is the lower number, without rounding.)
Your learning will be assessed in
several ways. Please consult the schedule to see when papers/
assignments are due and exams scheduled. You will receive more
written instructions for each assignment well in advance of the due
date. Here is a brief summary of each:
Assignment |
Weight |
|
Due Date |
Labs |
30% |
You will do a variety of labs/homeworks. Each lab writeup is nominally 5-10 points, 66 points total including an extra 5 extra credit points and the initial 1 point project writeup. 60 points will be taken to be the maximum lab grade (i.e., you can miss 6 points and get a perfect score). This score may be modified/moderated/adjusted by self and team evaluations. |
Typically Wednesdays, as below |
(5 points added to lab grade) |
Once during the semester your group may find an additional resource that addresses or relies upon topics covered in class. In one page or less, you will comment on how that resource relates to the class. You will share both the article and your comments with the class. You must arrange this before the end of October. Paragraphs on the readings, up to 20 of them at 1 pt/reading, can also be used this way. |
Once, varies by group | |
Mid-Term Exam |
30% |
In class, 70 points below. |
November 2011 |
Attendance |
5% |
In class, score pro-rated with 3 allowed unexcused absences | |
Project Example templates: |
35% |
71 total points make this up, which you can keep the 1 as a bonus. |
December 2011 |
|
100% |
|
|
Date
In Class
Read/Prepare
Due
points
1
22aug
Intro - Course, emerging technologies, new uses for old important technologies, UTCs, Cognitive psych review
Hand in: OSs, courses
2 24aug End of intro Install/test software
Summary installation notesSoftware test
(in class)1 3
29aug
Applications of models
two readings from 3.4.1
1 para on each reading
2
4
31aug
Soar/ PST1
Further reading on Soar
Preface/ABCS1
1 para on each reading
1-2
5
7sep
Soar
ABCS3
1-2
6
12sep
PST2
pst exercisesABCS4
Laird1
readings
10
1
1-2
7
14sep
Soar / Herbal
ABCS5 [from now, all ABCS up to now due]
Eclipse
Exercise on Eclipse (in class)
1
8
19sep
PST3
ABCS6
10
1
9
21sep
Model development
ABCS12(TA)
1-2
10
26sep
Herbal /dTank
10
11
28sep
Testing models/why
More from 3.4.2
1
12
3oct
Herbal/dTank
10
13
5oct
Data for developing
Ericsson & Simon 93A
Ericsson & Simon 80 (optional)
ABCS4-111 page on project
1pp
14
10oct
Herbal/dTank
15
12oct
Creating models
readings above in 3.4.3
1
16
17oct
Herbal/dTank
ABCS12
1
17
19oct
(project design)
dTank Hwk1 (email it in)
10
18
24oct
Herbal/dTank
19
26oct
Creating models two readings from 3.4.3 and ABCS11
10
1
20
31oct
Review ABCS17.4
1
21
2nov
midterm in class
+Group feedback
60
22
7nov
Discussion of projects
1
23
9nov
Video games, hedonics, software engineering
24
14nov
Project - writing techniques
Agre/APA Manual/Strunk/advanced Word
dTank HWK3
(do another dTank task)10
25
16nov
Future directions
SOAR report and other futures reports
1
21-23nov
*Thanksgiving*
26
28nov
Project work
27
30nov
Reports discussion
ABCS-A4
presentations in class
5
28 5dec Reports discussion presentations in class 29 7dec Hand in Project 65 >209 pts, keep 200
Model the effect of your learning, starting with a naive model, and then a less naive model.
Model how you learn, starting with a naive model, and then with the learning component turned on, show that the model learns like you do (considerably harder).
Extend the dTank interface in some interesting way, and show how it is easier for you and for the model.
Set up two opponents for people to train against. Show how one is a better sparing partner, either for some of your friends, or for your model to learn from.
All projects above, if extended, could be turned into an honors thesis.
Graduate students taking this course for independent study credit (or if approved as a 597 course) will need to modify both the task (multiple interfaces or tasks) and the model (multiple strategies or individual differences).
The laboratory portion of IST 402 provides students with the chance to become familiar with using the concepts, software, and data about modeling people. It is absolutely essential for understanding the material and will be useful for passing the exam and doing the labs.
You have been put into small groups to do your labs because we believe this generally leads to better learning. That means that you must turn in one lab report per group, that conferring within your group is not a violation of academic policy or of ethics on the lab section of this course, and that conferring with other groups *is* a violation of academic policy and ethics if it results in reports that are noticeably similar without citation.
The best way is to work on the lab and then meet to discuss and proofread the report. The worst way is to have each member of the group do (and thus learn) one of the sections. This will result in a noticeably inferior product.
As we explore these topics, we will also practice skills in working together, analytical skills, and information problem-solving approaches.
We have arranged time for group meetings that may need a lab.
Academic Integrity: According to the Penn State Principles and University Code of Conduct:
Academic integrity is a basic guiding principle for all academic activity at Penn State University, allowing the pursuit of scholarly activity in an open, honest, and responsible manner. In according with the University's Code of Conduct, you must not engage in or tolerate academic dishonesty. This includes, but is not limited to cheating, plagiarism, fabrication of information or citations, facilitating acts of academic dishonesty by others, unauthorized possession of examinations, submitting work of another person, or work previously used without informing the instructor, or tampering with the academic work of other students.
Any violation of academic integrity will be investigated, and where warranted, punitive action will be taken. For every incident when a penalty of any kind is assessed, a report must be filed This form is used for both undergraduate and graduate courses. This report must be signed by both the instructor and the student, and then submitted to the Senior Associate Dean.
In this course, academic integrity needs two explanations. The first is that you are expected to contribute to your group. This means making time in your schedule for the group's meetings, being in touch via email, and preparing for those meetings. The second explanation is that your group's writeups are to be done only by your group. This means that while you can discuss the problems with others, and we agree that this is a good thing, the writing up needs to be solely by your group. If you have a question as to whether or not two writeups are too similar, then use the following standard:
- If a pair of writeups are sufficiently similar that you can tell by reading them that the two groups worked together, then they are too similar.
By all means, talk to your colleagues, get help if necessary, but prove to us that, in the end, you understand what you are doing, and you can and must express it in your own words. If you and your group don't understand the material well enough to write it up on your own, and you need to copy, then four things are lost: Your integrity, useful feedback to us on how you are doing, your ability to perform well on the exam, and ultimately, your knowledge.
It is Penn State's policy to not discriminate against qualified students with documented disabilities in its educational programs. If you have a disability-related need for modifications in your testing situation, your instructor should be notified during the first week of classes so that your needs can be accommodated. You will be asked to present documentation from the Office of Disability Services (located in 105 Boucke Building) that describes the nature of your disability and the recommended remedy. You may refer to the Nondiscrimination Policy in the Student Guide to University Policies and Rules 1999.
As each student is an individual with specific needs, academic accommodations are provided on an individual basis based on the student's documentation. A reasonable accommodation is a modification or adjustment to a course, program, service, job, activity, or facility that provides the qualified individual with a disability to have an equal opportunity. An equal opportunity provides the means to attain the same level of performance or to enjoy benefits that are available to students without disabilities. For more information about services for individuals with learning disabilities, please contact the Office for Disability Services at (814) 863-1807.
Americans with Disabilities Act: IST welcomes persons with disabilities to all of its classes, programs, and events. If you need accommodations, or have questions about access to buildings where ISTactivities are held, please contact us in advance of your participation or visit. If you need assistance during a class, program, or event, please contact the member of our staff or faculty in charge.
An Invitation to Students with Learning Disabilities: It is Penn State's policy to not discriminate against qualified students with documented disabilities in its educational programs. If you have a disability-related need for modifications in your testing or learning situation, your instructor should be notified during the first week of classes so that your needs can be accommodated. You will be asked to present documentation from the Office of Disability Services (located in 116 Boucke Building, 863-1807) that describes the nature of your disability and the recommended remedy. You may refer to the Nondiscrimination Policy in the Student Guide to University Policies and Rules.
Affirmative Action & Sexual Harassment: The Pennsylvania State University is committed to a policy that all persons shall have equal access to programs, facilities, admission, and employment without regard to personal characteristics not related to ability, performance, or qualifications as determined by University policy or by Commonwealth or Federal authorities. Penn State does not discriminate against any person because of age, ancestry, color, disability or handicap, national origin, race, religious creed, gender, sexual orientation, or veteran status. Direct all inquiries to the Affirmative Action Office, 211 Willard Building.