Tutorial Program at Cognitive Science 2003, 30 July 2003

Page Contents

Introduction

Topics

Conference home

Arrangements

Eye Tracking

Original call

Organizing online seminars

Inquiry: Teaching Cog Sci

Latent Semantic Analysis

Introduction: The Tutorials program at Cognitive Science 2003 provides participants to gain new insights, knowledge, and skills from a broad range of areas in the field of cognitive science. Tutorial topics will be presented in a taught format and range from practical guidelines to academic issues and theory. This is the fourth year that tutorials in this format will be offered.

Tutorials will present tutorial material, that is, provide results that are established and to do so in an interactive format. They tend to involve an introduction to technical skills or methods (i.e., eye-tracking, statistical modelling, methods of supporting online seminars, and tools for teaching cognitive science). They will include substantial review of material. The level of presentation assumes that the attendees have at least a first degree in a cognate area. Several of the tutorials are related to this year's theme, The Social, Cultural and Contextual Elements of Cognition.

There is a student rate, and such students should bring their ID to show at registration.

Attendance at the tutorials does not require conference registration, but does not provide conference entrance.

Arrangements: There are four tutorials this year on Wednesday 30 July (rooms to be announced on the day). They cost $60 (about 40 pounds or 55 Euros) for a half-day tutorial and $40 for students. Payment can be made using the registration site on the conference page, or can be paid for on the day (if space is available). The program includes handouts, and a tea and a coffee break (including biscuits). Lunch is available nearby in downtown Boston. There will be a meeting of the tutorial committee and tutors after the tutorials, location to be announced on the day.

Registration for tutorial attendees will be from 8.30 am on 30 July in the lobby of the Park Plaza hotel. It should take less than 5 minutes to get from the tutorial desk to the tutorial rooms, but please allow yourself time to get to the room.

9.15 - 12.30 is the morning session (including 15 min. coffee break), and 2-515 pm is the afternoon session (including 15 min. tea break).

 Topics

Using Eye Movements to Study Cognitive Processes
Rayner, Half-day (1400-1715)
in the Park Plaza, room to be announced

How to Plan and Run Online Seminars
Neal and Anastas, Half-day (0915-1230)
in the Park Plaza, room to be announced

Inquiry, a Tool for Teaching Cognitive Science
Bechtel et al., Half-day (0915-1230)
in the Park Plaza, room to be announced

Latent Semantic Analysis: Theory, Use and Applications
Dennis et al., Half-day (1400-1715)
in the Park Plaza, room to be announced

========================================================================

Using Eye Movements to Study Cognitive Processes

Half-day tutorial (1400-1715)
in the Park Plaza, room to be announced

Keith Rayner
University of Massachusetts, Amherst
rayner@psych.umass.edu

For the past twenty-five years, many researchers have been using eye movement data to investigate various issues related to cognitive processing. Recording of eye movements provides a relatively unobtrusive moment-to-moment indicator of processing in a number of tasks of interest to cognitive scientists. In this presentation, an overview will be provided of different types of eye-tracking systems and the trade-offs in terms of temporal and spatial resolution that are inherent with these systems. The issue of how to best analyze the resulting eye movement data will be discussed and some representative examples of eye-tracking applications will be provided. Finally, some specific examples of how eye movement data can be used to answer certain research questions will be given.

Keith Rayner received his PhD in 1974 from Cornell University. He is a Fellow of APA and APS, and a member of the Society for Experimental Psychology and of the Experimental Psychology Society (UK). He is the author of over 200 publications and is best known for his work on eye movements in reading and language processing, but has also worked on scene perception and visual search. His web site is www-unix.oit.umass.edu/~rayner/

========================================================================

How to Plan and Run Online Seminars

Half-day tutorial (0915-1230)
in the Park Plaza, room to be announced

Lisa Neal
EDS Learning Solutions
Editor-in-Chief, eLearn Magazine, www.eLearnMag.org
lisa.neal@eds.com

Donna Anastas
Aptima
anastasi@aptima.com

Online seminars are an increasingly common way to reach a broader audience. However, deciding to offer a seminar online requires a number of steps, especially to achieve a flawless presentation. The critical pieces are:

  • Selecting the required features (audio, chat, slides, polling, archival, etc.) and technology for delivering and moderating the seminar
  • Announcing and promoting the seminar (including registration and instruction web pages)
  • Planning pre-seminar activities (forums, advance questions, pre-reading)
  • Preparing presentation materials
  • Orchestrating a session (interactivity, Q&A, chat, polling, back-up plans/redundancy, etc.)
  • Coaching a first-time online presenter (how presenting online is different from face-to-face)
  • Running and moderating the actual seminar
  • Creating an archive
  • Developing, sending, and using post-seminar evaluations
  • Planning post-seminar activities (archive of session and chat, further discussion or Q&A)

While not all of these are needed for all types of seminars, many of them are essential for a successful seminar. The tutorial will cover each topic, some in more depth than others, so participants walk away with the knowledge to run their own online seminars. The tutorial will also look at the differences in structuring and presenting an online seminar based on seminar purpose and group size, cohesion, and formality. Demos will be included (using archived seminars) and, if possible, participants will have a chance to experience presenting and participating online. If email addresses are available in advance, participants will be given some optional opportunities to listen to archived seminars and to prepare specific questions.

Lisa Neal is Editor-in-Chief of ACM eLearn Magazine, www.eLearnmag.org, a Managing Consultant with Electronic Data Systems, and an Adjunct Professor at Tufts Medical School. She received a Ph.D. in Computer Science from Harvard University. Dr. Neal consults on e-learning projects with corporate, academic, and government clients. At Tufts Medical School, she teaches a course on Designing and Delivering Distance Education with a focus on e-learning in medicine. In 2001-2 Dr. Neal delivered four keynote addresses at e-learning conferences in Italy, Poland, Mexico, and the U.S. and is a frequent presenter and tutorial instructor at conferences. One of her upcoming tutorials is announced at www.hcii2003.gr/program/tutorials/t24.asp.

Donna Anastasi is a usability designer at Aptima, a human-centered engineering company. Ms. Anastasi is currently project lead on a cognitive task analysis educational resource www.ctaresource.com and was a key contributor in established a web conferencing facility at Aptima that provides delivery technology and a full range of start-to-end services for successful virtual seminars. She manages development of a web-based training system to teach reading comprehension skills and a web based advisor for diagnosis and remedy of team performance issues. She received a B.A. Psychology and Mathematics from Smith College and an M.S. through ISyE in Human-Machine Systems from the Georgia Institute of Technology.

==========================================================

Inquiry, a Tool for Teaching Cognitive Science

Half-day tutorial (0915-1230)
in the Park Plaza, room to be announced

William Bechtel
Department of Philosophy
University of California, San Diego
bill@mechanism.ucsd.edu

Adele Abrahamsen, Washington University and UCSD

Carl Craver, Washington University

Peter Bradley, Washington University

This tutorial will introduce a web-based project (inquiry.wustl.edu) that is developing a modular, interdisciplinary approach to teaching research methodologies in the cognitive sciences. A common challenge in preparing undergraduate majors and minors in cognitive science is to assure that students graduate with at least a beginning-level mastery of how discoveries get made and defended in the varied disciplines of cognitive science.

A new course designed to meet these objectives, Inquiry in the Cognitive Sciences, is now being offered at Washington University. Funding for developing and disseminating course materials is provided by a three-year grant from Fund for the Improvement of Post-Secondary Education of the U.S. Department of Education. The content for the course is available on the web in modules that can be used by instructors at other institutions. So as to allow instructors elsewhere to use the materials to meet the needs at their local institution, the materials are organized according to a hierarchical modular design which will allow individual models to be used semi-independently.

This tutorial will present the goals for this project, introduce the materials available on the website, including tools for specializing the materials for individual courses, and solicit critical feedback. At the time of the conference the project will be half-way through its funding period, and thus in a position to incorporate suggestions and priorities from members of the broader cognitive science community into the final product.

William Bechtel is a philosopher of cognitive science whose interests include the use of mechanistic models in scientific explanation and strategies by which scientists evaluate new instruments and research methods. His publications include Discovering Complexity (1993, co-authored with Robert Richardson) and Connectionism and the Mind (1991 and 2001, co-authored with Adele Abrahamsen).

Adele Abrahamsen is a cognitive psychologist whose research has targeted language and cognition in both adults and children; she has also published theoretical papers on the orchestration of different types of inquiry in cognitive science. In addition to being coauthor of Connectionism and the Mind (1991 and 2001), she published Child Language: An Interdisciplinary Guide to Theory and Research (1977).

Carl Craver is a historian and philosopher of neuroscience with an active research interest in discovery, theory construction, and research methodology. He has published several papers on mechanistic models and mechanistic explanation.

Peter Bradley is a philosopher of mind interested in color and color perception - specifically, the philosophic problems posed by the existence of colors that are perceptible by non-human creatures like birds, but imperceptible for humans. In addition, he has an extensive background as a UNIX system administrator and bears major responsibility for the technical side of the course in addition to developing content.

========================================================================

Latent Semantic Analysis: Theory, Use and Applications

Half-day tutorial (1400-1715)
in the Park Plaza, room to be announced

Simon Dennis, Tom Landauer, Walter Kintsch, and Jose Quesada
Institute of Cognitive Science
University of Colorado
dennissj@psych.colorado.edu

Latent Semantic Analysis (LSA) is a theory and method for extracting and representing the contextual-usage meaning of words by statistical computations applied to a large corpus of text (Landauer & Dumais, 1997). The underlying idea is that the aggregate of all the word contexts in which a given word does and does not appear provides a set of mutual constraints that largely determines the similarity of meaning of words and sets of words to each other. These constraints can be solved using linear algebra methods, in particular, the Singular Value Decomposition.

The adequacy of LSA's reflection of human knowledge has been established in a variety of ways. For example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word÷word and passage÷word lexical priming data; and it accurately estimates passage coherence, learnability of passages by individual students, and the quality and quantity of knowledge contained in an essay. In addition, LSA has found application in a number of areas including selecting educational materials for individual students, guiding online discussion groups, diagnosing mental disorders from prose, matching jobs with candidates and facilitating automated tutors.

LSA has become both a widely debated theoretical model and a widely used cognitive tool. A search for Latent Semantic Analysis in Science Citation Index reveals over 200 published journal articles most of which appeared in the last three years. In last year's Cognitive Science Conference eleven articles referred to LSA and the majority of these employed LSA as their base technology. In addition, the LSA website that provides access to the computational tools received 3.5 million hits since April 2001.

However, while there has been substantial interest in LSA there remains a great deal of misunderstanding about the technical underpinnings of the technique and how it can be appropriately used. We have received large volumes of email that either asks for clarification or demonstrates misuse of the method. Our objective in proposing this tutorial is to address some of this confusion.

Audience: The tutorial will be aimed at the graduate school level with no special requirements for attendance, although a basic level of mathematical competence would be helpful.

Schedule: The schedule will be divided into three parts. The first section on LSA theory would be given by Walter Kintsch, Tom Landauer and Simon Dennis, the second section would be an interactive computer laboratory facilitated by Simon Dennis and Jose Quesada and the final section would overview some of the applications of LSA and would be given by Tom Landauer and Walter Kintsch.

Session 1: Mathematical Foundations and Cognitive Implications (1hr &endash; Walter Kintsch, Tom Landauer and Simon Dennis)

In this session, we will cover the basic mathematical foundations of LSA as well as overview LSA as a cognitive model. Topics to be covered will include: The representation of meaning, the method of solution, limitations of LSA and possible extensions, the cognitive foundations of LSA, contextualizing LSA (the Predication model).

Session 2: Using the LSA website to conduct research (1 hr &endash; Simon Dennis and Jose Quesada)

The LSA group maintains a website that allows researchers to use the technique in their work. The site provides interfaces that allow the extraction of nearest neighbor words and the comparison of multiple words or passages against each other. Some thirty different corpora are available for people to use as well as a number of options in terms of weighting functions and numbers of dimensions over which comparisons will be made. In this session, we will acquaint attendees with the use of the site and ensure that they fully appreciate the significance of parameter choices they may make. Upon completion attendees will be able to use the site to conduct their own LSA-related research.

Session 3: LSA Applications (1 hr &endash; Tom Landauer and Walter Kintsch)

LSA has been used in a variety of experimental applications. In this section of the tutorial, we will overview some of these applications, including:

(1) LSA has been and is currently being used to score student essays. Typically, students are asked to write short essays varying from around 50 to 500 words to cover an assigned topic or to answer a posed question. The experiments to date have involved a wide variety of topics, including heart anatomy and physiology, neuroscience and neuropsychology, experimental psychology of learning and child development, American history, business, sociology, information technology, and others. Overall the LSA-based measure correlated as highly with a single human's scores as one human's scores correlated with another.

(2) The technique has been used to improve automatic information retrieval, where it produces 15-30% gains in standard accuracy measures over otherwise identical methods by allowing users' queries to match documents with the desired conceptual meaning but expressed in different words (Dumais, 1994, Berry, Dumais & O_Brien, 1995).

(3) By training on overlapping sets of documents in multiple languages, LSA has been able to provide good retrieval when queries and documents are in different languages.

(4) LSA-based measures of the similarity of student essays on a topic to instructional texts can predict how much an individual student will learn from a particular text (Wolfe et al., 1998, Rehder et al., 1998). The principle involved is a version of Vygotsky's zone of proximal development that we have dubbed "the Goldilocks Principle". Experiments estimated that using LSA to choose the optimal text for each student rather than assigning all students the overall best text (which LSA also picked correctly) increased the average amount learned by over one standard deviation (Rehder, et al. 2001).

(5) LSA-based measures of conceptual similarity between successive sentences accurately predicted differences in comprehensibility of a set of experimentally manipulated texts (Foltz, Kintsch, &Landauer, 1998). The LSA method predicted empirical comprehension tests results with college students as well as the hand coding of propositional overlap used in creating the differentially comprehensible paragraphs. Prediction by literal word overlap between sentences had a near zero correlation.

(6) LSA has been used to evaluate and give diagnostic advice to sixth-grade students as they write and revise summaries of text they have read (E. Kintsch et al., 2000). Use of the system resulted in one standard deviation better summaries as measured by blind ratings, and the effect generalized to writing summaries a week later without the system_s help (Steinhart, 2000).

(7) LSA has been used to assess psychiatric status--schizophrenic or depressed patients compared to normal controls--by representing the semantic content of answers to psychiatric interview questions (Elvevåg, Fisher, Weinberger, Goldberg & Foltz, unpublished). Accuracy was as good as those that have been reported for clinical diagnostic reliabilities of mental health professionals (e.g., Regier, et al.,1998).

Simon Dennis is a computer scientist and psychologist and currently holds a research professorship at the University of Colorado, Boulder. His prior work has focused on empirical and computational modeling approaches to episodic memory. More recently he has been investigating statistical models of sentence comprehension.

Tom Landauer is Professor of Psychology and Fellow of the Institute of Cognitive Science at the University of Colorado, Boulder. He is also president of Knowledge Analysis Technologies, a company that supplies web-based applications of LSA in education, personnel, and tools for information workers. As Director of Cognitive Science Research at Bellcore in the 1990s he was closely involved in the original research and development of LSA, and is a named inventor on several patents for its application. His research over the last decade has focused on interlocking improvements in LSA and its use as cognitive theory, cognitive science research tool, and practical information technology.

Walter Kintsch is Professor of Psychology and Director of the Institute of Cognitive Science at the University of Colorado in Boulder. His research focus has been on the study of how people understand language, using both experimental methods and computational modeling techniques. In cooperation with the Dutch linguist Teun van Dijk, he formulated the first psychological process theory of discourse comprehension in 1978. In 1988, this work was reformulated as a constraint-satisfaction process. His latest book "Comprehension" appeared in 1998 and argues that many cognitive processes can be usefully conceptualized as comprehension processes. Kintsch is a member of the National Academy of Education and received the Distinguished Scientific Contribution Award of the American Psychological Association in 1992 and an honorary doctorate from the Humboldt University in Berlin in 2001.

Jose Quesada recently received his PhD in Cognitive Science from the University of Grenada. His thesis work focused on the extending the use of LSA principles to problem solving domains including simulated fire fighting and process control as well as aircraft landing efficiency.

========================================================================

Co-Chairs

Frank E. Ritter (Penn State)
Yvette Tenney (Bolt, Beranek, and Newman)

Local arrangements Chair

Yvette Tenney (BBN Labs)

Committee members

Elizabeth Churchill (Fuji/Xerox)
Randolph M. Jones (Colby College and Soar Technology)
Todd Johnson (UT/Houston)
Gary Jones (Derby)
Judith Kroll (Penn State)
Josef Nerb (Frieburg)
Padraic Monaghan (Warwick)
Chris Kello (George Mason)
Richard Young (Hertfordshire)

Conference Overview | Local Arrangements | Registration | Program | Sponsors