CN 530: Neural and Computational Models of Vision
Course Syllabus, Fall 2007
Professor Ennio Mingolla Teaching Fellow: Jasmin Leveille
Office: 677 Beacon Street, Room 210 Office: 677 Beacon Street, Room 106
Office hours: Tuesdays, 10:00--noon, and Office Hours: Mondays, 9:00 – 11:00 AM, and
by appointment by appointment
Tel: 617-353-9485; email:ennio-at-cns.bu.edu Tel: 617-353-6426; email: jasminl-at-cns.bu.edu
Overview: This course explores the psychological, biological, mathematical and computational foundations of visual perception. Lectures and readings combine with simulation and essay assignments to provide an intensive and self-contained examination of core issues in early and middle visual processing. Mathematically specified neural and computational models elucidate the structure and dynamics of the mammalian visual system. The course elucidates the psychophysics and physiology of mammalian vision, both as a means of better understanding our own human intelligence, and as a foundation for tomorrow’s machine vision architectures and algorithms. While some of the models developed in recent years at Boston University’s Center for Adaptive Systems (CAS) and Department of Cognitive and Neural Systems (CNS) are covered in depth, selected models by a variety of researchers are compared and contrasted.
Meeting times: Lectures will be given on Tuesdays, beginning on September 4 and ending on December 11, from 4:00-7:00 PM in Room B03 of 677 Beacon Street. An additional hour-long discussion period at a different time will be arranged.
SUMMARY OF WEEKLY TOPICS
Week 1 Sep 4 Fundamental problems of vision
Week 2 Sep 11 Shunting competitive networks and representation in early vision
Week 3 Sep 18 Early visual pathways
Week 4 Sep 25 Contrast sensitivity and spatial scales
Week 5 Oct 2 Brightness and lightness perception
Oct 9 NO CLASS – University Monday schedule
Week 6 Oct 16 Parallel visual pathways
Week 7 Oct 23 Boundary detection, completion, and sharpening
Week 8 Oct 30 The phenomena of motion perception
Week 9 Nov 6 Models of motion perception
Week 10 Nov 13 An in-class examination, covering topics in the readings and lectures
from the first 9 weeks, will be given during this class period.
Week 11 Nov 20 Approaches to textural segmentation and grouping
Week 12 Nov 27 Binocular vision
Week 13 Dec 4 Visual attention
Week 14 Dec 11 The great beyond
COURSE REQUIREMENTS AND GRADES: All students must complete five simulation assignments, an in-class midterm examination, and a written final course report. Students also participate in weekly discussion meetings and are also required to turn in weekly updates of a personal journal, as described below. Course grades will be based on a conventional 100 point scale, with A = 93 or better, A-minus = 90-92, etc. The weighting of assignments and exams on the final grade is:
40% Five simulation assignments; each counts for 8% of the total credit for the course
30% In-class midterm examination (Week 10)
10% Final report
10% Discussion meeting participation
10% Professional growth, as documented in a personal journal
ASSIGNMENT DUE DATES: Printed hardcopy is the preferred means of submission for all assignments. Simulation assignments and the final report should be turned in at my office; assignments may be placed under my door if I am not in. If you are unavoidably physically distant from CNS on a due date, you may submit assignments electronically in Adobe Acrobat “PDF” format only, and only after receiving prior permission from me. Please do not even THINK of asking me if it is “okay” to submit an assignment late; it is not okay, not even by a few minutes, and your assignment grade will suffer. Please assume that printers will not work in the hours just before an assignment is due, that the subway will run late, and so forth. Then, get your assignments turned in on time anyway. Simulation assignments and the final report are due by 3:00 PM on the following dates:
Simulation Assignment 1 Wednesday, Sep 19
Simulation Assignment 2 Wednesday, Oct 3
Simulation Assignment 3 Wednesday, Oct 17
Simulation Assignment 4 Wednesday, Oct 31
Simulation Assignment 5 Wednesday, Nov 21
Final report Monday, Dec. 17
Personal journal units are turned in weekly at the start of lectures. The first journal unit is due on Week 2, Sep 11. No journal unit is turned in on Oct 9 (University Monday Schedule) or Nov 13 (midterm exam). The last journal unit is due on Dec. 11, making a total of 11 units for the semester. Journal units may not be turned in late, absent a documented medical excuse or life crisis.
JOURNAL UNIT FORMATS: Each journal unit consists of an annotated reading list and a single paragraph summarizing your most important impressions of all the readings that you have completed that week. (There will be several readings annoted every week, but only one summarizing paragraph.) Your list should include articles or chapters from the required or supplementary sections of the course syllabus that you have actually read in the preceding days for your coursework. Over the course of the semester, your annotated list should include all of the required readings for the course, and at least a few supplementary readings. By “annotated” is meant that, after each item, you add from one to four consise, descriptive sentences about that reading. Here is an example of annotation from the Week 3 entry of this document:
“Levine, D. and Grossberg, S. (1976). Visual illusions in neural networks: Line neutralization, tilt after-effect, and angle expansion. Journal of Theoretical Biology, 61, 477-504. Levine was Grossberg’s first Ph.D. student; this reading is also relevant to Simulation Assignment 5.”
Note that the two sentences after the page numbers are not part of the citation, but are the annotation.
You must choose a standard citation style and stick to it throughout the semester. For example, see http://library.osu.edu/sites/guides/apagd.html, though you are free to choose a different style.
Each journal unit can be up to three single-sided pages long – no more – using 1.5 line spacing between lines and a legible font of at least 11pt size. In the upper right hand part of the first page of each journal unit, write your name, the date that you turn the unit in, and “Unit N,” where N is an integer from 1 to 11, corresponding to the number of that unit – which will not, in general, correspond to the number of that week in the course.
Journal units must be turned in at the start of class; I will take a dim view of any student arriving late “because” that student needed extra minutes to finish a journal assignment. Electronic submission of journal units is not an option, except in extreme circumstances and with prior permission from me. Please note that the effect on your grade of turning in less than 11 journal units is almost certain to be noticeable: With each unit accounting for nearly 1% of your total course grade, you will slip quickly from (say) an A to an A-minus to a B-plus if you fail to turn in all 11. At the end of the semester, I will review all of your journal entries for evidence of professional growth, and possibly adjust your final grade point credits for the journal units by one or two points (plus or minus, on a 100 point scale) based on this evidence.
Journals will be graded on a “check” system, with each unit that is turned in on time earning a default of “full” credit. I will not in general return journal units to you throughout the semester, although I will be happy to discuss your journal during office hours or by appointment. I will contact individuals directly if performance on journal units deviates significantly from expected quality. I will often share with the class ideas raised by individuals in their journals, and my reaction to those points. The student’s identity will not be disclosed in these cases.
SIMULATION ASSIGNMENT SUBMISSION, CONTENTS, AND FORMAT: The following guidelines apply.
1) Cover sheet and anonymous grading: Simulation assignments are to be graded anonymously. Turn in all simulation assignments on 8 1/2” x 11” paper, and include ONLY your name, the course number, the date of submission, and the words “CN530 Simulation Assignment N” (N = assignment number) on the upper right corner of the first page. Do not include any other information on the first page. If you are printing on both sides of the paper, please print the first page separately so that there is no information on the back side. These assignments are to be evaluated based on their content only, so do not include information about your identity on any pages other than the first “cover sheet.”
2) Length: Simulation reports are expected to be brief. “Brief” means up to 2500 words for total report text; simulation assignments will have additional pages for graphs and diagrams. Software such as MSWord or LaTeX typically generates approximately 250 to 350 words per page, depending on settings for margins and line spacing. You should use 1.5 line spacing. Assignments whose fonts are smaller than 12 point, or that are single spaced, or that do not have 1” margins on sides and bottoms, or that are not paginated will be returned to be properly reformatted before they are graded. There will be a grading penalty for such deviations from basic formatting requirements.
3) Required sections and headers: All simulation assignments must include an abstract, a short introduction, a short concluding section, and a “References” section with a properly formated listing of published resources used for that assignment, each designated by an appropriate header. Abstracts should be approximately 200-300 words and should be specific enough in their wording that a person reading only the abstract should come away with a reasonably accurate idea of the content of your report. The simulation and final reports must begin with a short title that is descriptive of the content of the report. Possible titles do not include “simulation assignment 2” but do include concise phrases like “simulations of brightness filling-in by boundary-gated diffusion.” Your introductions should be sufficiently self-contained to make sense to someone besides me or the course teaching assistant. That is, you should not use jargon or assume familiarity with terms not expected to be in general use in the field of vision research. For simulation assignments you must explicitly label which part of which question (e.g. 3a or 1b) a given section of your report addresses.
4) Legibility: All assignments must be produced on a word-processor. Clear handwritten annotation is acceptable in small amounts, including correction of typos, insertion of notation in mathematical equations, graphs, or figures. Assignments containing extensive handwritten passages will be returned ungraded. You are expected to employ proper English spelling and grammar, and to adhere to reasonable stylistic conventions (such as the use of margins, references, headings and so forth.) I am aware that for many of you English is not a first language, and I do not expect you to become accomplished authors overnight. I am referring to basic requirements that can and should be met by all: Make sure that your sentences contain verbs and end with a period. If you use a pronoun in a sentence, make sure its antecedent is unambiguous. Do not employ slang. Check your writing for clarity; do not expect to be given “the benefit of the doubt” if your words are ambiguous or vague.
5) Graphical plots: Describe simulation results in words, with the help of graphical plots whenever possible. Include scales on all axes, and explicitly label the quantities being plotted on all axes. Each plot on your page should be treated as a “figure” in a publication. That means there should be a label and a caption (e.g. “Figure 7: Results of center-surround filtering of step functions,” followed by some descriptive text. It also means that the body of your report should refer to specific figures by their figure numbers. Your plots can be generated by computer or by hand, or in combination. (For example, you may be able to plot locations of points by computer, but prefer to label the scale of axes by hand.) Be judicious in choosing which outputs to display overall. Choose figures that contribute materially to the reader’s understanding (by showing crucial “before and after” or “with-crucial-parameter-value-equals-this-or-that” juxtapositions), rather than showing dozens of simulations. A typical mistake made by novice simulators is to display the results of many computer runs that vary only by a single parameter on several pages, making it nearly impossible for the reader to discern the overall impact of parameter variation on simulation results. Often, some manual “cut and paste,” is necessary to get a coherent graphical presentation. Note that it is never sufficient to report results only by showing plots; some accompanying verbal description -- well keyed to the plots -- is necessary to receive full credit. Irrespective of the contents of the data plots, a “sprawling” presentation where the relevant variation occurs across pages or with insufficient labeling (of axes and parameter variations) will be awarded reduced credit. Listing of computer code is not desired.
Simulation assignment software: While it is assumed that every student in this course is capable of writing simple applications programs for coding assigned simulations, this is not a programming course, and programming will not be taught in the class as such. Note, however, that the CNS Department typically offers some non-credit tutorials (e.g., in MatLab), and that the course’s TF can help with trouble-shooting. The assignments are not meant to burden students with days of software development, and they have been constructed in such a way as to minimize program development time. You many use any commercially available software that you feel enhances your productivity. You may legitimately ask questions of me or of the course TF while doing the assignments. You may discuss simulation development with fellow students, up to the point of exchanging programming “tips” or information about available resources (for graphics, word processing, etc.), but you are not to work in groups for “division of labor.” If you anticipate difficulties in performing simulations (programming, graphical plotting, machine access), see the course teaching fellow immediately.
FINAL REPORT: The content of every student’s final report will be negotiated individually, but the basic considerations of formatting described in the preceding sections about simulation assignments continue to apply.
DISCUSSION MEETINGS: Credit for participation in discussion meetings will be based on the student’s understanding of core topics from readings and lectures, as expressed in comments initiated by students or in response to questions from the professor.
PLAGIARISM: What you write is to be the original expression of your own learning. If you must employ a phrase or more of words written by another person, clearly mark the passage used as a quotation, and cite the source in full. Note, even if what you write is not verbatim (identical to) what is written in the source document, if the key idea is from some other document, then that document must be cited. This requirement applies even if the source is the course lecture notes, a web site, or any “study guide” informally circulated among students, whether in paper or electronic form.
“EXTRA CREDIT” WORK: There will be none. The course already contains many pointers for “extra” work within its assignments. The answer to any request that a student be allowed to bring their grade up to some level (say, B-) through work not already described in course materials -- as opposed to doing a proper job on the regular assignments -- will be “NO.”
“MAKE-UP” WORK: (a) Assignments: Simulation assignments turned in after the due date for whatever reason are eligible for a maximum of 80% credit. For example, a student receiving less than an 8 (on a 10-point scale) on a given simulation assignment may resubmit a corrected version of that assignment within 5 weeks of the original due date, in order to bring the grade up to 8 (i.e. 80% of full credit). If a less-than- perfect assignment is submitted on the second round, the grade will be 80% of what that assignment would have earned on the first round. No amount of subsequent extra work on that assignment can make the resulting grade higher than 8. (b) Examinations: Students who are unavoidably absent from the in-class examination will take a special make-up examination consisting of written and oral portions as soon as one can be scheduled.
EMAIL: An alias called cn530-at-cns.bu.edu has been set up in order to broadcast information of interest to people in this class. Email will never be used as the sole source of important information, but may serve to speed up administrative matters. The accounts of the professor and teaching fellow are included. Any student may send a message to the account, for items likely to be of general interest.
CONFIDENTIALITY OF PERSONAL WORK: All students using University computers for doing simulations, word processing, or figure generation for assignments or take-home examinations are expected to read-protect their files.
LECTURE NOTES AND MISCELLANEOUS READINGS: Copies of all lecture notes for the course are available for from the course web site. Required readings that are not contained in the textbooks will be made available, as will be explained during the first class.
BU POLICY: The syllabus, course descriptions, and handouts created by Professor Mingolla, and all class lectures, are copyrighted by Boston University and/or Professor Mingolla. Except with respect to enrolled students as set forth below, the materials and lectures may not be reproduced in any form or otherwise copied, displayed or distributed, nor should works derived from them be reproduced, copied, displayed or distributed without the written permission of Professor Mingolla. Infringement of the copyright in these materials, including any sale or commercial use of notes, summaries, outlines or other reproductions of lectures, constitutes a violation of the copyright laws and is prohibited. Students enrolled in the course are allowed to share with other enrolled students course materials, notes, and other writings based on the course materials and lectures, but may not do so on a commercial basis or otherwise for payment of any kind. Please note in particular that selling or buying class notes, lecture notes or summaries, or similar materials both violates copyright and interferes with the academic mission of the College, and is therefore prohibited in this class and will be considered a violation of the student code of responsibility that is subject to academic sanctions.
BOOKS MOST RELEVANT TO THE COURSE: Two books have been ordered for CN 530:
Palmer, S. E. (1999). Vision science: From photons to phenomenology. Cambridge, MA: MIT Press. (Approx. $80.00). The bookstore lists it as “required,” in the sense that many required readings can be found there. You may choose to use the CNS library edition of this book, or to purchase it. In the weekly listing of required readings, this book is designated as “Palmer.” Note that all chapters of this book may be available for downloading with a subscription to Cognet: http://cognet.mit.edu/
Yantis, S. (2000) Visual Perception: Essential Readings. Psychology Press, (Approx $45, paperback.) In the weekly listing of required readings, this book is designated as “Yantis.”
You may wish to consider additional purchases, which have not been ordered for CN 530. Some comments are included below to help you make purchasing decisions. Books are interest in the order in which they are most likely to be useful for most students.
Strunk, W., Jr., and White, E.B. The Elements of style. 4th edition. Boston, Allyn & Bacon, 2000. At $6.95 (paperback) this may be the best book, on a price/performance basis, you ever purchase. One cannot overstate the importance of being able to communicate your ideas in forceful and direct English.
Wandell, B. A. (1995). Foundations of vision. Sunderland, Massachusetts: Sinauer Assoc., Inc. The bookstore lists this book as “optional.” This book contains an overview of current research issues in visual perception. Several chapters of this book were required reading in past editions of CN 530; in the weekly listing of supplementary readings, this book is designated as “BAW.” This book is tutorial in organization, with a clear emphasis on “vision science,” rather than visual perception. (Please ask me if you do not know the difference.) (Approx. $50.00).
Kandel (Editor) E. R., Schwartz, J. H., and Jessell, T. M. (Eds), Principles of Neural Science, 4th Edition. New York: McGraw-Hill. (hardcover $85.00) The 4th edition of “KSJ” is by now well out of date, and a new edition is due “soon.” You may wish to hold off on purchasing, unless you find a good price on a used volume. Many present CNS students own copies of this.
Kosslyn, S. M. & Anderson, R. A. (Eds.) (1992). Frontiers in cognitive neuroscience. Cambridge, MA: MIT Press. ($70.00, hardbound) This collection reprints “classic” papers in many in areas besides vision. In the weekly listing of required readings, this book is designated as “K & A.”
Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. ($61.50, paperback) This book includes core papers describing the work on vision done at the Center for Adaptive Systems. It also contains many other papers that will be used for other courses in CNS.
Answers to CN 530 FAQs:
Question 1: Why is there so much required reading? (In other words, what do I really have to do to get an A? What parts of this stuff can I skip?)
Answer: I have tried, through points raised in lectures, notes interspersed throughout the syllabus, the creation of a study guide, and the provision of ready access to prior examination questions to be as explicit as I know how to be about what you are expected to know. Further suggestions are welcomed. There’s simply a lot to know about vision before you can even start to model it.
Question 2: Aren’t you really making us read more than is really important? Couldn’t you tell us more explicitly which sections, figures, equations, paragraphs, or sentences really matter?
Answer: Yes, it’s true. There remain a few places where I could have been even more explicit than I have been about how you should separate wheat from chaff. By electing to take this course, however, you have embarked on study of an area so unformed that, for many topics, consensual “textbook” knowledge does not exist. Soon enough you will have to confront primary source material without any of the aids provided in this course! Part of my job is to train you to meet the challenge of transitioning from “undergraduate mode” to “researcher mode.” I have done this in part by assigning -- in a few places -- entire chapters or articles for which I know that whole sections could be skipped without undue harm! Part of your job is to figure out which are those sections, and not to worry about them.
Question 3: Some parts of some readings contradict parts of other readings. What’s going on?
Answer: Welcome to the real world of science.
Question 4: Why do I have to bother with all this silly psychology and complicated physiology? How is this going to help me design real world vision applications?
Answer: Let’s talk about this during our discussion periods.
Question 5: Why are so many of the course readings from primary sources (original research articles, as opposed to textbook chapters)? The authors use different terminology for the same concepts, and often contradict one another, and they take way more pages to explain things than a textbook does.
Answer: No single textbook appropriate for the entire course exists. The books that exist are either too elementary, too detailed, or too narrow in scope, and they barely cover much of the core material in the course. For certain ideas, there simply is no present substitute for primary sources. (Remember, that’s correlated with our department being involved in emerging, new, exciting, interdisciplinary, lemon-scented research!) Even in the case of psychophysics, and physiology, which the textbooks cover at least moderately well, I have asked you to read some primary sources. I believe that the extra effort required to read them will be rewarded by deeper understanding than can be gotten from textbooks. In any case, I encourage you to go back and forth, between the two types of readings, until you have satisfied yourself that you can master the material outlined in the study guide and presented in class.
Question 6: How will I know when I’m “getting it,” given how amorphous and confusing some of the readings are? How do I know which version of several descriptions of, for example, physiological functions of some visual area, is right?
Answer: Where experts disagree, you are entitled to make an informed choice among reasonable alternatives. You are, however, expected to understand the issues underlying the disagreement.
Question 7: How should I study for the in-class midterm?
Answer: During the mid-term, you will not be reading long articles, or reviewing lecture notes, so do not spend all of you preparation time in those activities! During the exam you will primarily be writing. You should practice writing. You should practice writing concise answers to short questions in limited time. I will do my best to make the exam less a measure of your rate of expression and more a matter of assessing your mastery of content. You can help avoid unpleasant surprises by giving yourself one or two “practice” tests based on the study guide, without notes or readings, and in a realistically short amount of time. I would be happy to give you feedback on sample answers that you show me.
Question 8: Much of this course seems rigidly laid out; what if we want to do things differently (e.g. read unassigned articles, or do different simulations than required in assignments.)
Answer: Everything about this course is evolving, and everything is negotiable. Remember, though, that any proposed improvement has a cost (in human effort) that must be budgeted. The class motto is: “To suggest is to volunteer.”
WEEKLY TOPICS AND READINGS
The readings are listed along with a short synopsis of the theme of each week’s lecture. Readings for each week are designated under the headings Required Reading, Supplementary Reading, and BONUS Reading. You will be responsible (in the sense of possibly being tested) for material covered under the “required” heading only – but note, a few “required” readings are also labeled by a boldface PLUS. These readings are typically recent reviews, and may seem dense with references to unfamiliar material. Spend about an hour on such readings and get what you can, but do not worry that material in these readings will be “on the test.” I will not include items on the midterm that are intended to probe these readings, nor are you likely to find basic definitions of fundamental concepts or terms in these readings. Material listed as supplementary generally falls into one of two categories. The first includes “enrichment” or “remedial” readings of relatively broad interest; these are generally followed by short parenthetical comments. The second category includes technical or scholarly citations. Those supplementary readings that are indicated by a bullet (•) are likely to be the most useful to you, ask me about them if you have any trouble locating copies. BONUS readings are listed because reading them might be fun. (Those contemplating careers as university professors can use these as a diagnostic; if you do not enjoy a significant portion of the bonus readings, you may wish to explore another line of work!) NOTE: Students will be expected to have read all of the required readings listed for a given week by the time that lecture is given, in the sense that the contents of the lecture will assume some familiarity with the readings. That is, the lectures will often comment upon the readings, rather than acting as a substitute for doing the readings.
Week 1: Fundamental problems of vision
1) Unit formation and grouping
2) Seeing and recognizing -- form/color interactions
3) Retinal veins and blind spot
4) Perceiving surface color: Constancy, contrast, and discounting the illuminant
5) Stabilized images: Boundaries and featural color and brightness
6) Complementary processing: Unoriented and oriented detectors
7) The noise--saturation dilemma
8) Reflectances and ratios; shunting and mass action
Required Reading:
There are no “required” readings for Week 1, insofar as you could not be expected to know what to read to prepare for the first lecture. However, three of the readings listed below are special in the sense that reading them is a “requirement” for saying that you know anything about current approaches to vision. Those readings are (parts of) Marr (1982) and Köhler (1947), and Gibson (1969), listed below on this page. In the best of all possible worlds, you would have been exposed to these three authors before starting this course. In any case: (1) Grossberg’s early career overlapped the abbreviated career of Marr; the two were intellectual rivals. (2) The intellectual underpinnings of the modeling of grouping and segmentation processes considered in the middle of the course are clarified by the Köhler reading, and (3) Gibson was a legitimate genius, whose views changed the course of 20th century research on vision. In particular, his views on the specification of environmental structure by information in the optic array were adapted by Marr and his colleagues into their tenets on the development of “computational theory.” (Note that Gibson is the only intellectual rival attacked by name in Marr’s first chapter.) If pressed for time, consider Marr the first priority. Köhler can wait for several weeks into the course, and Gibson can wait until later.
Supplementary Reading:
• Marr, D. (1982). Vision, Chapter 1. “The philosophy and the approach.” San Francisco, W.H. Freeman. Marr argues clearly and persuasively for a point of view that has enjoyed considerable popularity in recent years. Much of the CAS/CNS work in vision can be cast in counterpoint -- explicit or implicit -- to Marr’s views. Reprinted in Yantis, Chapter 5.
• Köhler, W. (1947). Gestalt Psychology. New York, New American Library. Chapter IV, “Dynamics as opposed to machine theory”, 60-79. (This chapter contains some obscure allusions to old psychological concepts, but is still one of the most inspiring statements of the “dynamical systems” view of psychological processes!) PDF.
• Gibson, J. J. (1979). The ecological approach to visual perception. Chapter 14: “The theory of information pickup and its consequences.” Boston, Houghton-Mifflin. Reprinted in Yantis, Chapter 4.
Week 2: Shunting competitive networks and representation in early vision
1) Brightness: Constancy and contrast
2) Shift property and Weber law
3) Retinal physiology
4) Hyperpolarization and featural noise suppression
5) Distance--dependent shunting networks
6) Another approach (Marr)
7) Recurrent competitive networks
Required Readings: (To be read BEFORE class)
PALMER. Read Chapters 1 and 2 for “background.” Also read Ch. 4, Sec. 3.
Grossberg, S. (1982). Why do cells compete? UMAP Unit 484, The UMAP Journal, Vol. III, No. 1. (Education Development Center, 0197-3622/82/010101.) (This is by far the “easiest” introduction to shunting inhibition Grossberg has ever written.) PDF.
KSJ. Read Ch. 25 and Ch. 26. Also, skim Ch. 21 if you have had no undergraduate introduction to perception or cognition. Ch. 26 contains some details of the pharmacology of receptor phototransduction that will not be “on the test for CN 530,” as clarified in class.
Yantis. Chapter 14. Wallach, H. (1948) Brightness constancy and the nature of achromatic colors. Journal of Experimental Psychology, 38, 310-324. (A “light touch” on reading this one is okay. Read the summary first.)
Supplementary Reading:
Note: While not “required,” the second article listed below will be of particular interest to those of you concerned with computer vision (and is very short!)
• Grossberg, S. (1973). Contour enhancement, short term memory, and constancies in reverberating neural networks. Studies in Applied Mathematics LII, 213-257. Reprinted in S. Grossberg, Studies of mind and brain (1982), Boston, Reidel. This is one of the foundational papers in the area of recurrent competitive networks. PDF.
• Boyer, K. L., and Sarkar, S. Computer Vision and Image Understanding. Perceptual Organization in Computer Vision: Status, Challenges, and Potential. Vol. 76, No. 1, October, pp. 1–5, 1999, Article ID IV990797. PDF.
Cornsweet, T. (1970). Visual Perception. New York, Academic Press. Chapter XI, “The psychophysiology of brightness -- I”, 268-310. (While this discussion is somewhat dated, it is lucid; also, Cornsweet’s views about subtractive inhibition are illustrative of views that many of Grossberg’s discussions of shunting inhibition are directed against.)
Borg-Graham LJ, Monier C, Fregnac Y (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons, Nature, 393(6683), 369-373. PDF.
BONUS Reading:
Yantis, Ch 2. Barlow, H. B. (1972) Single units and sensation: A neuron doctrine for perceptual psychology? Perception 1:371-394. Reprinted as Chapter 14 of J. A. Anderson, A. Pellionisz, and E. Rosenfeld (Eds.) Neurocomputing 2, Directions for Research. Cambridge, MA, MIT Press, 1988. This is one of the foundational papers from the recent “single unit” era in physiological psychology.
1) Anatomical and physiological techniques
2) Retinal structure and function
3) ON and OFF channels
4) Anatomy and physiology of the early visual pathways
Required Reading: (To be read BEFORE class)
KSJ. Read Ch. 27.
Schiller, P. H. On the specificity of neurons and visual areas. Behavioural Brain Research, 1996, 76 (21-35). PDF.
PLUS: Bullier J. (2001). Integrated model of visual processing. Brain Res Brain Res Rev.36(2-3):96-107. PDF. This paper may seem “dense,” particularly if you are new to the physiology of vision. Such papers are intended to provoke thought rather than to add to the list of items that you will appear on the mid-term exam. For now, just make an honest effort to read with some care.
PLUS: http://ohzawa-lab.bpe.es.osaka-u.ac.jp/ohzawa-lab/teaching/AA_RFtutorial.html Spend an hour on this site, viewing demos and downloading Receptive-field dynamics in the central visual pathways (TINS 1995) by DeAngelis, Ohzawa, and Freeman.
Supplementary Reading:
• Levine, D. and Grossberg, S. (1976). Visual illusions in neural networks: Line neutralization, tilt after-effect, and angle expansion. Journal of Theoretical Biology, 61, 477-504. Levine was Grossberg’s first Ph.D. student; this reading is also relevant to Simulation Assignment 5. PDF.
• Izhikevich, E.M. (2004). Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks, vol. 15 (5), pp.1063-1070. PDF.
Gilbert, C. D. Plasticity in visual perception and physiology. Current Opinion in Neurobiology 1996, 6:269-274. PDF.
Bullier, J. and Nowak, L. G. Parallel versus serial processing: new vistas on the distributed organization of the visual system. Current Opinion in Neurobiology, 1995, 5:497-503. PDF.
BAW: Read Chapter 7.
• Callaway EM Local circuits in primary visual cortex of the macaque monkey Annual Review of Neuroscience 21, 47-74 1998. Comprehensive review. PDF.
• Schiller, P. H. (1986). The central visual system. Vision Research, 26, (9), 1351-1386. This is a scholarly and entertaining review of early work in neurophysiology. See Part A, pages 1351-1362. PDF.
Ellias, S. and Grossberg, S. (1975). Pattern formation, contrast control, and oscillations in the short term memory of shunting on-center off-surround networks. Biological Cybernetics, 20, 69-98. PDF.
BONUS Reading:
Sacks, O. (1995). The case of the colorblind painter. Pages 3-41 in O. Sacks, An anthropologist on Mars. New York: Alfred Knopf.
Week 4: Contrast sensitivity and spatial scales
1) Structural scales: functional scales :: kernels: receptive fields
2) Peak shifts and lateral inhibition
3) Detectors and filters -- linear systems approach to vision
4) Contrast sensitivity and spatial scales
5) Brightness perception: Quantifying percepts
6) Isomorphistic and nonisomorphistic theories
7) Craik-O’Brien-Cornsweet (COCE) effect
8) Retinex algorithm
Required Reading: (To be read BEFORE class)
PALMER. Ch. 4.
Kaufman, L., (1974). Sight and Mind. New York, Oxford University Press. Chapter 5, “Contrast and contour”, 128-152. (This is a good overview of some classic issues in spatial vision. Don’t worry if a few parts seem obscure.) PDF.
Grossberg, S. (1983). The quantized geometry of visual space: The coherent computation of depth, form, and lightness. Behavioral and Brain Sciences, 6, 625-692. Reprinted as Chapter 1 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. Read Sections 1-3 and 21-25, 27, and 28, and the commentaries of Grimson and Stevens and Grossberg’s reply to those commentaries. Note that the Commentary section appears only in the journal article and is not reprinted in the book. Sections 21-25 restate and extend the discussion of the “UMAP Module” reading. PDF.
Adelson, E.H. (2000). Lightness Perception and Lightness Illusions, in M. Gazzaniga, M.S., ed., The New Cognitive Neurosciences, 2nd Ed.Cambridge, MA: MIT Press, pp. 339-351. PDF. This is a clearly written article that explains much of the important terminology used in the study of lightness perception.
PLUS: Sincich, L. C. and Horton, J. C. (2005). The circuitry of V1 and V2: Integration of color, form, and motion. Annual Review of Neuroscience, 28:303-26. While this article is dense and contains a lot more information than “will be on the test,” you should spend an hour or so on this reading, as an antidote to the simplifications of the KSJ treatment of these areas. PDF.
Supplementary Reading:
• Neumann H. (1996). Mechanisms of neural architecture for visual contrast and brightness perception. Neural Networks, 9(6), 921-936. NOTE: You may wish to consult this paper while doing Simulation Assignment 2. (Downloadable at: http://www.sciencedirect.com .)
• Kiper, D. and Carandini, M. The neural basis of pattern vision. Encyclopedia of cognitive science , 2000, Macmillan Reference, Ltd. PDF.
• Westheimer G., The Fourier theory of vision. Perception, 30(5), 531-541. This article is particularly lucid and worthwhile, if you have the background to appreciate it. PDF.
Gaudiano P. (1994). A nonlinear model of spatiotemporal retinal processing: simulations of X and Y retinal ganglion cell behavior. Vision Research, 34, 1767--1784. PDF.
Week 5: Brightness and lightness
1) Brightness assimilation
2) Grossberg and Todorovi (T) implementation of BCS/FCS
3) G & T simulations
4) Integration models (e.g. Retinex)
5) Challenges to brightness models
Required Reading: (To be read BEFORE class)
PALMER. Skim Ch 3. Read Sec. 3.3 carefully.
Todorovi, D. (1987). The Craik--O’Brien--Cornsweet effect: New varieties and their theoretical implications. Perception & Psychophysics, 42, 545-560. Do not fret the details; read for gist and concentrate on the distinction between isomorphistic and nonisomorphistic theories. While a bit dense, this article provides a useful way to partition contemporary “styles” of modeling. PDF.
Grossberg, S. and Todorovi, D. (1988). Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena. Perception & Psychophysics, 43, 241-277. PDF. Do a “first pass” on this paper. Concentrate on the role of boundaries and diffusion in explaning percepts. The lecture covers the Appendix in detail, and you will revisit this paper in Simulation Assignment 3.
Marr, D. (1982). Vision. New York, W.H. Freeman. Pages 250-258. Marr gives a lucid overview of Land’s Retinex theory; read this before Land (1986). PDF.
Land, E. H. (1986). Recent advances in Retinex theory. Vision Research, 26(1), 7-21. You will be
expected to understand this approach, an example of an “integration theory,” in excruciating detail. PDF.
Check out Retinex-based commercial image processing at: http://dragon.larc.nasa.gov/retinex/
Compare Simon Hong’s results by following “projects” link at: http://cns-alumni.bu.edu/~yhong/
Supplementary Reading:
For Retinex Matlab code, go to:http://www.cs.sfu.ca/~colour/publications/IST-2000/
Gilchrist A, Kossyfidis C, Bonato F, Agostini T, Cataliotti J, Li X, Spehar B, Annan V, Economou E. An anchoring theory of lightness perception. Psychological Review, 1999 Oct;106(4):795-834. PDF.
Cornsweet, T. (1970). Visual Perception. New York, Academic Press. Chapter XII, “Psychophysiology of brightness -- II, Modulation transfer functions,” 311-364.
Graham, N. (1980). Spatial frequency channels in human vision: Detecting edges without edge detectors. In C. S. Harris, Ed., Visual coding and adaptability. Hillsdale, NJ, Earlbaum, 215-262. The experiment described in Figure 6 (page 226) and the accompanying text is of fundamental importance to understanding issues related to “spatial frequency channels” in human vision.
BONUS READING
Daugman, John. (1990) Brain metaphor and brain theory. Chapter 2 of Eric Schwartz (Ed.) Computational Neuroscience. Cambridge, MA: MIT Press. Reprinted as Chapter 2 in Philosophy and the Neurosciences, edited by W. Bechtel et al. Oxford: Blackwell Publishers. (Scanned .PDF file here). (This essay is a timely and entertaining polemic on the meaning of the word “computational.”)
Week 6: Parallel visual pathways, boundaries and surfaces
1) A simple BCS-FCS model
2) Diffusion and time
3) Symbolic models and energy models
4) Edge detection?
5) How thin is “thin”?
6) Spatial and orientational competition
7) Hyperacuity
8) Neon color spreading
Required Reading: (To be read BEFORE class)
PALMER. Ch 6.
Neumann, H. and Mingolla, E. (2003) Contour and surface perception. In M.A. Arbib, Ed., Handbook of brain theory and neural networks, II. Cambridge, MA: MIT Press. PDF.
PLUS: Komatsu H. (2006) The neural mechanisms of perceptual filling-in. Nat Rev Neurosci. 7(3):220-31. Actually, this is NOT a “required reading,” (in the sense of a reading whose material will be “on the test.”) I have included it here to get the attention of any skeptics who might thing that this “filling-in” idea is just some obsession of a handful of researchers. PDF.
Supplementary Reading:
• Pessoa, L., Thompson, E., & Noe, A. Finding out about filling-in: a guide to perceptual completion for visual science and the philosophy of perception. Behavioral and Brain Sciences, 1998 Dec;21(6):723-48. PDF.
Davey, M. P., Maddess, T., and Srinivasan M. V. (1998). The spatiotemporal properties of the Craik-O’Brien-Cornsweet effect are consistent with “filling-in”. Vision Research, 38(13), 2037-2046. The title speaks for itself. PDF.
BAW: Read Chapter 6.
Paradiso, M. A. and Nakayama, K. (1991). Brightness perception and filling-in. Vision Research, 31, 1221-1236. PDF.
Hung CP, Ramsden BM, Chen LM, Roe AW., Building surfaces from borders in Areas 17 and 18 of the cat., Vision Res. 2001;41(10-11):1389-407. PDF. Take a look for some electrophysiological evidence regarding the effect of boundary contrast on surface lightness.
Gerrits, H. J. M., and Vendrick, A. J. H. (1970) Simultaneous contrast, filling-in process and information processing in man’s visual system. Experimental Brain Research, 11, 411-430.
<<Week 6 readings continue on the next page >>
Week 6 (Continued)
Supplementary Reading (continued):
Grossberg, S. and Mingolla, E. (1985). Neural dynamics of form perception: Boundary completion, illusory figures, and neon color spreading. Psychological Review, 92(2), 173-211. Reprinted as Chapter 2 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. PDF. Concentrate on the arguments for making the Boundary/Feature distinction in the first place; on the conditions that produce neon color spreading, and on the mechanisms of the theory’s explanation of neon color spreading. Skim lightly over the sections on cooperative completion of boundaries. In other words, concentrate on Sections 1-15. The Appendix of this article is subsumed by that of a subsequent article by the same authors in 1985.
Bressan, P., Mingolla, E., Spillmann, L. and Watanabe, T. (1997). Neon color spreading: A review. Perception, 26(11), 1353-1366. PDF.
Badcock, D. R., and Westheimer, G. (1985). Spatial location and hyperacuity: The center/surround localization contribution function has two substrates. Vision Research, 25, 1259-1267. PDF.
BONUS Reading:
Westheimer, G. (1983). Herman Helmholtz and the origins of sensory physiology. Trends in Neurosciences, Jan., 5-9. (Did you know that Helmholtz is considered by many to be the greatest sensory psychologist who ever lived?) PDF.
Week 7: Boundary detection, completion, and sharpening
1) How thin is “thin”
2) Spatial and orientational competition
3) Hyperacuity
4) Neon color spreading
5) Cooperative-Competitive (CC) Loop
6) Bipole cells, then and now
7) von der Heydt, Peterhans, & Baumgartner, 1984
8) Spatial impenetrability
Required Reading: (To be read BEFORE class)
Grossberg, S. and Mingolla, E. (1985). Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations. Perception & Psychophysics, 38, 141-171. Reprinted as Chapter 3 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. PDF. Yes, this is one of those articles containing paragraphs that “won’t be on the test.” Use the study guide and lecture notes for clues about which. (What is the difference between an “invisible” boundary and an “illusory” one?)
KSJ. Read Ch. 28.
Lamme VAF, Super H, Spekreijse H Feedforward, horizontal, and feedback processing in the visual cortex. CURR OPIN NEUROBIOL 8: (4) 529-535 AUG 1998. PDF.
von der Heydt, R., Peterhans, E., and Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262. PDF. (This paper describes some striking evidence for long- range cooperative interactions in early vision. Reprinted as Ch. 12 of K & A. You are expected to understand the reported experiments in detail.)
PLUS: Roelfsema PR. (2006) Cortical algorithms for perceptual grouping. Annu Rev Neurosci. 29:203-27. PDF.
Supplementary Reading:
• Francis, G., Grossberg, S., and Mingolla, E. (1994). Cortical dynamics of feature binding and reset: Control of visual persistence. Vision Research , 34 (8), 1089-1104. pdf
Spillmann, L., Werner, J.S. Long-range interactions in visual perception. TRENDS NEUROSCI 19: (10) 428-434 OCT 1996. PDF.
Fitzpatrick,D. Seeing beyond the receptive field in primary visual cortex. CURR OPIN NEUROBIOL 10: (4) 438-443 AUG 2000. PDF.
Takeichi H, Shimojo S, Watanabe T. (1992). Neon flank and illusory contour: interaction between the two processes leads to color filling-in. Perception 21(3):313-24. PDF. This shows how psychophysics can help localize the areas responsible for boundary and surface formation.
<<Week 7 readings continue on the next page >>
Week 7 (Continued)
Supplementary Reading (continued):
• Sarti A, Malladi R, Sethian JA Subjective surfaces: A method for completing missing boundaries
P NATL ACAD SCI USA 97: (12) 6258-6263 JUN 6 2000. (Of interest to students concerned with computer vision.) PDF.
Neumann, H. and Mingolla, E. 2001 Computational neural models of spatial integration in perceptual grouping. In From Fragments to Objects: Grouping and Segmentation in Vision. T.F.Shipley & P.J.
Kellman, Editors. Amsterdam: Elsevier, 353-400. PDF. Note: For reasons surpassing human understanding, this PDF file will print on some printers and not on others.
Gove, A., Grossberg, S., and Mingolla, E. (1995). Brightness perception, illusory contours, and corticogeniculate feedback. Visual Neuroscience, 12, 1027--1052. PDF.
Grossberg, S. Mingolla, E. & Ross, W. D. (1997). Visual brain and visual perception: A corticogeniculate model of perceptual grouping. Trends in Neurosciences (TINS), 20(3), 106-111. PDF.
Lesher, G. W. (1995). Illusory contours: Toward a neurally based perceptual theory. Psychonomic Bulletin and Review, 2(3), 279-321. (This is the literature review on illusory contours, at least up to its publication date.)
Lesher, G. W. & Mingolla, E. (1993). The role of edges in line-ends in the formation of illusory contours. Vision Research, 36(16), 2253--2270. B & G: Chapter 5. PDF.
Redies, C. and Spillmann, L. (1981). The neon color effect in the Ehrenstein illusion. Perception, 10, 667-681.
S. Petry and G. E. Meyer, Eds. (1987), The perception of illusory contours. New York, Springer- -Verlag. (This is the book on illusory contours.)
Koffka, K. (1935/1963). Principles of Gestalt psychology. New York, Harcourt, Brace, and World. Chapter 4, “The environmental field,” 106-176, contains many of the seminal notions of Gestalt psychology. See especially pages 148-176.
Ullman, S. (1984). Visual routines. Cognition, 18, 97-106. Reprinted in M. A. Fischler and O. Firschein, Eds., Readings in computer vision. 1987, Los Altos, CA, Morgan Kaufman, 298-328. PDF.
Zucker, S. W. (1985). Early orientation selection: Tangent fields and the dimensionality of their support. Computer Vision, Graphics, and Image Processing, 32(1), 74-103. Also in M. A. Fischler and O. Firschein, Eds., Readings in computer vision. 1987, Los Altos, CA, Morgan Kaufman, 333- 348. PDF.
Ramsden BM, Hung CP, Roe AW. (2001). Real and illusory contour processing in area V1 of the primate: a cortical balancing act., Cereb Cortex. Jul;11(7):648-65. This article shows the difference between V1 and V2 units in response to illusory contours. Reading the abstract, figure legends and discussion suffices. PDF.
BONUS Reading:
Bateson, G. (1979). Every schoolboy knows... Chapter 2 of Mind and Nature. Toronto, Bantam. Bateson was weird; do you know the definition of “sacrament” as given in the Baltimore Catechism of the Roman Catholic Church? If not, how will you ever understand “emergent” behavior of neural networks? For that matter, how will you ever know what remarks in a course syllabus to take seriously?
Week 8: The phenomena of motion perception
1) What is motion? Apparent motion and real motion
2) Long-range motion: Arguments for short and long range mechanisms
3) Korte’s laws and figural affinity: Traveling Gaussian waves (G-waves -- Grossberg and Rudd)
4) Short and long-range motion
5) Fourier and non-Fourier stimuli
6) Gradient models (Marr/Ullman)
7) Energy models (Adelson/Bergen)
8) Correlation models (Reichardt; van Santen/Sperling
Required Reading: (To be read BEFORE class)
PALMER. Ch 10.
Grossberg, S. and Rudd, M. E. (1989). A neural architecture for visual motion perception: Group and element apparent motion. Neural Networks,2, 421-450. Read pages 421-433. Do not get “bogged down” in the description of the 5 levels of the model and on oddities such as “rightward propagation of leftward motion signals.” Concentrate on the traveling Gaussian wave mechanism, period. PDF.
Go to the Magni-phi web site: http://www2.psych.purdue.edu/Magniphi/ View the demos, including Custom Magni-phi.
Supplementary Reading:
BAW, Chapter 10
• Cavanagh, P. and Mather, G. (1990). Motion: The long and the short of it. Spatial Vision, 4, 103- 129. PDF.
Nakayama, K. (1985). Biological image motion processing: A review. Vision Research, 25(5), 625-660. Read 625-651. PDF.
Kolers, P. A. (1972). Aspects of motion perception. Oxford: Pergamon Press. Read pages 1-58.
Adelson, E. H. and Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2(2), 284-299. PDF.
Anstis, S. (1988). Motion perception in the frontal plane. Chapter 16 of K. R. Boff, L. Kaufman, and J. P. Thomas, Eds., Handbook of perception and performance, Volume I, Sensory processes and perception, pages 16-1 to 16-27.
van Santen, J. P. H. and Sperling, G. (1985). Elaborated Reichardt detectors. Journal of the Optical Society of America A, 2(2), 300-321. PDF.
BONUS Reading:
Yantis, Chapter 4. Gibson, J. J. (1979). The ecological approach to visual perception. Chapter 14: The theory of information pickup and its consequences (pp. 238-263). Boston: Houghton Miflin Co. Gibson was the most important perceptual psychologist of the 20th century. The force of his arguments was so great that many today subscribe to important aspects of his views without realizing the source!
Week 9: Models of motion perception
1) Group and element motion
2) Motion pooling and aperture problem
3) Motion detection, segmentation and grouping
Required Reading: (To be read BEFORE class)
Grossberg, S., Mingolla, E. and Viswanathan, L. (2001). Neural dynamics of motion integration and segmentation within and across apertures. Vision Research, 41(19), 2521-53. PDF.
Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual area MT. Vision Res,earch 38(5), 743-61. Note: This reading is only “quasi-required,” in the sense that you should read it carefully enough to get the gist of the modeling effort, without worrying about every last implementation detail. PDF.
Pack CC, Born RT. (2001). Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain. Nature. Feb 22;409(6823):1040-2. PDF. This is a great paper.
Supplementary Reading:
• Born RT, Bradley DC. (2005) Structure and function of visual area MT.Annu Rev Neurosci. 2005;28:157-89. PDF.
• Allman, J., Miezin, F., & McGuiness, E. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105- 126. Reprinted as Ch. 11 of K & A.
Grossberg, S. and Rudd, M. E. (1992). Cortical dynamics of visual motion perception: Short-range and long-range apparent motion. Psychological Review, 99(1), 78-121. Read pages 78-82 and 90- 96. PDF.
Marshall, J. A. (1990) Self-organizing neural networks for perception of visual motion. Neural Networks, 3(1), 45-74. PDF.
Livingstone MS, Pack CC, Born RT. (2001). Two-dimensional substructure of MT receptive fields. Neuron. Jun;30(3):781-93. PDF. This article teaches you experimental techniques in motion electrophysiology with classic citations about each aspect of the technique. Figure captions are clear, therefore you can use the abstract-figure captions-discussion strategy again!
Mingolla, E., Todd, J. T., & Norman, J. F. (1992). The perception of globally coherent motion. Vision Research, 32(6), 1015--1031. PDF.
Albright, T. D., Desimone, R., and Gross, C. G. (1984). Columnar organization of directionally sensitive cells in visual area MT of the macaque. Journal of Neurophysiology, 51, 16-31.
BONUS Reading:
Gregory, R. L. (1991). What is caught in
neural nets? Perception, 19, 561-568.
Gregory is the most influential exponent of the “cognitivist” position in
vision today, though his views resist simple categorization. Regardless of
whether you agree or disagree with him, this essay is fun!
There will be an in-class EXAMINATION, covering the topics in the readings and lectures from the first 9 weeks, during the class period.
BONUS Reading:
Thompson, D. A., (1917/61). On magnitude.
Chapter 2 of On growth and form
(Abridged edition). Cambridge: Cambridge University Press. (For a system such
as the BCS to be automatically “scaled up” to deal with large images requires a
deeper understanding of the Principle of Similitude for nonlinear dynamics than
we seem presently to possess.) If you read only one bonus reading all year,
make it this one.
Week 11: Approaches to textural segmentation and grouping
1) What is texture?
2) Textural segmentation -- textons?
3) Representations for segmentation
4) LaminART
Required Reading: (To be read BEFORE class)
Yantis, Chapter 17. Mishkin, M. Ungerleider, L. G., and Macko, K. A. (1983). Object vision and spatial vision: Two cortical pathways. Trends in Neurosciences, 6, 414-417. This classic paper launched decades of research and debate.
Beck, J. (1993). The British Aerospace Lecture: Visual processing in texture segregation. In D. Brogan, A. Gale and K. Carr, Eds. Visual Search 2. London: Taylor and Francis. Read 1-11, 21-25. PDF.
Malik, J. and Perona, P. (1990). Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A, 7(5), 923-932. PDF. Note: For an overview of the Malik and Perona model, check out the texture segregation section in SEP - pp. 275-280.
Grossberg, S. and Raizada, R. (2000) Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex. Vision Res. 40(10-12):1413-32. PDF.
Supplementary Reading:
• Lund, J.S., Angelucci, A. and Bressloff, P.C. (2003). Anatomical substrates for functional columns in macaque monkey primary visual cortex. Cerebral Cortex, vol. 12, pp 15-24. This paper provides a more nuanced view of a cortical column than appeared in your earlier readings, and gives physiological background pertinent to LAMINART. PDF.
BAW, Chapters 8.
• Beck, J. (1983). Textural segmentation, second-order statistics, and textural elements. Biological Cybernetics, 48, 125-130.
• Julesz, B. and Bergen, J. R. (1983). Textons, the fundamental elements in preattentive vision and perception of textures. Bell Systems Technical Journal, 62(6) Part II: 1619-1645. Reprinted in M. A. Fischler and O. Firschein, Eds., Readings in computer vision. 1987, Los Altos, CA, Morgan Kaufman, 243-256.
Zucker, S. W., Dobbins, A., and Iverson, L., (1989). Two stages of curve detection suggest two styles of visual computation. Neural Computation, 1(1), 68-81.
Beck, J., Prazdny, K. and Rosenfeld, A. (1983). A theory of textural segmentation. In J. Beck, B. Hope, & A. Rosenfeld (Eds.), Human and machine vision. New York: Academic Press, 1-38.
BONUS Reading:
Stevens, P. S., (1974). Basic patterns (Chapter 2) and Spirals, meanders, and explosions. Chapter 4 of Patterns in Nature. Boston: Little, Brown, and Co. (Visual contours are also among the “patterns in nature,” more complex and more beautiful than even Stevens’s examples.)
1) Disparity and depth
2) Projection theories
3) The correspondence problem
4) Matching algorithms
5) Prazdny’s algorithm
6) Grimson’s wedding cake
7) Kaufman’s stereogram: Rivalry
8) Occlusion, depth, and da Vinci stereopsis
9) Modal and amodal perception (NOT transparency)
Required Reading: (To be read BEFORE class)
PALMER. Ch 5, Secs. 1-3.
Kaufman, L. (1974). Binocular stereopsis. In L. Kaufman, Sight and Mind. New York, Oxford University Press, 269-321. PDF.
Grossberg, S. 1983. The quantized geometry of visual space: The coherent computation of depth, form, and lightness. Behavioral and Brain Sciences, 6, 625-692. Reprinted as Chapter 1 of Grossberg, S. (Ed.) (1987). The adaptive brain II: Vision, speech, language, and motor control. Amsterdam: North Holland. Read lightly Sections 1 to 34, pages 625-647, and the commentaries and reply to the commentaries of Grimson and Stevens. Note that the Commentary section appears only in the journal article and is not reprinted in the book. Yes, parts of this reading have been assigned before.
DeAngelis GC Seeing in three dimensions: the neurophysiology of stereopsis TRENDS COGN SCI 4: (3) 80-90 MAR 2000. PDF.
Blake R, Wilson HR. Neural models of stereoscopic vision. Trends in Neurosciences, 1991 Oct;14(10):445-52. PDF.
Supplementary Reading:
• Grossberg, S. (1993) A solution of the figure-ground problem for biological vision. Neural Networks, 6, 463-493. PDF.
• Nakayama K, & Shimojo S. (1990). da Vinci stereopsis: depth and subjective occluding contours from unpaired image points. Vision Res. 30(11): 1811-25. This paper a good example of how psychophysical techniques can constrain the search for physiological substrates.
• I.P. Howard & B. J. Rogers, Seeing in depth: Vol. II. Depth perception. Toronto: Porteous Publisher. A truly comprehensive treatment.
• Tyler, C.W. and Kontsevich, L.L. (1995). Mechanisms of stereoscopic processing: stereoattention and surface perception in depth reconstruction. Perception, vol. 24, pp. 127-153. Introduces the “attentional shroud” that we see in Week 13, and more! PDF.
<<Week 9 readings continue on the next page >
Week 12 (Continued)
Blake, R. (1989) A neural theory of binocular rivalry. Psychological Review, 96(1), 145-167. Read for gist.
Nakayama, K. Shimojo, S. and Ramachandran, V.S. (1990) Transparency: relation to depth, subjective contours, luminance, and neon color spreading. Perception, 19, 497-513.
Dev, P. (1975). Perception of depth surfaces in random-dot stereograms: A neural model. International Journal of Man-Machine Studies, 7, 511-528.
Marr, D. (1982). Vision. New York, W.H. Freeman. Pages 111-158.
Sperling, G. (1981). Mathematical models of binocular vision. In S. Grossberg, Ed. Mathematical Psychology and Psychophysiology. Hillsdale, NJ, Earlbaum, 281-300.
Prazdny, K. (1985). Detection of binocular disparities. Biological Cybernetics, 52, 93-99. Also in M. A. Fischler and O. Firschein, Eds., Readings in computer vision. 1987, Los Altos, CA, Morgan Kaufman, 73-79.
Prazdny, K. (1985). On the disparity gradient limit for binocular fusion. Perception and Psychophysics, 37 (1), 81-83.
Grossberg, S. and Marshall, J. (1989). Stereo boundary fusion by cortical complex cells: A system of maps, filters, and feedback networks for multiplexing distributed data, Neural Networks, 2, 29- 51.
Yeshurun, Y. and Schwartz, E. L. (1987). An ocular dominance column map as a data structure for stereo segmentation. Proceedings of the IEEE First International Conference on Neural Networks, San Diego, CA.
Grossberg, S. (1987). Cortical dynamics of three-dimensional form, color, and brightness perception: II. Binocular theory. Perception & psychophysics, 41(2), 117-158. Reprinted as Chapter 2 of Neural Networks and Natural Intelligence. Cambridge, MA: MIT Press. Sections 1-9 contain some useful background material.
BONUS Reading:
Schwartz, E. (Ed.) (1990). Computational Neuroscience. Cambridge, MA: MIT Press.
Introduction, ix-xiii. Schwartz offers unvarnished statements of contrasting presuppositional attitudes.
Week 13: Visual attention, pop-out, and search
1) Facets of attention: Bottom-up and top-down
2) Feature integration theory
3) Search rate asymmetries: Pop-out and slow search
4) Guided search and iconic bottleneck
5) Surfaces and features
6) Attentional modulation of receptive fields
7) Change blindness
Required Reading:
PALMER. Ch 11.
Kastner, S. and Ungerleider, L. G. (2000) Mechanisms of visual attention in the human cortex. Annu. Rev. Neurosci. 23:315-341. Hypertext.
Yantis, Chapter 22. Moran, J., & Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Science, 229, 782-784.
Itti, L. Koch, C. and Niebur, E. (1998). A model of saliency-based visual attention for Rapid Scene Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20(11), 1254-1259. PDF.
Serre, T., Oliva, A. and Poggio, T. (2007). A feedforward architecture accountsfor rapid categorization. PNAS, vol. 104, pp6424-6429. PDF.
Supplementary Reading:
• Duncan J, Humphreys G, Ward R Competitive brain activity in visual attention CURR OPIN NEUROBIOL 7: (2) 255-261 APR 1997. PDF.
Grossberg, S., Mingolla, E. & Ross, W. (1994). A neural theory of attentive visual search: Interactions of visual, spatial, and object representations. Psychological Review, 101(3), 470-489.
Eriksen, C. W. (1990) Attentional search of the visual field. In Visual search, D. Brogan, Ed. London: Taylor and Francis.
Duncan, J. (1995). Target and nontarget grouping in visual search. Perception & Psychophysics, 57, 117-120.
Wolfe, J. M. (1994). Guided Search 2.0: A revised model of visual search. Psychonomic Bulletin and Review, 1(2), 202-238.
Moran, J. & Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Science, 229, 782-784. Reprinted as Ch. 26 of K & A.
Crick, F. (1984). Function of the thalamic reticular complex: The searchlight hypothesis. Proceedings of the National Academy of Science USA, 81, 4586-4590. Reprinted as Ch. 28 of K & A.
Week 14: The great beyond
1) Over
2) Under
3) Sideways
4) Down
Hochstein S & Ahissar M. (2002). View from the top: hierarchies and reverse hierarchies in the visual system. Neuron 5;36(5):791-804. PDF.
Bar, M. (2004) Visual objects in context. Nature Reviews Neuroscience 5, 617-629. PDF.
Pollen D.A. (1999) On the neural correlates of visual perception. Cerebral Cortex,9(1):4-19.
Wolfe JM, Horowitz TS. (2004) What attributes guide the deployment of visual attention and how do they do it? Nat Rev Neurosci. Jun;5(6):495-501. pdf
He S, Cavanagh P, Intriligator J. (1996). Attentional resolution and the locus of visual awareness. Nature. Sep 26;383(6598):334-7. PDF. Arash Y. says: The experiment in the paper is doable by yourself when you read the paper, and the explanation seems exciting, however it worth trying a simpler null hypothesis to interpret the result.
Yantis, all the other chapters . . .