The following content is provided by MIT OpenCourseWare
under a Creative Commons license.
Additional information about our license and MIT
OpenCourseWare in general is available at ocw.mit.edu.
PROFESSOR: What I'm going to do for the next three lectures
is to talk about how we see.
What I'm going to do today is talk about how we take
information in.
Next Tuesday, I'll talk about the fact that we take in too
much information -- more information than we can use
and we have to select amongst that information.
On Thursday of next week, what I'll talk about is the fact
that perception is never just a direct, somehow registration
of the outside world.
It's always an active interpretation of the world,
and I'll talk about how we go about interpreting the
information that we've gotten from the outside.
I will be barely able to scratch the
surface of the topic.
I will abandon all discussion of the other senses, for
instance, which is a great pity.
Now, it's true that I can barely scratch the surface of
almost any topic in this course, but here the danger is
greater because this is actually
what I do for a living.
So, on any little bit of this, I can talk forever.
So, exactly how much of the surface I scratch will be
determined by how far afield I end up going.
This slide is up there to illustrate, in some sense,
those three bits.
Let's see, if I take the stage, chalkboard light off --
there we go, you want to go off?
Then you can see a little bit better.
You can sort of see stuff.
That's this Bruegel's painting of the battle between the good
angels and the bad angels.
For me, the purpose is to say look, it's clear that you see
something, it's clear that what you are seeing is an
active interpretation.
In fact, you may not have particularly known that it was
a battle between good angels and bad angels
before you look at it.
It's also clear that even though you, in some sense, see
the whole thing, you're not processing it fully, at least
not all at once.
So, for instance, you may not have particularly noticed --
I don't know what he is, he's a really cool beast --
I'm not sure if that's a bad angel falling or something.
But you probably hadn't particularly
noticed this fish guy.
Actually, in this picture you're probably still not
noticing the fish guy -- he just looks like a blob.
It's a great picture, look at it sometime.
And this guy's got very bad stomach problems, whatever
he's doing.
OK.
Rather than continuing to try to look at a murky picture,
let's go to a less murky picture and let's start by
talking about how you actually get information in from the
visual system, which obviously involves
going through the eye.
On the handout, which -- do I have a copy of
it here, yes, OK.
A lot of terminology.
Are you responsible for it?
Yes, of course.
If you miss some little bit of terminology, that's not going
to sink you in the course, either on this topic or on any
other topic.
You want to put your effort into thinking about the
broader topic, but it's useful in being able to talk
intelligently about it to get a grip on some of this
terminology, and hence, the tour of the visual system that
I will now embark on.
From the outside-in, the purpose of the eye, or at
least the front half of the eye is to form an image on the
back half of the eye.
So all of this, sometimes called the anterior segment of
the eye, is there to form an image, like a camera would, on
the back surface of the eye.
Most of the optical power of the eye is at the very front
surface of it.
That front surface is called the cornea.
It has, apart from its virtues as a lens focusing light, it
is also one of the two surfaces in the body most
sensitive to pain.
You may try this out if you want at home.
I don't recommend it, but if you scratch your cornea you
will know it and you will be happy.
What's the other one?
What's the other surface, do you think?
Anybody care to venture a guess?
Thumbs?
No, that's a novel thought.
No, look --
he's thinking like the cartoons, right, you hit the
thumb with the hammer and ow, it hurts.
Let me tell you, if you hit your cornea with the hammer it
hurts a lot more.
Yes?
AUDIENCE: Some place inside the mouth?
PROFESSOR: Some place inside the mouth.
No.
Again, you think about that inside the mouth is a place
where pain happens, but that's because you do stupid things
with your mouth, like hey, that pizza looks good.
Ow.
You would never stick the pizza on your eyeball.
So, again, nicely innervated for pain but not particularly.
Not the other great location.
I'll take one more here.
AUDIENCE: [INAUDIBLE]
at the bottom of your foot?
PROFESSOR: Yeah, yeah, same sort of thing, right.
You step on nails or something.
The bottom of your foot's a good place.
There's two people here, I've just gotta take their answers
because they're so eager.
AUDIENCE: Yeah, that's what I was going to say, somewhere
inside your ear.
PROFESSOR: Somewhere inside your ear is the point, not the
chunk that so many people are thrilled with.
Oh, let's put a hole in this -- oh, that was
fun, let's do it again.
That obviously hurts, I guess.
I've never gone in for this myself.
One of the reasons your mother said don't put anything inside
your ear that's bigger than your elbow is because if you
put something small and pointy in there, it eventually
reaches your eardrum and that's the other surface.
When you had an ear infection as a kid and the pressure from
your middle ear, the fluid in your ear, stretched that, it
hurt, and that hurts a lot.
So, don't put the hot pizza in your ear either.
OK.
The cornea is just a relatively thin layer.
It's held in shape by fluid, a watery-like fluid, which is
known technically as the aqueous humor -- humor in this
case has nothing to do with being funny.
It's an old word for a fluid, It forms a space that
separates the cornea from the iris.
The iris is the bit that gives your eye its color, but it's
basically a sphincter of muscle with a hole in it, and
that hold is the pupil of the eye, which is the aperture of
the camera.
That's where light's getting in from the outside into the
inner portions of the eye.
OK.
The pupil changes size, because you've got this
sphincter of muscle that either relaxes or contracts.
What makes it change size?
AUDIENCE: Light.
PROFESSOR: Light.
We know that.
More light makes it do what?
AUDIENCE: Get smaller.
PROFESSOR: Get smaller.
Why?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Yeah, so there's less light.
Sure, it's going to be less light, but
why do you have this?
What did they teach you in 3rd grade?
AUDIENCE: [INAUDIBLE].
PROFESSOR: Never stick your elbow in your eye.
Yes, I know that.
No, I need a hand here.
There's too many mutterings.
There's a hand.
AUDIENCE: Because you don't want to go blind.
PROFESSOR: Because you don't want to go blind.
Why would you go blind?
AUDIENCE: There's too much light.
PROFESSOR: There's too much light.
See, that's what they teach you in 3rd grade.
It's bogus.
It sounds good and it seems to make sense, but ask yourself
what level of protection that pupil can give you.
Right, so how much light is it going to cut out?
The answer is it cuts out a bit more than a factor of 10.
From the largest it can be to the smallest it'll be if you
stare at -- you know, if you're out in
really bright light.
So it can take it can change light levels by
about a factor of 10.
If you go from in here right now -- actually, it's fairly
dim in here --
but if you go from here to just go outside into the
courtyard, the change in light level is going to be on the
order of a factor of 1,000 right there.
The change in light level that your visual system can deal
with from, say, a moonlit night to bright sun on a snow
field is on the order of 10 to the twelfth or 10 to the
thirteenth.
So there's no way that the need to protect yourself is
going to be taken care of by the pupil of the eye.
There's just not enough range there.
The pupillary constriction, what it's probably doing for
you, those of you who are photographers, is it's
increasing the depth of field on your camera.
A pinhole camera will focus many different
distances at once.
A larger aperature will only focus one plane at one time.
And so if you've got lots of light, you crank down the
pupil in order to be able to focus -- in order to get
better focus, basically.
But the protection thing just doesn't --
it's not a stupid answer because it's what everybody
believes, but it doesn't turn out to be a correct answer.
Oh, now there's several other things that will change the
size of your pupil, but one other of interest, what else
changes the size of your pupil?
Anybody know anything else that they
can change the size--?
Yup?
AUDIENCE: Looking all the way down there at you.
PROFESSOR: Looking all the way down there at me.
Well, yes, but for reasons that are--.
Why?
Well, no, well you're changing --
sorry, that's the next bullet here.
What you're changing when you change focus of the eye is
you're changing the shape of the lens -- at least you are.
I'm not because at my age the lens has basically become a
rigid lump, and so when I want to look from there to here, I
use an external lens.
Yes, back to the pupil.
AUDIENCE: When you lie.
PROFESSOR: When you lie.
Yes, that will work a little bit maybe, not reliably enough
to -- you know, the CIA or something every now and then
gets the idea that that really works.
It's on the right general track.
Yeah?
AUDIENCE: Some drugs do that?
PROFESSOR: Yup.
Some drugs will certainly modulate it.
All right, [UNINTELLIGIBLE].
AUDIENCE: When you look at the object of your affection.
PROFESSOR: There you go.
That's particularly the one I was fishing for.
Your level of arousal will change the size of your pupil.
Notably, if you look at something that you like or
someone that you like, your pupils get bigger.
Obviously, not everybody here knew that explicitly, but you
sort of knew that implicitly, because the result of that is
that you think that things with big pupils are cuter than
things a little pinhole pupils, right.
That's why sappy cartoons or cards have big pupils.
It is also why in years gone by, women would put a drug
known as atropine or known as a cosmetic as belladonna for
pretty woman, pretty girl.
In their eyes what it did was it paralyzed and relaxed the
muscles of the irish and made a great big pupil.
It didn't do anything good for their vision because it made
things kind of blurry.
It also paralyzing the lens.
But it gave them great big pupils, and so they're looking
at the guy who may or may not be the object of their desire
-- they're looking at some guy, they can't see him so
well, which who knows, that may have helped.
But he's looking at her thinking implicitly her pupils
are really big, I know what that means.
Well, it turns out, following up on whoever it was over
there, it actually means she's on drugs.
In any case, pupil size is modulated by emotional state
and it's one of these things that we sort of knew
implicitly, people know implicitly, and think big
pupils are attractive for that reason.
It's only when you show up here you find out these things
explicitly.
All right, I've already said something about the lens.
The most notable thing about the lens is it gets harder
with age and --
you could try this out.
In fact, it's a good time in your life to start this
experiment.
Take something like the handout and ask yourself how
close you can get and have the letters
still be in sharp focus.
This is so sad it like makes me weep.
But I know you're getting there, too.
Now what you can do --
you could actually, if you're compulsive about these sort of
things, write it down somewhere and
just check every year.
This is one of those things where the signs of aging start
showing up now.
You will be able to see, even during the course of your
twenties that that so-called near point is
moving further out.
For me without my glasses we're out here somewhere.
And eventually you're locked at one plane of focus, and
depending on where that is, determines which particular
set of glasses you need -- do you need glasses to read, do
you need bifocals, whatever.
It's because you've lost the ability to
fine-tune that focus.
Now, you need something to hold the rest
of the eye in shape.
What you've got in there is it's really a jelly called the
vitreous humor for the glass-like substance.
For our purposes it's not desperately interesting, it's
there to hold your eye out so that the front end of the eye
can make an image on this back surface.
The back surface of the eye where the transduction from
photons, the light energy, into signals in the nervous
system where that takes place, that tissue is known as the
retina -- the word comes from the word for a net.
The fovea is the point --
fovea comes the word for pit --
it's the point on the retina where you have the highest
visual aquity, the greatest ability to
resolve small details.
You don't particularly realize in the course of your normal
day-to-day existence just how bad vision gets
away from the fovea.
You have this notion of a highly detailed world that's
in focus all over the place.
But that, to jump ahead to the third lecture in this series,
is a construction.
It's not reality.
And that you can prove to yourself.
Take the handout again, hold it at some comfortable reading
distance, fixate on the l on that second line there where
it says Lecture 4.
If you're fixating on the l, the l is forming an image on
your fovea.
Ask yourself while you're fixated on the l, how many
letters off to the right you can actually read.
And the answer is going to be not many.
If you think you can read to the end of line it's because
you're not fixating --
you're moving your eyes around.
Again, like the pupil thing, this is something you knew
implicitly.
Nobody sits there and says I'm going to read this handout.
I can see it, it's there, but I can't read it because the
outside of the fovea, I simply don't have the resolving
ability to do it.
Why not?
Well, I can give you at least one reason.
You have about a hundred million
photoreceptors in your retina.
Photoreceptors are the cells that convert light energy into
a signal in the nervous system, they're the
transduction elements.
You have only about a quarter of a million to a million
axons, nerve fibers running from the eye to the brain.
So that hundred million has got to get boiled down to the
much smaller number that you've got available to you.
So that means you can't treat each photoreceptor as a pixel
with a private line to the brain.
You do have something approximating this in the
fovea where you have fine detailed resolution.
But in the periphery, away from the fovea, what you have
is many, many, many, many photoreceptors that are all
ganged together to send their signal off to the central
nervous system.
So you lose resolution out there in the periphery.
Let's say a little more about the details of that retina.
Oh, the optic nerve is the bundle of fibers going off
from the eye to the brain.
I'll say a bit more about that in a minute or so.
A quick tour of the retina itself.
Up at the top there you've got the photoreceptors, these
transduction elements.
There are two flavors of them called rods and cones -- named
rods and cones because under the microscope rods are
rod-shaped and cones are cone-shaped.
They are different in their functions and capabilities.
Cones operate in bright light at daylight levels.
They do mediate color vision and they are
concentrated at the fovea.
They are most densely represented at the fovea.
Rods work in dim light levels.
They are not able to mediate color vision, which is why on
a moonlit night you don't see colors -- it's not because
color's somehow drained out of the world.
It's because the visual system that's looking at that world
can no longer analyze the input for color information,
but only for grey scaled information.
Rod photoreceptors are concentrated away
from the retina --
I'm sorry, away from the fovea.
They are, in fact, absent from the central fovea.
The best place to see this actually is if your room gets
dim enough at night that you can't see color, you're
working in the rod realm, and then you will discover that if
you fixate on a dim spot of light you can't see it.
That there might be something out here -- you notice it and
then you move your eye to look at it and it'll disappear,
because the image, if it's small enough, will now fall
into the fovea and there are no rods there,
you can't see it.
This is why if you're a star gazer and you're looking at a
dim star, you want to look at that star out of the so-called
corner of your eye.
The corner of your eye is about 20 degrees away from
straight ahead because that's where the density of rod
photoreceptors turns out to be greatest.
I was going to say something else about rod photoreceptors,
but that has disappeared from my mind.
So I will say that the rest of this is a collection of nerve
cells, of neurons.
It's really a little chunk of brain.
Developmentally the retina is a chunk of brain that's pushed
out into the eye, and this is the start of the visual
nervous system.
For present purposes it's worth thinking about two
aspects of this organization.
One is a through path that runs from the photoreceptors
through so-called bipolar cells to ganglion cells.
There's sort of a vertical path in this picture heading
from the photoreceptors out to the brain.
Then there are cells whose processes are sitting here and
here making horizontal connections so that
photoreceptors don't behave like isolated pixels so that
they can talk to each other, and so you can start doing
some sort calculations, if you like.
Those are cells like horizontal cells that have
their processes up here, and amacrine cells that have their
processes down here.
So those are things making horizontal connections across
the retina.
Photoreceptors, bipolar cells, ganglion cells are forming
this through pathway going from eye to brain.
Now, if you were to look at this and I was to say where is
light coming from in this picture, the reasonable
answer would be?
AUDIENCE: [INAUDIBLE].
PROFESSOR: It would seem like a reasonable answer would be
like it's coming from the top hit in the photoreceptors and
heading off to the brain and out the bottom.
But in fact, light's coming from there.
The retina is put on backwards in an interesting kind of a
way, or intuitively backwards at least, so that light shines
through all of this stuff before actually getting to the
photoreceptors.
Why is that?
Well, there are a variety of possible answers.
One of the answers is -- let's see, the remaining term on the
list that I haven't mentioned in that retina list, which is
actually at the top ofo the list, it says pigment
epithelium.
Up here there would be a layer of what boils down to black
gunk coating the back surface of the eye, and the
photoreceptors are stuck into that.
That's the pigment epithelium.
What it's there for is -- well, let's suppose that all
of you guys are photoreceptors and here I am, I'm a photon
and I'm part of this image of the world on the outside and
I'm coming flying in, and with luck I'll be absorbed by Mr.
Photoreceptor here.
But suppose I don't?
Suppose I hit over here.
Well, photons don't have much conscious life
as far as we know.
You, the owner of this visual system, don't want that photon
bouncing around anywhere else.
You want it either to be absorbed by a photographers or
to be gone.
Because if it bounces and hits up there somewhere it's going
to degrade the image, right.
It's going to be like haze across the image if it's not
landing where it should to make a nice starp image.
So this black gunk is there to soak up extra photoreceptors.
A very good idea if you are a diurnal animal, an animal who
works out in bright light.
Not such a good idea if you're an animal who is nocturnal or
running around in dim light.
There what you want to do is catch every photon that you
can possibly catch.
So if you're a cat, for example, you do this
differently.
Instead of a pigment epithelium at the back of your
eye, you have a structure known as the tapetum --
it's actually very beautiful, it looks like Mother of Pearl,
but you only get to see that if you take it out of the cat,
which makes the cat less beautiful.
But it's a reflective surface.
This is why if you get a cat in your flashlight beam, its
eyes seem to glow at you, because the light comes in --
if it doesn't get absorbed by, say, the photoreceptors, it
can bounce and an amount of it will bounce back out.
Some amount of light will bounce back out of your eyes.
That's what you see as red eye in an unlucky
photograph, for example.
Is that a hand?
AUDIENCE: Wouldn't that distort the image?
PROFESSOR: To the cat.
AUDIENCE: Yeah.
PROFESSOR: Yeah, absolutely.
That's why you almost never see cats reading
The New York Times.
No, cat visual aquity is pretty lousy.
A cat has different visual desires than you and I do.
The cat's not big on sweating the visual aquity thing.
The cat wants to know, did it move?
Can I jump on it?
Can I bite it?
Is it another cat?
Can we do stuff?
All this sort of stuff.
This sort of thing doesn't require a lot of visual aquity
and the cat wants to be able to do it in dim light.
So, we have this notion that cats can see in the dark,
which, of course, is not really true --
no light, no vision.
People think cats can see in the dark because cats can see
stuff in dim light that you and I can't, and part of the
reason for that is that they are willing to let stray
photons go where they will.
Because, in fact, I gave a sort of a radical example.
If my hypothetical photon missed my hypothetical
photoreceptor here, odds are it's not going to go out to
outer Mongolia, it's going to get absorbed someplace nearby,
and so the effect will be a blurring, not a complete
degradation of the image.
I think the cat can cope.
The cat doesn't mind too much.
By the way, if you're interested in design, taking a
look at the design of eyes across the animal kingdom is a
beautiful--.
The number of different ways that evolution or the hand of
God, you can take your pick, has chosen to solve the
problem of collecting light, an amazing array of
possibilities.
Once you start --
cats are one thing, but you go off into insects and other
beasties and it's great.
Wonderful stuff.
If you want to be an engineer, probably lots of good
industrial design there.
All right, let me take a quick trip up into the rest of the
visual system here.
This picture that's up on the screen now is mimicking the
picture on the upper left of the
second page of the handout.
It's showing the connections -- oh, I just remembered.
I want to do a quick demo here before I do that.
Let's go back and do that.
If the retina is on backwards, you got
this interesting problem.
So, there are those nerve fibers to the optic nerve.
How are they going to get out of the eye if light is coming
from there?
What they do is they run across the surface of the
retina until they reach the optic nerve and then they go
out up to the brain.
Where are the photoreceptors at the optic nerve?
The answer is there aren't any photo receptors
at the optic nerve.
If there aren't any photoreceptors at the optic
nerve, what do you see there?
The answer isn't nothing because you're not aware of
the fact that there's some big hole in your visual field.
But there is.
You will automatically fill it in because if it's green here
and green here and there's no information there, it's a good
bet that it's probably green all the way across and your
visual system can figure this out.
But it's worth it to prove it to yourself that there is a
blind spot there.
Let's see, on the handout --
well, flip to the back of the handout where you got a little
extra white space, and put an x on handout here like this,
and a little ways away like about like that, put a little
black dot or a little dot --
I don't care if it's black.
So the dot is to the right of the x.
Close the left eye, cover your left eye.
Look at the x, hold it so that the dot is straight out to the
right and then move the paper back and forth and you will
find that there's a spot where the dot completely disappears.
What you've done at that point is move the dot into the optic
nerve head -- the place where the optic nerve
is leaving the eye.
If you're saying I can't do this, I don't see a blind
spot, it's not because you're special and don't have one,
it's because you're doing it wrong.
Well, there is an alternative.
I'm given to understand that the octopus retina is not put
on backwards in this way and so perhaps you're an octopus.
OK, let's go back to this wiring diagram here.
You can play with this later.
It's cool.
Do things like -- well, if you flip it around you can find
that you got one in the other eye, too.
You can also do things like draw a line with a hole in it.
If you draw a line with a gap in it and put the gap in the
hole, the line will appear to complete.
If you get really adept at this you can do what King
Charles the Second of England was reputed to have liked to
do, which is you can look at your friends and put their
heads in blind spots and watch their heads disappear.
Charles the Second didn't get to chop off as many heads as
his predecessors had, so this was the best
he could do I guess.
All right.
How do the axons from the optic nerve feed the visual
centers of the brain?
You will recall, from the first lecture perhaps, that
everything from the right side of my body on the skin senses
gets represented in the sensory homunculus on the
left, and everything from the left in the sensory homunculus
on the right.
The way to not do this individual system would be to
have let's have everything from the right, that's my
right eye, my right eye go to my left hemisphere, and
everything from the left eye go to the right hemisphere.
That wouldn't be any good.
Isaac Newton figured out without bothering to do any
anatomy, Isaac Newton figured out that that
wouldn't be any good.
Because I want to look out at the world with two eyes and I
want to see one world.
So I'm going to have to do something that brings the
information from the two eyes together.
What you actually do is the following.
Everything from the left side of each retina ends up in the
left hemisphere, and everything from the right side
of each retina ends up in the right hemisphere.
The image on eye is flipped --
simple lenses do that, right.
So I've got a simple lens in that anterior
portion of my eye.
That means that the image of you guys landing on my retain
has your feet up, your head down and
left and right reversed.
So what that means here is if I stare at the person wearing
yellow here --
if I stare at the person wearing yellow she'll look
embarrassed.
It always works but that's a different phenomenon.
All right, but I'm staring at you but I'm not actually
paying any attention to you.
What I'm actually doing is looking at --
my aquity's so bad, it's a black blob right about there.
I assume she's a person --
yeah, she's a person, OK.
Anyway, acquity's very bad.
If you're on my fovea, the woman to the right is in the
right half of my visual field landing on the
left side of my retina.
She lands on the left side in both eyes, and as a result,
she's landing in my left hemisphere.
If I'm looking at you, some arbitrary blob -- there's a
blob that just moved over there.
Oh, she's a person, too.
This person in the left visual field ends up on the right
retain in each eye and in the right hemisphere.
So there's only one of her, in a sense, up in my brain, and
only one of her up in my brain and they're
getting together in that--.
And the result is that half the fibers from each retina
have to cross to the other side.
That's what this cute little map on page 2 is telling you.
But the important thing not to get confused about is don't
tell us on some exam that your left eye goes to your right
hemisphere or something.
It's left half of each retina goes to left hemisphere, right
half of each retina goes to the right hemisphere.
What it does once it gets there is -- this is a monkey
brain, and what you're looking at is the
cortical surface here.
That must be the outside surface and this is the inner
surface of the hemisphere, if there are two hemispheres.
The one on the bottom is looking at the inside surface.
The colored areas are ones that are important in vision.
I talked at the beginning about the fact that primary
visual cortex is right here at the back of the brain, but
it's amazing how much of the brain is devoted to vision.
It's very hard to look at in a picture -- well, it's very
hard look at, period.
What vision researchers like to do is to make maps like
this where you take the cortical surface
and flatten it --
it's all this wrinkley involuted structure --
use a variety of techniques to flatten it out like a map.
Then you see that this is the piece that's at the back of
the head, the one primary visual cortex.
Then you've got all these gazillion other regions that
are important in visual processing, some of them for
rather precise sort of things.
Where did mt go?
I think right there is mt, for example, which is important in
motion processing.
Lots of work being done trying to figure out what different
bits of it do.
Look, let me give you a broad organizing scheme for how to
think about this.
It's not perfect but it ain't bad.
From primary visual cortex where very basic information
about spots and lines and primitive bits of motion gets
pulled out, more advanced processing can be thought of
as in two large streams.
One large stream going up towards the parietal lobe is
telling you things about where stuff is in the world.
How is the world laid out?
So, my notion that this space is laid out as a big slanted
plane in front of me is probably a product of this
sort of a pathway.
A second pathway going down into the temporal lobe is very
concerned with what stimuli it might be.
So there, if you're a monkey and I spear individual cells,
it might be where I would find a cell that was particularly
interested in a hand, a monkey hand, or a hairy leg or
something like that.
It turns out that the hairy part can be important.
Some of the work on things like monkey hand detectors was
being done by one of my professors at Princeton, and
actually my TA in intro psych, when I was an undergrad, was a
grad student who was working on this, and he reported to us
that he was recording --
you go spear a cell.
It's one thing if you're just trying to figure out if it
likes spots of light, but how do you figure out if the cell
likes something like a hand.
Well, they had in those days drawers full of stuff and you
waved all this stuff in front of the monkey and see what
makes the cell go.
These experiments in the good old days tended to run for
very long time, because this was an anesthetized monkey and
the monkey wasn't going to wake up from
this particular operation.
You ran the experiment for days and days and days and you
stayed up the whole time.
So, Bob Desimone, now a famous visual neuroscientist is up
late at night with this stupid monkey with his stupid cell
and the cell isn't doing anything.
He's waving everything at it.
So finally, he took off his pants, wrapped a towel around
his waist and did a little dance in front of the monkey.
The cell went nuts.
Being a good scientist, he then narrowed this down to the
notion that this was a cell that actually seem to be
interested in hairy legs, and the hairy part turned out to
be important.
I feel free to tell this story about Bob Desimone because Bob
Desimone is running around the country with a talk at the
moment that has a picture of me in it kissing a wombat.
Don't ask.
It all seems fair to me.
All right, so if you go up into, way down here in the
temporal lobe, you might find cells that respond only to
something like a hairy leg or a monkey hand or
something like that.
If you're hanging out back here in primary
visual cortex --
well, actually let's go back and hang out
in the retina even.
Look at that, we'll hang out right there.
If you're hanging out in the retina, if you like much
simpler needs than this hairy leg detector somewhere up in
the temporal lobe, I mentioned before that you might be
bringing together a whole bunch of photoreceptors
through, say, a bipolar cell to a ganglion cell, to bring
together a whole bunch of them to send a
signal off to the brain.
But what I didn't mention is that not all the
photoreceptors that are hooked up to stimulated ganglion cell
are going to excite that cell.
Let's draw some pictures over here.
What actually happens, and I'm just going to mimic this over
here, is that you'd have some photoreceptors who are set up
to excite a ganglion cell.
Then some other guys around them who would be set up to
inhibit it.
The result is that what this cell --
that's a one-dimensional slice and this would be a
two-dimensional picture -- the retina's after all a
two-dimensional surface.
So what you'd really get is some region of photoreceptors
that excite the cell, and some surrounding region where light
inhibits the subsequent ganglion cell.
Then light out here does nothing.
So this is the receptive field of this cell -- this is the
only part of the retina that this cell
over here cares about.
It's going to send its signal off to the brain.
What does this cell do?
Well, among the things that this cell can do is it can
tell you I am most excited by a spot
that's exacty this size.
If the spot gets any bigger it's going to encroach on this
inhibitory stuff and reduce my excitation.
If the spot's any smaller, I'm not going to get all of my
excitatory guys excited, and the response will get smaller.
So I'm most excited by a spot of exactly this size.
If I build receptors with different size centers, this
guy's only going to be excited by a little spot and I could
make one that would be excited by a bigger spot.
So, even at the level of the retina, you can start to get
cells that are telling you more than just light on, light
off, they're starting to tell you something about the size
of things in the visual field.
When you get up to the cortex, you find that now cells come
to be interested in things like lines -- not just a spot
but a line.
And this cell will way, I like lines in this orientation.
I don't like lines in this orientation.
In fact, I like lines in this orientation only if they move
this direction.
So you can imagine that what you start to do early in the
visual system is to pull out a whole bunch of little
features, if you like, that you can then use subsequently
to try to figure out is there a monkey hand here.
Is there a laptop computer or something like that.
Early in the system you've got basic features being pulled
out of the image.
Now, this presents a problem.
Suppose that what you want to do --
Let's start with orientation.
Suppose you want to decide that a line is a particular
orientation.
How good are you at that?
The answer turns out to be that you are good enough to be
able to tell roughly 1 degree increments.
How are you going to do that?
Well you could have a cell specifically designed to look
for zero degrees vertical, 1 degree, 2 degree, 3 degree, 4
degree, and you need one of those everywhere in the visual
field, and at all different sizes --
a little line that's vertical, a big line that's vertical.
You're going to need an awful lot of cells then.
That's not what we do.
What we seem to do all over the place in the visual
system, and in sensory systems in general, is to make
comparisons.
That comparisons are very powerful in this game.
So how do you decide that something's vertical?
To a first approximation what you do is you look at whether
or not the lift-tilted guys and the right-tilted guys, the
cells that like stuff --
the cells that look for something that's tilted a
little to the right, cells that are interested in stuff
that are tilted to the left.
Are they roughly equally excited?
If they're roughly equally excited in balance one with
the other, then odds are that you're looking
at a vertical stimulus.
OK?
If the right-tilted stuff is stronger than the left-tilted
stuff, so that balance gets tiled, you're going to say
that the stimulus is tilted.
It turns out that if you do something to make --
let's say you go off and you make the left-tilted stuff
weak by tiring it out -- how do you tire it out, you show
it a bunch of left, show it a bunch of left lines, that
makes this weaker.
Then something that's vertical's going to turn out
to look tilted to the --
you're not going to believe that.
Let me show this to you.
I'll show it to you in --
I'm going to jump to here.
Well, we'll jump here first.
OK.
Here are a very boring picture.
It will be even more boring if I turn out a little more of
the lights here.
OK.
How you know that something's white or grey or achromatic?
Again, in this case you've got color mechanisms that say how
red or green is this?
If the answer is well, it's balanced between red and
green, and you also ask how blue or yellow is this?
If it's not blue and it's not yellow and it's not red and
it's not green, well, you know what it's probably white.
OK, well, so here let's take this red/green thing and let's
make the green side weak by wearing it out.
We'll show you some green.
Now I'm going to show you something white -- the green
is weak, the red is strong, the balance goes this way, and
you think that something that was once white now
looks kind of green.
There, see, isn't that amazing?
They're not convinced.
The ones who are convinced are confused.
Stare at the black dot at the center here.
You just want to stare at the black dot and think about,
say, that red dot up at 12 o'clock.
What you're doing is you're tiring out the red side of
this red/green equation in this particular case, and just
keep staring there, and then I'm going to go boink.
So, the people who went "oh," presumably saw colors.
Let's do that again, here.
What you want to notice is that the color is this
opposite color, it's going the other direction.
So the red one will turn green, the yellow one will
turn bluish, the green one will turn reddish, right.
So stare at the center, boink.
It works.
That's what's known as a negative after image.
You can just toggle back and forth.
If you look at the black dot on the right and then move
your eyes to the left, you'll see the colors show up in the
left hand circles, because that after image is
essentially cooked onto your retina --
it's there in retinal coordinates and
moves with your eyes.
Alright, now let me illustrate to you that this is a
ubiquitous phenomenon that shows up for basic phenomena
all over the place by turning this back on.
Is that going to be enough light for this?
So, I won't show you the orientation one, even though I
managed to get a PhD studying it, but I'll show you the
motion equivalent of it.
Here's what you want to do.
You'll still see it over there, except for the guy
who's asleep behind the pillar, that's his tough luck.
So, look at this, and we're doing the same game.
We're just taking the game and transferring from the color
realm, or the orientation realm, down
to the motion realm.
So this is contracting, right?
Stare at the center, that's important.
So stare at the center.
Again, think about 12 o'clock.
All the contours are moving down.
You know that something's stationary because it's not
moving up, down, left or right.
But at 12 o'clock I'm wearing out your down detectors, and
at 6 o'clock I'm wearing out your up detectors and so on.
The consequences of this evil act of mine will be seen
vividly if, when I take it away, what you do
is look at my nose.
Of course, what really worries me here is several people's
pupils got much larger, which means they think I
look better that way.
Now, a word of warning.
Two words of warning.
This is the best demo in the whole class.
It's all downhill from here.
The other word of warning is it's a very salient demo, but
it's there for a purpose.
It's really lame on the mid-term or the final when
some motion -- this is called a motion after effect -- when
a motion after effect question shows up to say oh, that was
cool, man, his head was shrinking and stuff.
Because you don't get much points for that.
What you want to know is why this is actually interesting,
which I will reiterate by going the other direction
since it seems a pity not to, right?
OK, so again, the way you figure out that something is,
in this case, stationary is by making a set of comparisons,
and Iam using this clever little tool to systematically
distort the comparisons that you're making.
By the way, I'm also causing you to make one of these
inferences that we'll talk about next week.
By seeing all these various bits of odd motion, you are
making the conclusion that my head is shrinking or growing
or something like that.
So this is expanding, right, so this is the
amazing pinhead version.
Ready?
The original version of this, by the way, is known in the
literature as sometimes it's the waterfall illusion,
because it gets described multiple times in the
literature, but one of them is by a Scotsman named Adams who
was staring at a waterfall and then noticed that the rocks on
the side of the waterfall seem to be drifting up.
This is a known waterfall in a known spot in Scotland and
there's a group of vision researchers who meet there
every year and the entry requirement for joining this
particular elite group is a bottle of scotch of a variety
that they have not tried before, which I think helps
make the rocks move in all sorts of interesting ways.
They've never invited me so how would I know.
OK, this is what I'm going to do.
Let me follow up.
So you've seen a negative color after image.
You've seen a negative motion after effect.
I've asserted that these exist in orientation.
You can show this often -- you show somebody big stuff and
then medium stuff looks small.
It's ubiquitous.
It shows up all over the place.
I want to show you one more version of this kind of a
effect, and that's going to take a certain amount of
build-up here.
So the first thing to do is say the vertical and
horizontal stripes, do they look different to you?
If they look different somebody raise their hand and
tell me what the difference is.
What's the difference?
AUDIENCE: The horizontal ones are horizontal.
PROFESSOR: Yeah, alright.
In their color or shading in some fashion.
Yeah?
AUDIENCE: The vertical ones look blurry.
PROFESSOR: The vertical ones look a little -- they do look
a little blurry.
Yeah?
AUDIENCE: The vertical ones -- the black lines look like
they're thicker than the ones--.
PROFESSOR: Well, we're not getting a lot of vivid color
reports here.
AUDIENCE: The horizontal lines look brighter.
PROFESSOR: Brighter?
Nobody knows what the word color means.
Last try.
AUDIENCE: The horizontal lines are greyer and the vertical
lines are black.
PROFESSOR: OK, look, this makes the point just fine.
Never mind guys.
There is no in general consensus here.
I'm about to produce a general consensus.
That's the motivational part of today's lecture.
No, actually what I have to do now is for the next few
minutes what you want to do is keep your head upright.
You don't have to stare at any one spot.
You don't even have to your head upright, just keep it in
the same orientation.
If you flop all over the place and fall asleep
nothing good happens.
So you're going to look at these green vertical lines and
these red horizontal lines.
All I need to do here is toggle back and
forth between them.
Now you can see the negative after image here.
If you stare at this green -- oh, I should have done that
quickly before.
So stare at this green thing for a while.
You get an nice purply red thing?
OK.
So that's just the negative after image.
But we're not interested in the
negative after image anymore.
Actually, we are.
There's an aspect of the negative after image that's
really cool here, which you may start to notice shortly.
During the blank period, shout out the
orientation that you see.
AUDIENCE: [INAUDIBLE PHRASE]
PROFESSOR: That's a little weird.
Wouldn't you think?
Now that's a little weird because if you stare at these
horizontal stripes -- in fact, I should be able to get you
back to seeing horizontal.
So, stare at these --
fixate, don't move your eyes around and we'll just stare at
that for a minute or so.
Has it been long enough?
Yeah.
Does it look horizontal?
Still looks vertical, oh boy.
Well, here's the get rich quick part of this lecture,
assuming that you're unscrupulous, you can use this
to make your fortune claiming that you can teach people to
be psychic, right.
What's the next orientation I'm going to show?
If you can make your fortune out of that, just send me 10%.
But what's really going on here, what this is actually
illustrating, this aspect of it is illustrating, is that
you're seeing with your whole visual system, not with your
retina all by itself or something like that.
So, each time you look, say, the red horizontal, you're
building up a green horizontal negative after image on your
retina, right?
And this is building up a red vertical kind of after image
on your retina.
If you go back and forth, you build up sort of a plaid.
So think about your retina as having a plaid on it.
Now you don't see it here, but the plaid is --
when you're looking at a blank screen, it's just that plaid
that's getting shipped up to your brain, basically.
OK, well what else is happening?
Well, when you're staring at this green vertical thing, up
in your brain the cells that are interested in vertical are
saying we're getting tired.
Go to the blank part, tired vertical guys and more awake
horizontal guys are looking at this weak plaid and they're
only seeing the horizontal part.
Now you look at the horizontal, the horizontal
guys get tired out -- oh, we can only see the
vertical, and so on.
You may ask isn't it about time we saw something
interesting here?
The answer is undoubtedly so, but this particular effect
takes a few minutes to build up, and so you gotta keep
looking at this.
Oh, you've got to ask yourself what the effects ought to be.
What I'm building up here is an effect called the
McCullough Effect, named after Celeste McCullough, who unlike
most people who discover visual effects, was actually
looking for this.
She knew what she was looking for and she knew what she
could find.
Now I already showed you that the pattern you're going to
look at again, right, it was that black and white pattern
of vertical and horizontal lines.
What do you think the vertical lines should look like when
we're done?
They should look kind of reddish maybe.
The horizontal should look?
AUDIENCE: Green.
PROFESSOR: Green.
Let's see, well we might have done this long enough.
This is one of these things where if I do it long enough I
can hit absolutely everybody, and if I quit too soon not
everybody will see the effect.
All right, here we go.
So, close your eyes, because that will let me get to the
right slide, but it will also let you perhaps see that
checkerboard floating there in the dark?
Murmur if you can see a checkerboard
floating in the dark.
OK, good.
That's that's the negative after image thing.
OK, now take a look at this test pattern again.
How many people think that the vertical and horizontal look
different now in terms of their color?
How many people think it might look a little sort of maybe
kind of different, but it's not exactly
biting me on the posterier?
Alright, that's a better test.
Now, the way to find out whether you've actually got
the right effect is without knocking out your neighbor,
tilt your head 90 degrees.
So that's cooler.
I'll say a word more about this, but I won't say a word
more about this until you've had a chance to get up and
stretch for a second.
OK.
How many people when they look at these dots still see
reasonably compelling colors?
Not many.
How many people when they look at my head
still see it shrinking?
Yeah, except for the weird people.
That's because your visual system is continuously
re-generating itself.
You go stare at the red thing, you wear out, in a sense, the
red detecting chunk of your visual system, and then if you
flip your eyes over you find out that you see a
white thing is green.
But then you recover from it, right?
Good.
How many people still see this effect?
Tilt your head just to make sure if you think it's there.
A little weak today.
We probably should have gone a little bit longer.
Those of you who are seeing it, and perhaps even those of
you who are not --
did I put a test pattern on the handout, I forget?
No, OK.
I'll just bring the Powerpoint back next week.
Not only are you still seeing this some minutes afterwards,
but particularly if I don't show you the test pattern a
lot you may see it for days and weeks.
The longest report in the literature for the McCullough
effect is three months.
The only reason it's only three months is because at
that point the undergraduates, who, of course, were the
subjects, went away on vacation and
the experiment ended.
The evidence suggests that if we locked you in the dark that
the McCullough effect would last forever.
We won't lock you in the dark.
I teach this up at Harvard where there are many more
pre-law students, and the immediate reaction to this
news is half of them consider filing suit.
This will not, in fact, ruin your life in any fashion.
In fact, it'll only be noticeable if you happen to
end up looking at a test pattern like this or perhaps
serving time in the near future.
But this was something that McCullough was not expecting.
McCullough thought that what she had done here was, OK, you
can fatigue color, you can fatigue orientation, well I'll
fatigue the cells in the brain and there are cells in the
brain that are interested in red vertical lines.
They'll get tired, I'll see white vertical as green,
they'll recover, I'll see white vertical as white.
But what happened was you looked at red vertical lines
for a while, white vertical looked green and white
vertical looked green for like a really, really long time
thereafter.
What seems to be going on here is something like this.
Let's suppose that the way you decide that something is white
vertical is by looking at the balance of -- well here, let's
draw it this way.
You've got red vertical input and you've got green vertical
input, and they're being like let's put a little dial here.
If they're roughly equal you decide something looks white.
So I show you red vertical for a while and this thing gets
weak and cruddy, and now something that should be
producing equal input gets more from there and it ends up
looking a little green.
Alright, that's fine.
What happens if this cell gets sick -- it's not going to be
one cell, but suppose that this process gets weak in some
fashion, not because you were looking at some stimulus, but
because you went out and ate the mushroom that you
shouldn't have eaten or whatever and this cell just
took a hit.
And so now this is kind of a cruddy cell.
Well, there are a couple of possibilities.
One of them is that for the rest of time you're going to
end up seeing white things as looking green.
That's not going to be good right.
You don't want that happen.
So if you decided that this was the situation, what you
would want to do is change the gain on here.
So if this was previously just equal gain, let's crank this
up a little bit so that we can drag that
back up to being equal.
That sounds OK.
That seems like a clever enough thing to do.
But how are you gonna do that?
How are you going to figure out that you're broken.
How would you know that your visual system was not
reporting the truth to you.
Well, I mean one way would be you're saying hey, that looks
white to me, and somebody else says that looks green and they
laugh at you or something.
Nah, it doesn't work.
No evidence that that works.
What you do is that you seem to have some sort of notion of
the statistics of the world, the way the world ought to be.
If the world is systematically different from that, you draw
the conclusion that you've got trouble here of
some variety or other.
Well not necessary, you don't necessarily draw the -- let me
give you a non-troubled version of this.
How many people have been to one of these omni
theater kind of movies?
The main reason for going to an omni theater movie is what?
You want to feel like you're moving, right,
when you're not moving.
Because all the movies that they ever show you at the
science museum are things where you go driving around in
race cars, stuff like that.
Why does this work?
You're sitting at this huge screen, and everything in your
visual field is moving in the same way.
You know something about the world.
The world on average is stationary.
If the whole world is moving there's a good explanation for
that typically.
The answer is not the world, it's you.
What is it that causes the whole world to slide across
your retina?
This is what causes the whole world to
slide across the retina.
Sit in the omni and the world is sliding across your retina.
Your brain says uh-huh, the world isn't moving, I'm
moving, and I think I'll throw up now for a while or
something like that.
That's a separate interesting question.
What is it here that could tell you that
you're out of whack?
Well, what is the normal relationship between color and
orientation in the world?
Typically in the world if you see something that's vertical,
what do you know about its color?
That's right, nothing.
So what's going on here?
It's vertical.
It's green.
If it's horizontal it's red.
If it's vertical, it's green.
If it's horizontal, it's red.
You do this for long enough and a little chunk of your
brain, rather like the little chunk of your brain that's
saying I ate the cupcakes, I threw up, I'd better do
something about this.
This little chunk of your brain is saying, oh, there's a
correlation between color and orientation.
The only way that happens is if I'm broken, and the only
thing to do about that is to adjust the gain on chunks of
my visual system.
So, everything looks green.
All the vertical things look green.
Oh my goodness, what am I going to do?
Well I better crank down the gain on the green side, it's
obviously too high, and I'm want to crank up the gain on
the red side.
Unfortunately, that was a goof.
And it was wrong, so then when you look at a white thing it
now looks red because you cranked the gain up
too much over here.
How do you recover from this?
Well, if I now show you the opposite one -- what were we
doing before?
What was it, green vertical?
If I show you red verticals and green horizontals for a
while, that'll push it back the other way.
But that doesn't typically happen in the world.
Simply being exposed to a world where there's no
correlation will slowly push you back.
But the reason the effect lasts so long is that the
evidence from the real world is kind a weak compared to
this artificial situation that I cooked up.
The way I'm thinking about this is like --
here's another way of thinking about this.
Suppose you're measuring average weight of people.
I'm just going to keep weighing people and what's the
average weight of people.
It'll bounce around a little bit.
Alright.
This is how much people on average weigh.
Now what happens is the amazing 1,000 pound guy comes
into my experiment.
What happens to the average?
Now I start weighing normal people again.
Does it just go -- no.
Weighing regular people it'll take a comparatively long time
for me to get back to normal.
So that's what's going on here.
The McCullough effect adapting stimulus is the 1,000 pound
guy that's causing you to think that
your brain is broken.
You go and re-adjust your brain and now slowly it will
get better.
How's it look there?
Not better yet.
How many people have a nice after effect?
Still, nice McCullough effect?
Celeste McCullough would be so proud.
You will notice that I have not said a word about signal
detection theory.
I noticed that, too.
There is a chunk in [? Gleichman ?] about it.
I might decide to talk about it on Thursday.
Oh, you may also notice that you have lost your cell phone.
If that is true come and talk to me about it
because we found it.
One last thing.
How many people know what that is?
I didn't ask you what it was, I just wanted to know if you
knew what it was.
How many people recognize this?
Only a few.
OK, now you can shout out.
AUDIENCE: It's a dog.
PROFESSOR: It's a dog.
How many people does that help?
It's head is -- well, wait a second.
There's its head, foot, back paw, it's back, it's sort of
sniffing at a ground plane that's like here.
You can store that one away till next week and
we'll talk about it.