The following content is provided
by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license
and MIT OpenCourseWare in general
is available at ocw.mit.edu.
PROFESSOR: Let's get started.
Could I start first with an announcement of a talk
this afternoon.
I know your schedules are full, but --
the abstract for the talk I think will go up on the top
of our web page.
It's a whole range of other applications
that I would hope to get to, by an expert.
Professor Rannacher, his applications
include optimal control, which is certainly
a big area of optimization.
Actually, MathWorks, we think of them as doing linear algebra,
but their number one customer is the control theory world,
and it totally connects with everything we're doing.
He's also interested in adaptive meshing for finite element
or other methods -- how to refine the mesh where it pays
off in arrow problems, all sorts of problems.
The math behind it is this same saddle point structure,
same KK -- when he says KKT equations,
that's our two equations.
I've been calling them, sometimes, Kuhn-Tucker,
but after Kuhn and Tucker, it was
noticed that a graduate student named Karush, also a K,
had written a Masters thesis in which these important equations
appeared.
So now, they're often called KKT equations.
I think Rannacher will do that.
Anyway, I don't know if you're free,
but it'll be a full talk in this program of computational design
and optimization.
I don't know if you know MIT's new masters degree
program in CDO.
So it's mostly engineering, a little optimization
that's down in Operations Research,
and a couple of guys in math.
So that's a talk this afternoon, which
will be right on target for this area
and bring up applications that are highly important.
Well, so I thought today, my job is going
to be pretty straightforward.
I want to do now the continuous problem.
So I have functions as unknowns.
I have integrals as inner products.
But I still have a minimization problem.
So I have a minimization problem and I'll call,
it's often a potential energy, so
let me use P. Our unknown function is u,
so that's a function u of x; u of x, y; u of x, y, z.
And instead of inner products we have integrals,
so there's a c of x.
This is going to be a pure quadratic,
so it's going to lead me to a linear equation.
In between comes something important --
what people now call the weak form of the equation.
So, this is would be the simplest
example I could put forward of the calculus of variations.
So that's what we're talking about.
Calculus of variations.
What's the derivative of P with respect to u?
Somehow, that's what we have to find,
and we're going to set it to zero to minimize.
Then why do we know it's a minimum?
Well, that's always the second --
the quadratic terms here are going to be positive.
So we have a positive definite problem.
Positive definite means things go this way, convex,
and you locate the bottom, the minimum
is where the derivative is zero.
But what's the derivative?
That's the question.
It's not even called derivative in this subject,
it's called the first variation.
Instead of saying first derivative,
I'll say first variation.
And instead of writing dP/du, I'll write --
where can I write it?
So this is key word, so this is going
to be the first variation, and I'm
going to write it with a different d, a Greek delta,
dP/du.
It's just sort of a reminder that we're
dealing with functions and integrals of functions
and so on.
So it's just change the notation in the name
a little as a trigger to the memory.
But this board really has a lot --
not all the details are here, of course.
This is a summary of what the calculus of variations does.
So it takes a minimization problem.
So I'm looking for a function u of x.
And always since we're in continuous problems,
we have boundary conditions.
So as always, those could be -- let me imagine that those are
the boundary conditions.
So I would call those essential conditions.
Every function that's allowed into the minimum
has to satisfy the boundary condition.
So it's a minimum over all u with the boundary conditions,
with those boundary conditions.
There are two kinds of boundary conditions,
and maybe I'll postpone thinking about boundary conditions
till I get the equations, the differential
equation inside the interval.
So all these intervals go from zero to one.
I won't put that.
We're in 1D.
So, how do you find the function, u of x?
that stands for du/dx.
So the given data for the problem
is some load, some source term, f
of x, which is going to show up on the right-hand side,
and some coefficient c of x, which is going
to show up in the equation.
They depend on x, in general.
Many, many, many physical problems look like this.
Sort of steady-state problems, I would say.
I'm not talking about Navier-Stokes fluid flow
convection, I'm talking about static problems, first of all.
So here's the general idea of the calculus of variations.
It's the same as the general idea of calculus.
How do you identify a minimum in calculus?
If the minimum is at u, you perturb it a little,
by some delta u, that I'm going to call v to have just one
letter instead of two here.
You say OK, if I look at that neighboring point, u plus delta
u, my quantity is bigger.
The minimum is at u.
So we're remembering calculus -- I guess I'm saying I've written
that here.
Compare u with u plus v, which you could
think of as u plus delta u.
It's like -- v you might think of as a small movement away
from the best function.
In calculus it's a small movement
away from the best point.
So let me draw the calculus.
If you think of this blackboard as being function space instead
of just a blackboard, then I'm doing calculus of variations.
But let me just do calculus here.
So there's the minimum, at u.
And here is u plus v near it.
Could be on this side or it could be on this side.
Those are both u plus v. Well, you maybe
want me to call one of them u minus v.
But the point is v could have either sign.
I'm looking at minimum sort of inside, where I can
go to the right or the left.
So what's the deal?
Well, that point is then that at that point or at that point
or at any of these other points, P of u plus v
is bigger then what it is at the minimum.
That tells us that u is the winner.
Now how do we get an equation out of that?
Calculus comes in now.
We expand this thing -- this is some small movement away from
u.
So we expand it, we look at the leading term -- well,
of course, the leading term is P of u.
Then what is the next term?
What's the first-order, first variation
in P when I vary u to u plus v?
Well, it's the whole point of calculus, actually.
The central point of calculus is that this is some function
that we call P prime of u times v, plus order of v squared.
When v is small, v squared is very small.
So what's our equation?
Well, if this has to be bigger than P of u --
I could just cancel P of u -- so this thing has to be bigger
equal zero now.
I've squeezed it in a corner, but since it's calculus
we kind of remember it.
Also, it's easy to learn calculus and forget
the main point.
So this has to be greater equal zero.
Now this is going to be -- if v is small,
this is going to be very small, so it won't help.
So this thing had better be zero, right?
That had better be zero.
Because if it isn't zero, I could take v of the right sign
to make it positive, and take v small, so
that this would dominate this.
Maybe I wanted to take v of the right sign
to make that negative anyway.
I need P prime of u to be zero.
So that's what I end up with, of course, as everybody knew.
That P prime of u had to be zero.
So in that tiny picture, I've remembered what we know.
Now let me come back to what we have
to do when we have functions.
So what happened?
I compare P of u -- u is now a function, u of x --
with P of u plus v. So I plug in u plus v,
I compare with what I get with only u,
and what's the difference?
I look at the difference and the difference
will have a linear term from u prime plus v prime squared
and then I'm going to take away the u prime squared because I
gotta compare the two.
So what's left?
There will be a 2 u prime, v prime.
I'm maybe just being lazy here.
I'm asking you to do it mentally and then
I'll do it a little better.
So the difference in this comparison is a 2 u
prime v prime times c, and the 2's cancel.
There's the difference right there.
What's the difference over in this term?
Well, I have that term and then I
have the same term with u plus v, and then when I subtract,
I just have the term with v.
So this is the dP/du that has to be zero for every v.
Now I'm really saying the important thing.
This weak form is like saying this
has to be zero for every v. Then, of course,
in this scalar case, it was like a very small step
to decide, well, if this is zero for every v,
then that's zero, which is the strong form.
Are you with me?
So we have a minimum problem, minimize P,
we have a weak form, the first variation,
the first derivative, the first-order term
has to be zero for every v. Then if it's zero for every v, that
forces the derivative to be zero and that's the strong form.
Now over here it took more space.
But the ultimate idea is the same.
We looked at P of u plus v compared
with P of u, subtracted, looked at the linear term
and there it is, the first variation.
That has to be zero for every v. I'll just
mentioned boundary conditions now,
as long as we're at this weak form.
Don't think of this weak form as just some mathematical nonsense
to get to the differential equation,
because the weak form is the foundation for the finite
element method -- all sorts of discrete methods,
discretization methods will begin with the weak form,
the weighted integral form, rather than the strong form.
I was just going to say a word about boundary conditions.
What are the boundary conditions on v?
So you could say well, v stands for a virtual displacement.
Virtual meaning kind of we just imagining it,
it's a displacement that we can imagine moving by that amount,
but the whole point is the nature fix the minimum.
What's the boundary condition?
Well, all the candidates have to satisfy these boundary
conditions.
So I have to have u plus v at zero also has to equal a,
and u plus v at 1 also has to equal b,
if I took these simple boundary conditions.
So, by subtraction, I learn the boundary conditions
on v. When I say all v, I mean v of zero has to be what?
Zero.
And v of 1 has to be zero -- when we had those boundary
conditions.
Different problems could bring different boundary conditions,
of course, but this is easier than -- simplest.
So when I say all v, I mean every function
that starts at zero and ends at zero
is a candidate in this weak form.
I have to get the answer zero for all those functions.
Now somehow, I want to get to this point,
the differential equation, and why don't we give it the name
that everybody -- the two guys' names, Euler-Lagrange -- well,
pretty famous names.
This is the Euler-Lagrange equation.
So maybe before where I said Kuhn-Tucker or something,
if I'm talking about differential equations,
I go back to these guys.
Now, how did I get from weak form to strong form?
That's a key.
If you see these two steps from the minimum principle
to the weak form, that's just, again, plug in u plus v,
subtract and take the linear part.
Then it's true for all v's.
Now, how do I get from here to here?
Notice that this form is an integrated form.
This form is at every point.
So it's much stronger and much more demanding.
You could say OK, that gives us the equation --
that's the equation as we usually see them.
I'll do 2D and that'll be Laplace's equation or somebody
else's equation, but minimal surface equation,
all sorts of equations.
Everything.
All sorts of applications including
this afternoon's lecture.
How to get from here to here?
Well, there's one trick in advanced calculus, actually.
The most important trick in advanced calculus
is integration by parts.
Well, we use those words, integration by parts,
in 1D here.
So I'll do that.
We use maybe somebody else's name --
Green's formula or Green's theorem,
or Green-Gauss or the divergence theorem or whatever,
in more dimensions.
But 1D, I just integrate by parts.
Do you remember how integration by parts goes?
I want to get v by itself, but I got v prime,
because when I plugged in u plus v here, out came --
it's the derivative so I've got a derivative.
So how do I get rid of a derivative?
Integrate by parts.
Take the derivative off of v -- can I just do that this quick
way?
Put the derivative onto -- I almost said onto u.
I'm doing integration by parts and saying how important it is
but not writing out every step.
So I take the derivative off of this and I put it onto this,
and it's gotta be -- and a minus sign appears when I do that.
Where the heck am I gonna put that minus sign?
Right in there.
Minus.
Then everybody knows that there's also a boundary term,
right?
So I have to squeeze somewhere in this boundary term.
So now that I've done an integration by parts,
the boundary term will be the v times the c u prime
at the ends of the interval.
I think that's right.
What we hope is that that goes away.
Well, of course, if it doesn't, then we have to --
that there's a good reason that we don't want it to, but here
it's nice if it goes away.
You see that it does, because we just
decided that the boundary conditions on v
were zero at both ends.
So, v being zero at both ends kills that term.
So now, do you see what I have here?
I could write it a little more cleanly.
The whole point is that the v -- I now have v there and I have v
there so I can factor v out of this.
Just put it there.
That was a good move.
This minus sign is still in here.
So I now have the integral of some function times v is zero.
That's what I'm looking for.
The integral of some function, some stuff, times v,
and v can be anything -- is zero.
What happens now?
This integral has to be zero for every v.
So if this stuff had a little bump up,
I could take a v to have the same bump
and the integral wouldn't be zero.
So this stuff can't bump up, it can't bump down,
it can't do anything.
It has to be zero and that's the strong form.
So the strong form is with this minus sign in there,
minus the derivative -- see, an extra derivative came onto
the c u prime because it came off the v,
and the f was just sitting there in the linear,
in the no derivative.
So, do you see that pattern?
You may have seen it before, but calculus variations
have sort of disappeared as a subject
to teach in advanced calculus.
It used to be here in courses that Professor Hildebrand
taught.
But actually it comes back because we so much need
the weak form in finite elements and other methods.
What I wrote over here is the discrete equivalent.
I can't resist looking at the matrix form, for two reasons.
First, it's simpler and it copies this.
Do you see how this is a copy of that?
That matrix form is supposed to be
exactly analogous to this continuous form.
Why is that?
Because u transpose f, that's an inner product of u with f --
that's what this is.
That integral.
A*u is u prime in this analogy.
So this is u prime times c times u prime with the 1/2,
and that, again, that transpose is telling us inner product
integral.
So if I forget u prime and think of it as a matrix problem,
that's my minimum problem for matrices.
I want to find an equation for the winning u.
In the end, this is going to be the equation.
That's the equation that minimizes that.
Half of 18.085 was about this problem.
Well, I concentrated in 18.085 on this one,
because minimum principles are just that little bit trickier,
so that's 18.086.
And then, in between, something people seldom
write about but, of course, it's going to work,
is that I change u to u plus v, I multiply it out, I subtract,
I look at the term linear in v, and that's it.
That would be if I make that just a minus sign
and put it all together the way I did here,
that's the same thing.
So this is the weak form.
This is the minimum form, this is the weak form,
and this is the strong form.
You see that weak form?
Somehow in the discrete case, it's pretty clear that --
let's see.
I could write this as -- you see, it's u transpose,
A transpose C*A*v equal f transpose v.
The conclusion is if this holds for every v,
then this is the same as this.
If two things have the same inner product with every vector
v, they're the same, and that's the strong form.
You'd have to transpose the whole thing, but no problem.
So now I guess I've tried to give the main sequence of logic
in the continuous case, and it's parallel
in the discrete case for this example.
For this specific example because it's the easiest.
Let me do what Euler and Lagrange did by extending
to a larger class of examples.
So now our minimization is still an integral --
I'll still stay in 1D, I'll still keep these boundary
conditions, but I'm going to allow some more general
expression here.
Instead of that pure quadratic, this could be whatever.
Now I'm going to do calculus of variations.
Still in 1D.
Calculus of variations, minimize the integral
of some function of u and u prime with the boundary
conditions, and I'll keep those nice.
So that integral's still 0 to 1 and I'll
keep these nice boundary conditions just
to make my life easy.
So that will lead to v of zero being zero, and v of 1
being zero.
What do I have to do?
Again, I have to plug in u plus v
and compare this result with the same thing having u plus v.
So essentially I've got to compare F
at u plus v, u plus u prime plus v prime with F of u and u
prime.
I have to find the leading term in the difference.
So I'll just find out leading term,
and then will come the integral.
But the first job is really the leading term,
and it's calculus, of course.
Now can we do that one?
It's pure calculus.
I have a function at two variables,
the function of u and u prime.
Actually, I did here.
I had a u prime there and a u there.
Once I write it down you're going to say sure, of course,
I knew that.
So I have a function of two variables
and I'm looking for a little change.
So a little change in the first argument produces the dF --
the derivative of F with respect to that first argument times
the delta u, which is what I'm calling v.
That's the part that the dependence on u is responsible
for.
Now there's also a dependence on u prime.
So I have the derivative of F with respect
to u prime times the little movement in u prime, which
is v prime.
I can't leave it with an equal there,
because that's only the linearized part,
but that's all I really care about.
This is order of v squared and v prime squared.
Higher order, which is not going to --
when I think of v as small, v prime as small,
then the linear part dominates.
So can you see what dP/du is now?
Now I've got to integrate.
I integrate this, that very same thing.
This -- dx.
That has to be zero for all v. You don't mind if I lazily
don't copy that into the -- that's the weak form.
This was the minimum form, now I've got to the weak form.
This is the first variation.
The integral of the change in F, which has two components, when
there's a little change in u.
I should be doing an example, but allow
me to just keep going here until we get to the strong form.
So this is the weak form for every v.
Let me just repeat that the weak form is quite important
because, in the finite element method,
we have the v's are the test functions,
and we discretize the v's -- you know,
we have a finite number of test functions.
Well, I can't go entirely -- I'll come back to finite
elements.
Let me stay with this continuous problem, calculus
of variations problem.
v is v of x here, and it satisfies these boundary
conditions.
That's the only requirement that we need to think about here.
What's the strong form?
Also called the Euler-Lagrange equation.
How do I get to that strong form?
How did I get to it before?
I would like to get this into something times v. Here
I've got something times v but I've also
got something times v prime.
I wanted to get it like up here where
it was times v. What do I do?
You know what I do.
There's only the one idea here.
Integrate by parts.
So I have the integral -- well, dF/du*v was no problem --
that had the v that I like.
But it's this other guy that has a v prime and I want v. So,
it doesn't take too much thinking.
Integrate by parts, take the derivative off of v,
get a minus sign, and put a derivative --
can I do it with a prime, but I'll do better below --
onto this, and then there's a boundary term,
but the boundary term goes away because of the boundary
condition.
So now I have the v. Can I make it on this board?
There's the dF*du multiplying v, and then there's the minus --
this is the derivative d by dx of dF / d u prime.
Now all that is multiplying v and giving me
the integral of zero.
I promise to write that bigger now.
But, again, the central point was
to get the linear term times v. That's always the main point.
Then what's the conclusion?
What's the Euler-Lagrange equation?
This integral is this quantity times v and v can be anything,
and I've got to get zero.
So, what's the equation?
That stuff in brackets is zero.
That's the Euler-Lagrange equation.
Finally, let me just write it down here.
Euler-Lagrange strong form.
In this general problem it would be dF/du minus d by dx of dF /
d u prime equals zero.
Would you like me to put on what a --
if F happened to depend on u double prime,
it would be a plus -- this would be what would happen --
just so you get the pattern -- equaling zero.
So I've gone one step further by allowing
f to depend on curvature, and writing down
what the resulting term would be in the Euler-Lagrange equation.
Could you figure out why it would be that?
Where would this thing have come from?
It would have come from -- there would have been a dF / d u
double prime times v double prime in the weak form.
Then I would have done two integrations by parts --
two minus signs making a plus, two derivatives moving off of v
and onto the other thing and it would give me that.
All times v, but now I've got everything times v,
it's true for every v, so the quantity has to be zero.
That's the Euler-Lagrange equation,
strong form for 1D problem.
Yes?
AUDIENCE: [INAUDIBLE PHRASE]?
PROFESSOR: Oh, you're right.
You're right.
If we had this situation then we would be up to --
this would typically be a fourth-order equation and we
would have two boundary conditions at each end.
Absolutely.
When I slipped in this just to sort of show the pattern,
I didn't account for the boundary conditions.
That would be one level up as well.
Exactly.
And nature -- well, fortunately we don't get many equations
of higher degree than four.
This would be like the beam equation, the parallel to that
would be a beam equation or a plate
equation or a shell equation, God forbid.
In shell theory they're incredibly
complicated because they're on surfaces.
But the pattern is always this.
Well, and, of course, they're also
complicated because they're in 2D.
So maybe I should say a little bit.
Could I maybe write -- I'm trying to do a lot today.
Trying to do kind of the formal stuff today.
So another step in the formal stuff
would be to get into 2D, which I haven't done.
What would be the famous 2D problem that
leads to Laplace's equation?
So 2D, what would I minimize, just to --
so P of u would be a double integral of du/dx squared --
maybe times a c -- du/dy squared.
I would really like 1/2 on that just to make life good.
Then minus a double integral of f times u.
dx*dy, to emphasize these are double integrals.
That's a very, very important problem.
Many people have tried to study that.
Euler and Lagrange would produce an equation for it.
You might say OK, solve the equation, that
finishes the problem.
But mathematicians are always worried, is there a solution?
Is there a minimum?
So I'm dodging the bullet on that one.
When I say minimize over all functions u,
I could create problems where worse and worse and worse
functions got closer and closer to a minimum
and there was no limiting minimizer.
I won't do that.
This is a problem that works fine.
What happens to it?
Well, should we try to do the same weak form?
We'd have a dP/du, and what do you think it would look like?
It would have the integral, double integral.
What would it have?
It will have a c*du/dx and there will be a dv/dx.
But then integration by parts will
take that x derivative off of v and onto this, with a minus.
I did it fast, did it way fast.
Then out of this thing will come, if I look at the v term
they'll be a dv/dy and I take that y derivative off
of that and onto this.
Oh, I can't use d anymore, I have to use partials.
d by du of c*du/dy.
Then the f is just sitting there.
Oh, it's all multiplied by v. dx/dy I didn't put yet.
Equals zero for all v. Again, this is the same thing
that we had in ordinary calculus where it was just P prime of u,
v equals zero.
This is a level of sophistication up.
It's producing differential equations, not
scalar equations.
So what's the deal?
I've done the integration by parts,
so I've got everything multiplying a v.
So what's the strong form of this problem?
Well, if this integral has to be zero for every v,
then the conclusion is that this stuff in the brackets is zero.
That's always the same.
That's the strong form.
So that's Laplace's equation or actually
Poisson's equation because I have a right-hand side, f of x.
Well, those are the mechanics.
Now, what else comes into the calculus of variations?
You've seen the pattern here.
There's one important further possibility
that we met last time, which was constraints.
We're dealing here with a pure minimization.
I didn't impose any side conditions
on u except maybe the boundary condition.
So let me give you an example which --
I'll just close with an example that I'm going to follow up,
and it's going to have a constraint.
So it'll look like the original problem, but, well,
there will be two u's.
So can I try to get it right here?
My minimization, my unknown will have two components, u_1
and u_2.
And it'll be in 2D actually.
What I'm going to produce here is called the Stokes problem.
I'll study it next time, so if I run out of time, as I probably
will, that's part of the plan.
So it's Stokes and not Navier-Stokes.
All I want to do is to write down a problem in which there
is a constraint.
So I write it as a minimization, say, dv_1/ d -- oh,
probably all these guys are in here.
dv_1/dx and dv_1/dy and dv_2/dx and dv_2/dy --
sorry about all this stuff.
Probably an f_1*v -- oh, I've written v,
because my mind is saying that the usual notation,
I should be writing u because that would fit with
today's lecture.
It's a velocity and that's why many people call it v,
and then they have to call the perturbation some w.
f_1*u_1, f_2*u_2, all that stuff.
No problem.
That would lead to Laplace's equation, just the same,
Poisson's equation.
But I'm going to add a constraint.
This [u 1, u 2] is a velocity, despite the letter.
I want to make the material incompressible.
I have flow here and it's like flow of water,
probably incompressible.
So incompressible means that dv_1/dx plus dv_2/dx is zero.
So that's a constraint.
How the heck do we deal with it?
So I could do this minimization but with the constraint.
So all this stuff I'm totally cool with now.
That would just be calculus of variations,
that would get me the dP/du, but I have
to account for the constraint.
So how do you account for a constraint?
You build it into the problem with a Lagrange multiplier.
So I multiply this thing by some Lagrange multiplier,
and as I emphasized last time, Lagrange multipliers always
turn out to mean something physically,
and here it's the pressure.
So it's natural to call the Lagrange multiplier p of x, y.
I build that in, so I subtract the Lagrange multiplier
times this thing that has to be zero.
That gets in the problem.
Now my function now depends on u and the pressure.
I'm not going to push this to the very limit
to find the strong form.
But the strong form is the Stokes equations
that we'll study.
So we have a lot to do here to make this
into practical calculations where we can compute something.
And finite elements is a powerful way to do it.
So we have to turn these continuous problems
into discrete problems.
And then, later, we have to turn this type of problem, which
will be a saddle point problem because it's got this Lagrange
multiplier in there, into a discrete problem.
Let me just stop by putting the words saddle point there,
and just as in the lecture this afternoon, saddle points
appear as soon as you have constraints and Lagrange
multipliers.
Well, thanks for your patience.
That's a lot of material that will quickly -- Now,
those basic steps will be section 7.2 and will go up
on the web quickly, just as soon as we get them revised.
And I'm writing notes on your projects
and I hope I'll have them ready for Friday.
I'll aim for Friday because Monday is Patriot's Day
and you have to run the marathon.
So I'll see you Friday.
Good.
Thanks.