The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high quality educational resources for free.
To make a donation or view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
ERIK DEMAINE: All right.
Today we start a new section in advanced data structures,
and it's data structures for memory hierarchy.
So the idea here is while almost all data structures think
about memory as a flat thing.
And in the RAM model, you've got a giant array.
You can access the i-th element in your giant array
in constant time.
The memory hierarchy, you admit the reality
of almost all computers since, I don't know, '80s,
which have caches.
The idea is you have your CPU connected
with a very high bandwidth channel to a relatively
small cache, which is connected via a relatively
narrow bandwidth channel to a really big memory.
And in real computers, this keeps going.
You've got level 1 cache, then level 2 cache,
then level 3 cache maybe, then main memory, then disk, then
network, whatever.
Usually the disk has a cache.
So you have this very deep hierarchy,
which is typically growing exponentially in size,
and also growing exponentially in slowness, in latency.
So you'd like to design data structures
that most of the time are working at the cache level,
because that's really fast.
And as little as possible, you'd like
to go to the deeper levels.
So we're going to define two models for working
with memory hierarchies.
The first model is called the external memory model.
It's also called the I/O model.
It's also called the Disk Access Model,
I think just so that it could be called the DAM model.
I usually call it the external memory model.
And this is sort of the simplest model.
It's also the most well studied, I would say.
And it's going to relate closely to the other models
that we're going to look at.
And the idea with the external memory model
is, yeah, OK, a real computer has
many levels in this hierarchy.
But let's just focus on the last one and the next to last one,
because typically that last one is going to be way,
way slower than all the others.
And so you really want to minimize how many
memory transfers happen here.
So I want to count the number of memory transfers between two
levels.
And so this model just thinks about two levels.
I'll call them cache and disk.
We think of disk as infinite and slow.
We think of cache as not only fast, but infinitely fast.
It takes zero time to access something in cache.
All we're thinking about is how many
transfers between the left and right have to happen.
So computation here is free.
Access over here is free.
And we just pay for this.
That's the external memory model.
You can also think about running time, make sure running time
is still optimal and that will guarantee this part's good.
But usually the hard part is to minimize this part,
minimize number of memory transfers.
Now, to make this problem really interesting,
we need to add one more assumption.
And this is also the case in practice,
that when you request--
when you transfer an item from disk to cache,
you get not just that item, but entire block
of items at the same speed.
So in fact, disk we're going to think
of as divided into blocks.
Block size here is B, capital B.
Also over here, caches are divided into blocks of size B.
And we have a limited number of them.
We're going to suppose we have M over B cache lines
they're usually called at the cache level.
And so the total size of the cache
is M. M stands for memory.
Sorry.
This notation is slightly confusing
for historical reasons, but that's the way it is.
So B is block size.
M is cache size, total cache size.
There's M over B blocks of size B. Disk
has infinitely many blocks of size B.
There's lots of motivations for this.
But basically, the main cost you're paying here is latency.
You have to send a request and then
the answer has to come back.
Think of that as the disk is far away from the CPU.
And so it takes a long time for the request to get there
and for the data to get back.
So if you're sending data back, you
might as well send-- it's just as fast to send a lot of data
as a little bit of data.
In fact, typically you set this block size
that the amount of time it takes for data to arrive
equals the latency to balance those two costs.
With disk it's especially the case,
because you have this latency of moving the disk head.
Once you move the disk head, you might as well
read the entire track, the entire circle of that
if you're dealing with hard drives.
It's just as fast once the head gets there.
OK.
So that's the model.
And so now we want to minimize the number of block memory
transfers this happens.
And the blocks make things a lot more interesting,
because that means when you read an item,
you really want to use all the items right around it
soon, before it gets kicked out of cache.
Obviously, cache is limited size,
so when you bring something, you've
got to kick something out.
That's the block replacement strategy.
In the external memory model, you can do whatever you want.
You have full control over this cache.
So let's see.
Let me write some obvious bounds.
And then I'll tell you some of the main results in this model.
So number of memory transfers is what we care about.
And it's going to be at most the running time on a RAM,
so this is a comparison between two models.
At most, every operation you do on a RAM data structure
incurs one memory transfer.
That would be the worst case.
So that's what we want to beat.
There's also a lower bound.
So this is also at least the number of cell probes divided
by B. I think we've talked briefly about the cell probe
model, I think in retroactive data structures.
Cell probe model is this model where all you count is--
you just count how many words did I
have to read in order to do this data structure operation?
So computation is free.
We just count number of word reads from memory.
So it's a lower bound on RAM data structures.
And in this model, we're only counting memory accesses,
but where cell probes you count word numbers,
here we're counting blocks.
And so at best, we can improve by a factor of B.
So this is a lower bound.
This is an upper bound, so there's roughly a slop
of a factor of B here.
We want to do closer to this bound than this bound.
This is the obvious thing.
Want to get down to here whenever we can.
So let me give you some examples of that,
so sort of a bunch of basic results in this model.
Result 0 is scanning, so if I have an array of items
and I just want to, say, add them all up
or do a linear scan to search for something, anything that
involves accessing them in order,
this will cost I guess ceiling of N over B.
If the array has size N, number of blocks in that array
is ceiling of N over B. And so that's how many block accesses
it costs.
So the annoying part here is the ceiling.
If N is smaller than B, then this is not great.
I mean, you have to pay one memory read, so it's optimal.
But anyway, that's scanning.
Pretty boring.
Slightly more interesting, although not that much more
interesting, is search trees.
If you want to be able to search for an item
in this external memory model--
in this external memory model, what do you do?
B-trees, conveniently named B-trees.
We want the branching factor of a B-tree
to be B. Anything that's theta B will do,
but exactly B is great.
If you can read all of these pointers
and read all of these key values, if that fits in B,
then we're happy.
So yeah, typically we assume the branching factor here
is like B plus 1.
So in particular, if B is 1, you still
have a branching factor of 2, just
to make it a reasonable tree.
And so now in a constant number of block
reads you can read all the data you need to decide
which child pointer you follow.
You follow that child pointer, and so it
takes log base B plus 1 of N to do a search.
This is counting memory transfers.
So this is not as big an improvement.
Here, we've improved by a factor of B roughly.
Here we're improving by a factor of basically log B.
So not as good as you might hope to do, dividing by B,
but it turns out this is optimal.
You need log base B plus 1 of N to do
a search in comparison model.
In the same way, you need log base 2 of N time
to do a search in the comparison model,
in this model you need log base B of N.
Let me talk a little bit about the proof of that.
It's a pretty simple information theoretic argument.
So we prove this lower bound.
The idea is the following.
You want to figure out among N items
here where your item fits.
Let's say it fits right here.
So you want to know it's between this element and this element.
How many bits of information is that?
Well, there's I guess N plus 1 different outcomes of where
your item could fit.
If you allow it to exactly match, it's maybe twice that.
But there's theta N options for your answer.
And so if you take logs, that is log N bits of information.
And this is how you normally show
that you need log N comparison.
You need log N comparisons because each comparison
could at most give you one bit of information, yes or no.
And there's log N bits to get, so that's the lower bound
in binary search.
But now we're in this model where we're not
counting comparisons, even though we're
in the comparison model.
We're counting number of block transfers.
So when I read in some data, I get B items from my array,
or from something.
So when I do a block read, all I can do in the comparison model
is compare my item to everything that I read,
so you might as well compare all of them.
But the information you learn is only log B plus 1 bits,
because what you learn is among these B items, where do I fit?
And in the best case, that is log of B
plus 1 bits of information.
It could be worse, if these guys are not well distributed
among your N items.
It's at most that.
And so you take the ratio and you need log N over log B
plus 1 block reads.
And that's log base B plus 1 of N.
AUDIENCE: [INAUDIBLE]
ERIK DEMAINE: Yeah.
Question?
AUDIENCE: Why is that you learn log B plus 1 bits when
you [INAUDIBLE]?
ERIK DEMAINE: All right.
So the claim is, in the comparison model,
you compare your item to everyone in here.
But if your array is presorted, then all you're learning is--
I mean, information theoretically,
all your learning is where your item fits among these B items.
So there's order B options of where you fit, and so only log
B bits of information there.
You might consider this a vague argument.
You can do a more formal version of it.
But I like the information theoretic perspective.
Cool.
So that's the best thing you can do for search in this model.
Now, you might also wonder about insertions and deletions.
B-trees support insertions and deletions in log
base B plus 1 of N time, but that is not optimal.
We can do better.
So for example, if you want to sort,
the right answer for sorting is this crazy bound.
N over B, which is really good, that's
linear time in this model, times log base M over B of N over B.
This is a slight-- this is roughly log base M of N.
The over B's don't make too big a difference.
So this is an improvement.
Normally sorting takes N log N with a base 2.
We're improving the base of the log to M over B instead of 2.
And we're improving the overall running time by a factor of B.
So this is actually a really big improvement.
We're going faster than the RAM time divided by a B,
because to sort in the cell probe model
just takes linear time.
You read all the data.
You sort it.
You put it back.
So we can't quite get N over B, but we get this log factor.
So this is really good.
And it's something that would not result from using B-trees.
This is a little funny, because in regular RAM model,
if you insert everything into a balanced binary search tree,
then do an in order traversal, that's
an optimal sorting algorithm.
It gives you N log N.
Here, it's not optimal.
You throw everything into a B-tree,
you get N times log base B of N. So there's
two things that are bad.
One is that the base of the log is slightly off.
It's B versus M over B. That's not so big a deal.
The big thing is it's N over B versus N.
We don't want to pay N. We want to pay N over B. There's also
a matching lower bound in the comparison model, which we will
not cover here.
By next lecture, we'll know how to do sorting optimally.
First, I want to tell you about a bunch
of different ways or different problems and results.
So a funny problem, which actually does
arise in practice, but it's interesting just
to think about, instead of sorting--
sorting is I give you an array.
I don't know what order it's in.
You have to find the right order and then
put it into the right order.
There's an easier problem, which is called permuting.
I give you an array.
I give you a permutation that I'd like to do.
I just want you to move the items around.
So you don't need to do any comparisons to get here.
I tell you what the order is.
You just have to physically move the items.
This is harder than it sounds, because when you read,
you read blocks of items in the original order.
And so it's hard to just do an arbitrary permutation.
And there's two obvious algorithms.
One is you could sort your items.
So you could go to every item in order
and just write down its desired number in the new permutation,
write down the permutation.
Then run a sorting algorithm.
That will take N over B log base M over B of N over B.
Or the other obvious algorithm is to take the first item,
put it where it belongs in the array, take the second item,
put it where it belongs in the array,
in a new copy of the array.
And that will take at most N memory transfers,
each one a completely separate memory transfer.
And the claim is the optimal thing
to do is the min of those two.
So you just check how does N and B
and M compare, do the better of the two algorithms.
Again, there's a lower bound.
In a model called the indivisible model,
which is assuming that each word of your data
doesn't get cut into subwords.
So this is assuming--
the lower bound assuming your words do not get cut up.
I believe it's still unsolved whether you could do better
otherwise.
The last result in this model I wanted to talk about
is called buffer trees.
Buffer trees are pretty cool.
They're essentially a dynamic version
of sorting, which is what we lack--
what B-trees do not give you.
So they achieve the sorting bound
divided by N per operation, so this is optimal.
I haven't told you what the operations are.
The amount is amortized.
It has to be amortized, because this bound is typically
little o of 1.
Think of this as 1 over B. There's a log factor.
But assuming that log factor is smaller than B, which is often
the case, this is in order 1 over B.
It's a little o of 1 cost, which is a little fun.
But of course, every real cost is an integer.
It's an integer number of block transfers.
But amortized, you can get a little o of 1 bound.
And what it lets you do is delayed queries and batch
updates.
So I can do an insert and I can a delete.
I can do many of them.
And then I can do a query against the current data
structure.
I want to know, I don't know, the successor of an item.
But I don't get the answer right then.
It's delayed.
The query will sit there.
Eventually the data structure will
decide to give me an answer.
Or it may just never give me an answer.
But then there's a new operation at the end, which is flush,
which is give me all the answers to all the questions
that I asked.
And the flush takes this much time.
So if you do N operations and then say flush,
then it doesn't take any extra time.
It's going to cost one sort of pass
through the entire data structure, one sorting step.
Yeah.
The one thing you can get for free is a zero-cost find min.
All right.
So buffer tree, you can do find min instantaneously.
It maintains the min all the time.
So this lets you do a priority queue.
You can do insert and delete min in this much time online.
But if you want to do other kinds of queries,
then they're slow to answer.
You get the answer much later.
So this is a dynamic version of sorting.
It lets you do sorting in particular.
It let's you do priority queues, other great things.
We will see not quite a buffer tree
but another data structure achieving similar bounds
in the next lecture.
Today's lecture is actually going
to be about B-trees, which seems kind of boring,
because we already know how to do B-trees.
But we're going to do them in another model, which
is the cache-oblivious model.
So let me tell you about that model.
All right.
So the cache-oblivious model is much newer.
It's from 1999.
And it's from MIT originally, Charles Leiserson
and his students invented it back then,
in the context of 6406, I believe.
And it's almost the same as this external memory model.
There's one change, which is that the algorithm doesn't
know what B or M is.
Now, this is both awkward and cool at the same time.
So the assumption is that the computer looks like this.
The algorithm has no--
and the algorithm knows that fact,
but it doesn't know what the parameters of the cache are.
Doesn't know how the memory is blocked.
Now, as a consequence, it cannot manually say, read this block,
write this block.
It can't decide which block to kick out.
This is going to be more like a typical algorithm.
I mean, the algorithm will look like a regular RAM algorithm.
It's going to say read this word,
write this word to memory, do an addition, whatever.
So algorithm looks like a RAM algorithm.
But the analysis is going to be different.
So this is convenient, because algorithms--
I mean, how you implement an algorithm becomes
much clearer in this model.
And this is-- now we're going to assume that the cache is
maintained automatically.
In particular, the way that blocks are replaced
becomes automatic.
So the assumption is when you read a word from memory,
the entire block comes over.
You just don't know how much data that is.
And when something gets kicked out,
the entire block gets kicked out and written to memory.
But you don't get to see how that's
happening, because you don't know anything about B or M.
We need to assume something about this replacement
strategy.
And in the cache-oblivious model,
we assume that it is optimal.
This is an offline notion, saying
what block we're going to kick out
is the one that will be used farthest in the future,
so this is very hard to implement in practice,
because you need a crystal ball.
But good news is, you could use something
like LRU, Least Recently Used, kick that guy out, or even
a very simple algorithm, first in, first out.
The most recent-- the oldest block
that you brought in is the one that you kick out.
These are actually very close to optimal.
They're constant competitive against or given
a cache of twice the size.
So this is from the early days of competitive analysis.
It says if I compare offline optimal with a cache size M
versus--
or let's say offline optimal with a cache size
M over 2, versus LRU or FIFO, which
are online algorithms on a cache of size M,
then this cost will be at most I think twice this cost.
Now, the good news is, you look at all these bounds,
you change M by a factor of 2, nothing changes.
Assuming everything is at least some constant,
and M and B are not super super close,
then changing M by a factor of 2 doesn't affect these bounds.
These ones don't even have M in them, so it's not a big deal.
So we're going to assume this basically for analysis.
It's not really that big a deal.
You could also analyze with these models.
The point is they're all the same
within a constant factor for the algorithms we care about.
What other fun things?
If you have a good, cache-oblivious algorithm,
it's good simultaneously for all B and M.
That's the consequence.
So whereas B-trees for example, need to know B,
they need to have the right branching factor for your value
of B, the cache-oblivious model, you have to have--
your data structure has to be correct,
has to be optimized for all values of B and all values of M
simultaneously.
So this has some nice side effects.
In particular, you essentially adapt
to different values of B and M. So two nice side effects
of that are that if you have a multi-level memory hierarchy,
each level has its own B and its own M.
If you use a cache-oblivious algorithm
and all of those caches are maintained automatically
using one of these strategies, then
you optimize the number of-- maybe I should draw a picture.
Here's a multi-level memory hierarchy.
You optimize the number of memory transfers
between these two levels.
You optimize the number of memory transfers
between these two levels, between these two levels,
and between these two levels.
So you optimize everything.
Each of them has some--
if you just think of this as one big level versus this level
or versus all the levels to the right,
then that has some B and some M. And you're
minimizing the number of memory transfers here.
Also, if B and M are changing, lots of situations for that.
If you have multiple programs running at the same time,
then your cache size might effectively
go down if they're really running
in parallel on the same computer, or pseudo-parallel.
Block size could change.
If you think of a disk and your block size
is the size of your track, then closer to the center,
the tracks are smaller.
Closer to the rim of the disk then blocks are bigger.
All these things basically in the cache-oblivious model,
you don't care, because it will just work.
Now, there's no nice formalization
of adapting to changing values of B and M.
That's kind of an open problem to make
explicit what that means.
But we're going to assume in the cache-oblivious models you
have one fixed B, one fixed M, analyze relative to that.
But of course, that analysis applies simultaneously to all
B's and all M's.
And so it feels like you adapt optimally, whatever that means.
In some sense, the most surprising thing
about the cache-oblivious world is that it's possible.
You can work in this model and get basically
all of these results.
So let me go over these results.
So in the cache-oblivious model, of course scans are still good.
Scans didn't assume anything about block size here,
except in the analysis.
Scanning was just a sequential read.
In general, we can do constant number of scans.
Some of them can go to left.
Some of them could go to the right.
Some of them could be reading.
Some of them could be writing.
That's the general form of scan.
But all of that will take order N over B ceiling.
OK.
Search.
This is what we're going to be focused on today.
You can achieve order log base B plus 1 of N search,
insert, and delete.
Just like B-trees, but without knowing what B is.
We're going to do most of that today.
What's next?
Sorting.
Sorting, you can do the same bound.
That we'll see next lecture.
OK.
Next is permuting.
Here there's actually a negative result, which
is that you can't do the min.
It doesn't say exactly what you can do.
In particular, we can certainly permute
as quickly as we can sort.
And we can permute--
we can achieve this min in a weird sense,
in that we can achieve N and we can achieve the sorting
bound for permutation in the cache-oblivious model,
either one, but not both with min
around them, because to know which one to apply,
you need to know how B and M relate to N,
and you don't here.
And so there's a lower bound saying
you really can't achieve this bound,
even up to constant factors.
But it doesn't explicitly say anything more than that.
So a little awkward.
Fortunately, permuting isn't that big a deal.
And most of the time the sorting bound
is much faster than the end bound, so not so bad.
But slightly awkward.
So there's some things you can't do.
This was the first negative result in cache-oblivious.
Another fun negative result is actually in search trees.
You might ask, what is this constant factor here
in front of the log base B plus 1 of N?
And for B-trees, the constant factor is essentially 1.
You get 1 times log base B plus 1 of N.
In cache-oblivious world, you can
prove that 1 is unattainable and the best possible
constant factor is log base 2 of e, which was like 1.44.
So that is the right answer for the constant.
Today we'll get a constant here of 4.
This is for search.
But 4 has since been improved to 1.44, so a tiny separation
there between cache-oblivious and external memory.
Let's see.
One more thing.
Continuing over here is priority queue.
Cache-oblivious we can do in the optimal time
1 over B log base M over B of N over B.
Only catch with that result is it
assumes something called the tall cache assumption.
Let's see what I want to say here.
We'll say M is omega B to the 1 plus epsilon.
That's an ugly-looking omega.
Certainly M is at least B. If you have one cache line,
you have M equal B. But we'd like
M to be a little bit bigger than B. We'd like the cache to be--
if M was at least B squared, then the height of the cache
would be at least as big as the width of the cache, hence
tall cache.
But we don't even need M equal to B squared.
Just M to the--
M equaling B to the 1.001 would be enough.
Then you can get this result.
And there's a theorem that shows for cache-oblivious
you need to assume this tall cache assumption.
You cannot achieve this bound without it.
But this is usually true for real caches, so life is good.
All right.
So that's a quick summary of cache-oblivious results.
Any questions about those?
Then we proceed into achieving search trees log base B of N
performance cache-obliviously.
Make it interesting, because external memory,
we already know how to do it with a B-tree.
So we did both models, cache-oblivious B-tree
is what remain.
This was the first result in cache-oblivious data
structures.
I'm going to start with the static problem.
Cache-oblivious static search trees.
This was actually in a MEng thesis here by Harold Prokop.
So the idea is very simple.
We can't use B-trees because we don't know what B is,
so we'll use binary search trees.
Those have served us well so far.
So let's use a balanced binary search tree.
And because it's static, might as well
assume it's a perfect balanced binary search tree.
Let me draw one over here.
I'll draw my favorite one.
So how do you search in a binary search tree?
Binary search.
Our algorithm is fixed.
So the only flexibility we have is
how we map this tree into the sequential order in memory.
Memory is an array.
So for each of these nodes, we get
to assign some order, some position
in the array of memory.
And then each of these child pointers, that
changes where they point to.
And now cache-obliviously, you're
going to follow some root to leaf path.
You don't know which one.
All of them are visiting nodes in some order,
which looks pretty random.
In the array of memory, it's going
to be some sequence like this.
We would like the number of blocks
that contain those nodes to be relatively small somehow.
So that's the hard part.
Now, you could try level order.
You could try in order, pre order, post order.
All of them fail.
The right order is something called van Emde Boas order.
So let's say there are N nodes.
What we're going to do is carve the tree
at the middle level of edges.
So we have a tree like this.
We divide in the middle.
Leaves us with the top part and many bottom parts.
There's going to be about square root of N nodes
in each of these little triangles, roughly speaking.
The number of triangles down here
is going to be roughly square root of N--
I think I have written here square root of N plus 1.
It's a little overestimate, because I multiply those two
numbers, it's bigger than N.
But it's roughly that.
If there were root N nodes up here,
then there would be square root of N plus 1 children,
and square root of N plus 1 things down there.
So roughly, root N different things, each of size root N.
Then what we do is recursively lay out
all those triangles of size roughly squared of N,
and then concatenate.
It's a very simple idea.
Let's apply it to this tree.
So I split in the middle level and I recursively
lay out this triangle up here--
draw it as a box.
Then I recursively lay out this.
Then I recursively lay out this and this and this.
So up here, I mean, to lay this out,
I would again split in the middle level,
lay out this piece, so this guy will go first, then
this one, then this one.
Now I'll go over and recursively do this, so it's 4, 5, 6.
Then this one, 7, 8, 9, 10, 11, 12, 13, 14, 15.
That is the order in which I store the nodes.
And I claim that's a good order.
For example, if B equals 3, this looks really good,
because then all of these nodes are in the first block.
And so I get to choose which of these four children trees
I should look at.
I mean, I happen to go here and then maybe here.
I look at those two nodes, look at their keys to decide,
oh, I should go into this branch.
But I only pay one memory transfer
to look at all those nodes.
So this looks like a tree tuned for B equals 3.
But it turns out it's tuned for all B's simultaneously,
if we do this recursively.
It's hard to draw a small example
where you see all the B's at once,
but that's the best I can do.
OK.
Let's analyze this thing.
So the order of the nodes is clear.
If you know what the van Emde Boas data structure is,
then you know why it's called this.
Otherwise, see lecture 10 or so in the future.
Forget exactly where we're covering it.
So this order was not invented by van Emde Boas,
but it looks very similar to something else that
was invented by van Emde Boas, so we call it that.
But that's the name of a guy, Peter.
Analysis.
Order's clear.
The algorithm is clear.
We follow our root to leaf path.
What we claim is any root to leaf path and any value of B,
the number of blocks visited is order log base B of N.
So to do that, in the analysis we get to know what B is.
The algorithm didn't know.
Algorithm here doesn't know B and doesn't need to know B.
But to analyze it, to prove an order log base B of N,
we need to know what B is.
So while the algorithm recurses all the way down,
keeps dividing these blocks into smaller and smaller pieces,
we're going to look at a particular level
of that recursion.
I'm going to call that a level of detail.
Say I call it straddling B.
So my picture is going to be the following.
These little triangles are going to be size less than B.
But these big triangles are size greater than or equal to B.
OK.
At some level of recursion that happens, by monotonicity,
I guess.
I could keep going.
I mean, this tree is going to have substantial height.
I don't really know how tall it is,
or that's sort of the central question.
Draw in some pointers here.
But just look at--
if you subdivide all the big triangles
until you get to these small triangles that
are size less than B, then you stop recursing.
Now, the algorithm, of course, will recursively lay that out.
But what we know, we recursively lay things out
and then we concatenate, meaning the recursive layouts remain
consecutive.
We never like interleave two layouts.
So however this thing is laid out, we don't really care,
because at that point it's going to be stored in memory,
which is this giant array.
It's going to be stored in interval of size
less than B. So at worst, where are the block boundaries?
Maybe draw that in red.
Block boundaries are size B. We don't know exactly how they're
interspersed.
Let's say these are the block boundaries.
So it could be that this little subarray falls into two blocks,
but at most two blocks.
So the little triangles of size less than B
live in at most two memory blocks.
So to visit each of these little triangles,
we only spend two memory reads at most.
So all that's left is to count how many of those
little triangles do we need to visit on a root to leaf path?
I claim that's going to be log base B of N.
So I guess the first question is, what is the height of one
of these little triangles?
Well, we start with a tree of height
log N. Then we divide it in half and we divide it in half
and divide it in half.
And we take floors and ceilings, whatever.
But we repeatedly divide the height in half.
And we stop whenever the number of nodes in there
is less than B. So that means the height is going
to be theta log B. In fact, it's going
to be in the range 1/2 log B to log B,
probably with a paren that way.
Doesn't really matter, because you stop as soon as you hit B.
You're dividing the height in half repeatedly.
You might overshoot by at most a half.
That's it.
So height is log B.
So if we look along a root to leaf path,
each of these things we visit, we
make vertical progress log B, theta log B.
And the total vertical progress we need to make is log N,
and so it's log N divided by log B.
So I guess 4 times log N, 2 log N over log B.
Triangles, we reach a leaf.
It's total progress we need to make divided by 1/2
log B here, progress per triangle.
So we get 2 times log base B of N. And then for each triangle,
we have to pay 2 memory reads, so
this implies at most 4 log base B of N memory transfers.
And I implicitly assumed here that B is not 1.
Otherwise, need a B plus 1 here to be interesting.
So long as B isn't 1, then it doesn't matter.
OK.
So that's the log base B of N analysis, actually really
simple.
Essentially what we're doing here
is binary searching on B in a kind of weird way, using divide
and conquer and recursion.
So we say, well, maybe B is the whole thing,
then B equals N. Then we don't have to do anything.
Everything fits in a block.
Otherwise, let's imagine dividing here and supposing
B is root N, and then we need two accesses.
But B might be smaller than that, so we keep recursing,
keep going down.
And because we're dividing in half each time--
sorry.
We're not binary searching on B. We're
binary searching on log B, which is the height
of a tree with B nodes in it.
And that's exactly what we can afford to do.
And conveniently, it's all we need to do,
because to get log base B of N correct,
you only need to get log B correct up
to a constant factor.
All right.
Now, that was the easy part.
Now the hard part is to make this dynamic.
This is where things get exciting.
This works fine for a static tree.
And this is how we do search.
But we can't do inserts and deletes in this world easily.
But I'll show you how to do it nonetheless.
So this is what's called a cache-oblivious B-tree.
We're going to do it in five steps.
And the first step, unfortunately,
is delayed until next lecture, so I need a black box.
It's a very useful black box.
It's actually one that we've briefly talked about before,
I think in the time travel.
If you remember, in time travel there
was one black box, which we still haven't seen
but we will do finally next lecture, which
was you can insert into a linked list
and then given two items in the linked list
know which one comes before which other one
in constant time per operation.
Closely related to that is a slightly different problem
called order file maintenance.
Here's what it lets you do.
You can store N elements, N words in specified order,
so just like it's a linked list.
But we don't store it as a linked list.
We want to store it in an array of linear size.
So there's going to be some blank spots in this array.
We've got N items, an array of size, say 2N or 1.1 times
N, anything like that will suffice.
The rest, we call them gaps, gaps in the array.
And the updates-- so this is physically what you have to do.
You can change this set of elements
by either deleting an element or inserting an element
between two given elements.
So this is just like the linked list updates.
I can delete somebody from a linked list.
I can insert a new item in between two given
items in the linked list.
But now it has to be stored in a physical array of linear size.
Now, we can't do this in constant time.
But what we can do is move elements around.
OK.
So let me draw a picture.
So maybe we have some elements here.
They could be in sorted order, maybe not.
There are some gaps in between.
I didn't mention it, but all gaps will have constant size.
So this will be constant-sized gaps.
And we're able to do things like insert a new item
right after 42.
Let's say we insert 12.
So 12, we could put it right here, no problem.
Now maybe we say insert right after 42 the number 6.
And we're like, oh, there's no room to put 6.
So I've got to move some stuff.
Maybe I move 12 over here and then I put 6 here.
And then I say, OK, now insert right after 42 the number 7.
And you're like, uh-oh.
I could move these guys over one and then put 7 here, and so on.
Obvious algorithm to do this.
Takes linear time, linear number of shifts in order
to do one insertion.
Claim is you can do an insertion in log squared amortized.
It's pretty cool.
So worst case, you'll still going
to need to shift everybody.
But just leaving constant-sized gaps
will be enough to reduce amortized costs to log squared.
And this is conjectured optimal, though no one
knows how to prove that.
And that's order file maintenance.
Now, we're going to assume this is a black box.
Next lecture, we'll actually do it.
But I want to first show you how it
lets us get cache-oblivious dynamic B-trees.
Log base B of N, insert, delete, search.
OK.
We're going to use an ordered file maintenance data
structure to store our keys, just stick them in there.
Now, this is good--
well, it's really not good for anything,
although it's a starting point.
It can do updates in log squared of N--
I didn't mention this part--
in a constant number of scans.
So think of these as this kind of shift.
I have to scan left to right, copy 23 to the left,
42 to the left, and so on.
Scans are things we know how to do fast in N
over B memory transfers.
So if you're going to do log squared--
if you're going to update an interval log squared guys
using constant number of scans, this
will take log squared N over B memory transfers.
Not quite the bound we want.
We want log base B of N.
This could be larger or smaller than log base B of N.
It depends how big B is.
But don't worry about it for now.
This is still-- at least it's poly log.
Updates are kind of vaguely decent.
We'll improve them later.
On the other hand, search is really bad here.
I mean, we're maintaining the items--
I didn't mention it, but we will actually maintain the items
in sorted order.
So yeah, you could do a binary search.
But binary search is really bad cache-obliviously,
because most of the time you'll be visiting
a new block until the very end.
So binary search-- this is a fun exercise--
is log N minus log B. We want log N divided by log B.
So we can't use binary search.
We need to use van Emde Boas layouts, this thing.
So we just plug these together.
We take this static search tree and stick it
on top of an ordered file maintenance data structure.
So here we have a van Emde Boas layout static tree
on top of this thing, which is an ordered file.
And we have cross-linking.
Really here we have gaps.
And we don't store the gaps up here.
Or do we?
Yeah, sure.
We do, actually.
Store them all.
Put it all in there.
All right.
I'm going to have an actual example here.
Let's do it 3 slash 7, 9 slash 16, 21, 42.
OK.
Cross-links between.
I won't draw all of them.
Then we store all the data up here as well.
And then the internal nodes here are
going to store the max of the whole subtree.
So here the max is 3.
Here the max is 9, 16, 42, 42, and 9.
If I store the max of every subtree, then at the root.
If I want to decide should I go left or should I go right,
I look at the max of the left subtree
and that will let me know whether I
should go left or right.
So now I can do binary search.
I do a search up here.
I want to do a search, I search the tree.
This thing is laid out just like this.
And so we pay log base B of N to search up here, plus 1,
if you want.
And then we get to a key and we're done.
That's all we wanted to know.
So search is now good.
Updates, two problems.
One is that updating just this bottom part is slow.
Second problem is if I change the bottom thing,
now I have to change the top thing.
But let's look at that part first.
It's not so bad to update--
I mean, this structure stays the same, unless N doubles.
Then we rebuild the entire data structure.
But basically, this thing will stay the same.
It's just data is moving around in this array.
And we have to update which values
are stored in this structure.
The layout of the structure, the order
of the nodes, that all stays fixed.
It's just the numbers that are being
written in here that change.
That's the cool idea.
So that's step 2.
Let's go to step 3.
I actually kind of wanted that exact picture.
That's OK.
I need to specify how updates are done.
So let's say I want to do an insertion.
If I want to insert a new key, I search for the key.
Maybe it fits between 7 and 9.
Then I do the insertion in the bottom structure
in the ordered file.
So we want to insert 8.
These guys move around.
Then I have to propagate that information up here.
And the key thing I want to specify
is an update does a search.
Then it updates the ordered file.
That's going to take log squared N over B.
And then we want to propagate changes
into the tree in post-order, post-order traversal.
That's the one thing I wanted to specify,
because it actually matters in what order I update the item.
So for example, suppose I move 9 to the right
and then I insert 8 here.
So then this thing--
let's do it, what changes in that situation.
9 will change to an 8 here, which
means this value has to change to an 8
and this value needs to change to an 8.
And this one I can't quite update yet,
because I need to do the other tree.
So here I change this value to a 9.
This one doesn't change, so this is OK.
This is OK.
Just taking the maxes as I propagate up.
So the red things are the nodes that I touch and compute
the maxes of.
And the order I did it was a post-order traversal.
I need to do a post-order traversal in order
to compute maxes.
I need to know what the max of this subtree
is and this subtree before I can re-compute the max of the root.
So it's really the obvious algorithm.
Now, the interesting part is the analysis.
So searches we've already analyzed.
We don't need to do anything there.
That's log base B of N.
But updates are the interesting thing to analyze.
And again we're going to look at the level of detail that
straddles B, just like before.
So let me draw a good picture.
I'm going to draw it slightly differently
from how I drew it last time.
OK.
What I'm drawing here is actually
the bottom of the tree.
So a real tree is like this.
And I'm looking at the bottom, where
the leaves are, because I care about where
I connect to this ordered file.
So there's the ordered file at the bottom.
And I'm refining each triangle recursively
until they're size less than B. They're
going to be somewhere between root B and B.
These big triangles are bigger than B. OK.
So that's where I want to look at things.
Now, when we update in the ordered file,
we have this theorem.
It says you move elements in an interval.
So what changes is an interval.
The interval has size log squared N amortized.
So we are updating some interval here,
maybe something like this.
And so all of these items change, let's say.
Now, this is not going to be too big.
It's going to be log square N amortized.
And all of the ancestors of those nodes
have to be updated, potentially.
So all these guys are going to get updated.
Some of this tree, this path, all of this stuff
is going to get updated, some of this stuff,
the path up to here.
The path up to the root gets updated, in general,
and all these nodes get touched.
Now, I claim that's cheap, no more expensive than updating
down here.
Let's prove that.
OK.
First issue is, in what order do I visit the nodes here?
I want to think about this post-order.
Maybe I'll use another color to draw that.
So let's say we start by updating this guy.
We update this value.
Then we go--
I'm doing a post-order traversal, this tree.
Once I've completely finished here,
I can go up here, maybe go up here, visit over down this way,
fix all the maxes here.
Then I go up, fix all the maxes here,
fix all the maxes, the maxes.
When I come up, then I can fix the maxes up here, and so on.
OK.
Think about that order.
If you just look at the big picture, which
is which triangles are we visiting, what we do
is we visit this triangle, then this one, then this triangle,
then this one, then this triangle, then this one.
I guess this is the better picture.
This triangle, left one, this triangle, next child,
this parent, next child, this parent, next child.
And when we're done, we go over to the next big triangle.
Within a big triangle, we alternate
between two small triangles.
As long as we have four blocks of cache,
this will be good, because there's
at most two blocks for this little triangle,
at most two blocks for this little triangle.
So jumping back and forth between these
is basically free.
So we assume-- white chalk.
We assume that we have at least four blocks of cache.
Post-order traversal alternates between two little triangles.
So this part is free.
The only issue is, how many little triangles
do I need to load overall?
Sorry.
I should say I'm just looking at the little triangles
in the bottom two levels.
So this is in the bottom two levels.
Sorry.
I forgot to mention that.
As soon as you go outside the bottom big triangle,
let's not think about it yet.
Worry about that later.
So just looking at these bottom two levels that I drew,
we're just doing this alternation.
And so the only issue is, how many little triangles
do I have to load?
So there could be these guys.
These guys are good, because I'm touching all these nodes.
The number of nodes here was bigger than B,
so if I take this divided by B in the ceiling,
the ceiling doesn't hurt me, because the overall size is
bigger than B.
These guys I have to be a little bit careful.
Here the ceiling might hurt me, because I only
read some of these nodes.
It may be much less than B. It could
be as low as root B. I divide by B, take the ceiling, I get 1,
so I have to pay 1 at the beginning, 1 at the end.
But in between, I get to divide by B.
So the number of little triangles in the bottom two
levels that I visit is going to be 1 plus log squared N over B
amortized.
The size of the interval is log squared N amortized,
so this would be if I'm perfect.
But I get a plus 1 because of various ceilings.
And also, the number of blocks is the same,
because each of these little triangles
fits inside at most two blocks.
So it's going to be order 1 plus log squared N over B.
OK.
So I can touch all those in this much time,
because I assume that I can alternate between two of them
quickly.
And once I finish one of these leaf blocks,
I never touch it again, so I visit each block essentially
only once as long as my cache is this big.
Cool.
So that deals with the bottom two levels.
I claim the bottom two levels only pay this.
And we were already paying that to update the ordered file,
so it's no worse.
Now, there's the levels above this one that we can't see.
I claim that's also OK.
Sorry.
Also need number of memory transfers is equal to that.
OK.
One more part of the argument.
Levels above the bottom two.
Nice thing about the levels above the bottom two
is there aren't too many triangles.
All right.
Well, there aren't too many nodes involved.
Let's see.
So down here, there were like log squared N nodes.
There was a lot of nodes.
Once I go up two levels, how many of these nodes
can there be, these roots?
Well, it was log squared N at the bottom.
And then I basically shaved off a factor of B here,
so the number of these roots is at most log squared N
over B. That's small.
For those nodes, I could afford to do an entire memory
transfer for every single node.
So the number of nodes that need updating--
these are ancestors-- is order log squared
N over B. That's the number of leaves down there.
At some point, you reach the common ancestor
of those leaves.
That tree-- so here, draw a big picture.
Here's the bottom two layers.
We're shaving that off.
Up here, the number of nodes here, these roots that we're
dealing with, that's the log squared N over B.
So if we build a tree on those nodes,
total number of nodes in here is only log
squared N over B times 2, because it's a binary tree.
But then there's also these nodes up here.
This is potentially log N nodes.
So number of nodes that need updating
is at most this plus log N. So what's going on here
is we first analyzed the bottom two layers.
That was OK because of this ordering thing,
that we're only dealing with two triangles at once
and therefore we pay this size divided by B,
log squared N over B.
Then there's the common ancestors of those nodes.
That forms a little binary tree of size log squared N over B.
For these nodes, this triangle up
here, we can afford to pay one memory transfer for every node,
so we don't care that we're efficient up here, which is--
I mean, well, let's not worry about whether we are.
It doesn't matter.
So for these nodes, we pay at most one memory transfer each.
And so we pay at most log squared N over B for them.
That's fine, because we already paid that much.
And finally, we have to pay for these nodes
up here, which are from the common ancestors of these guys
up to the root.
That has length at most log N. Could be smaller
if this was a big interval.
So for these guys, we pay order log base B of N. Why?
Because this is a root to node path.
And this is the static analysis.
We already know that following a root to node path
we pay at most log base B of N.
So we pay log base B of N plus log
squared N over B for this, plus log squared N over B for this,
plus log squared N over B for the ordered file.
So overall running time for an update
is log base B of N plus log squared N over B.
We actually paid log base B of N twice,
once to do the initial search.
I said an update first has to search for the key.
Then it updates the order file.
That's where we've paid the first log squared N over B.
And then to propagate all the changes up the tree,
we pay log squared N over B for this part,
and then another log base B of N for this part.
OK.
That's updates with a bad bound.
It turns out we're almost there.
So at this point we have searches optimal,
because searches are just searching through the van Emde
Boas thing.
That's log base B of N.
Updates, not quite optimal.
I mean not quite matching a B-tree, I should say,
because it's not an optimal bound we're aiming for.
We're just aiming for B-trees.
We'd want to get this bound, same as searching.
But we have this extra term, which
if B is small, for example if B is log N,
this term will be bigger than this one.
The breakpoint is when B equals log n times log log N.
So for small B, we're not happy.
And we want to be happy for all B's simultaneously.
So I want to fix this.
We're going to fix it using a general technique,
great technique, called indirection.
So this is step 5, indirection.
The idea is simple.
We want to use the previous data structure
on N over log N items.
And, well, we need to store N items, so where do we put them?
On the bottom.
So I'm going to have theta log N nodes down here,
theta log N nodes next to them--
or items, I should say.
Theta log N.
OK.
And then one of these items, let's say the min item,
gets stored in this tree.
The min item over here gets stored in this tree.
The min item here gets stored up here.
OK.
So this item only--
the data structure we just developed,
I only store N over log N of the items.
The rest that fit in between those items
are stored down here.
Now I use standard B-tree tricks, actually.
If I want to do an insertion, how do I do a search?
I follow a root to leaf path.
And then I do a linear scan of this entire structure.
How much does it cost to do a linear scan of log N?
Only log N divided by B. This is smaller than log base B of N,
so the hard part of search is the log base B of N.
OK.
How do I do an update?
Well, first, I search for the guy.
I find the block that it fits into and just insert
into that block.
Done.
As long as this block doesn't get too big,
how do I insert into a block?
Rewrite the entire block.
It's a scan.
It only takes log N over B, so it's
basically free to rewrite this entire block every time
as long as it has size at most log N. If it gets too full,
then I split it into two parts that are half-full.
So in general, each of these blocks
will maintain the size to be between 1/4 log N and log N.
So if it gets bigger than log N, I
split into two guys of 1/2 size.
If it gets smaller than 1/4 log N,
then I'll merge it with one of its neighbors
and then possibly have to re-split it.
Or you can think of it as stealing neighbors
from your siblings.
That's what you do in B-trees.
But with a constant number of merges and splits,
you can restore the property that these guys have
sizes very close to 1/2 log N. And so
if they end up getting full, that means
you inserted 1/2 log N items.
If they ended up getting empty, that
means you deleted 1/4 log N items.
And so in both cases, you have log N items to charge to.
Whenever you do an update up here--
so we update up here every time we split
or merge down here, because this guy has
to keep track of the blocks, not the items in the blocks.
So we effectively slow down the updates
in the top structure by a factor log N,
which is great, because we take this update bound
and we get to divide it by log N, which removes this square.
So we end up with--
so we used to have log base B of N plus log squared N over B.
We get to divide that by log N, plus
we have to do this part at the bottom.
Plus we're doing log N over B at the bottom.
So basically, this log cancels with this square,
so this is log N over B as well.
This thing gets even smaller, so we end up
being able to do updates in log N over B.
Not really though, because at the beginning,
we have to do a search to figure out where to do the update.
If you knew where you had to update,
you could update a little faster.
You could update in log N divided
by B. If you don't know where to update,
you've got to spend log base B of N to search initially.
But that's the fun part.
We can now update this data structure
fast, just with this one layer of indirection with log N
chunks at the bottom, speeding this data structure up
by a factor of log N for updates.
Searches don't take any longer.
You just pay this extra term, which
was already much smaller than what we were paying before.
Phew.
That's cache-oblivious B-trees, other than ordered
file maintenance, which we'll do next class.