---------- Forwarded message ----------
Date: Fri, 1 Mar 2002 02:00:45 -0600 (CST)
From: Yale Patt <patt@ece.utexas.edu>

        Dear Dr Patt,
             I was just wondering about how the L1 size is estimated in real
        processor. My guess is that you  can design your system and see how
much
        time can you actually afford for a memory acess, run a few simulations
        and then tailor the L1 size to fit exactly the "single cycle acess
        time".and then L2/L3... can take multiple cycles.
        however if i recall correctly, in lecture you mentioned that we have L1
        caches which  no longer are single cycle acess (it now takes 2-3
cycles).
        wont it be more useful to have maybe very small ,but a single cycle L1
)
        and not stalling the processor for 2/3 cycles for every data memory
acess.
        Thanks,
        <<name removed to protect the guilty>>

You pretty much have all the answers yourself.  Yes, you would like the L1 to
be single cycle.  However, with today's clock speeds, that would make the size
far too small and in fact, may not even be buildable.  So, we have mostly gone 
to two and three cycle L1 caches.  Like everything in our business, it is the 
science of tradeoffs: you decide on a latency you can live with and then make 
it as big as you can and still have that latency.  If too many cache misses for
that size, i.e., If it is not big enough, see what happens if you increase the 
latency by one cycle.  Simulate the different choices and pick the size/latency 
that gives the best results. 

L2 is usually driven by how much you can actually fit on the chip, given 
everything else you want to put on the chip.  Then you calculate the latency 
for that.

I remember the Alpha 21164, that had a 96KB L2 that was 3-way set associative.
People thought that was a strange beast.  The *real* answer: they did not have
enough room for a 4-way, 128KB L2 cache, and they were sure not going to build
a 64KB cache and waste all that space on the chip.

Since the L1 will take more than one cycle, you don't want to have to wait for
one access to complete before starting the next (pipelining, sort of).  So you
play some tricks here.  Simplest one is interleaving, where you can start an
access to a different bank while the previous access is still in progress.

In my group, we are trying other techniques, like caching some stuff in one
part of L1 and other stuff in another part of L1, and only accessing the
"right" one.  For example, when you look at an instruction, you can tell if
the memory reference is stack based or heap based.  Cache the stack in one
partition, cache the heap in the other.  Make both accesses simultaneously.

Hope this helps.

Yale Patt