3.03 Do caching controllers really help?
Description
This item is from the PC Hardware FAQ, by Willie Lim and Ralph Valentino with numerous contributions by others. (v1.25).
3.03 Do caching controllers really help?
[From: backbone!wayne@tower.tssi.com (Wayne Schlitt)]
The short answer, is that if you are using a multi-tasking operating system with a good memory manager, caching controllers should be ignored. If you are running DOS or Windows, then *maybe* they will help, but I am not sure that they are a good buy.
There are lots of people who have said "I put a caching controller in my computer, and it runs faster!". This is probably true, but they never have measured the speed increase compared to putting the same memory into main memory instead. More importantly, the caching controllers cost more money than non caching controllers, so you should be able to add _more_ main memory instead of buying a caching controller.
The following is a shortened up version of a much longer article. If you want a copy of the longer article, send me email at "wayne@cse.unl.edu".
*** Why a multi-tasking operating system?
A multi-tasking operating system can allow the application to continue immediately after it does a write, and the actual disk write can happen later. This is known as write behind. The operating system can also read several blocks from the file when the application requests just part of the first block. This is known as read ahead. When the application requests the block later on, the block will already be there and the OS can then schedule some more reads.
A multitasking operating system is required because these operations can cause interrupts and processing when control has been given back to the application.
Basically, operating systems such as DOS, MS-Windows, MacOS and such do not allow true preemptive multitasking and can not do the read a heads and the write behinds. For these systems, the latency of a disk drive is the most important thing. The application does not regain control until the read or write has finished.
*** The controller can't speed up the disk.
Remember, the bottleneck is at the disk. Nothing that the controller can do can make the data come off the platters any faster. All but the oldest and slowest controllers can keep up with all but the newest and fastest disks. The SCSI bus is designed to be able to keep *several* disks busy without slowing things down.
Speeding up parts of the system that are not the bottleneck won't help much. The goal has to be to reduce the number of real disk accesses.
*** First, isn't the caching controller hardware and isn't hardware always faster than software?
Well, yes there is a piece of physical hardware that is called the caching controller, but no, the cache is not really "in hardware". Managing a disk is a fairly complicated task, complicated enough that you really can't implement the controller in combinatorial logic.
So, just about all disk controllers and for that matter all disk drives have a general purpose computer on them. They run a little software program that manages the communication between the main cpu and the disk bus, or the disk bus and the disk. Often this cpu is put in with a bunch of other logic as part of a standard cell custom chip, so you might not see a chip that says "Z80" or such.
So, we are really not comparing "hardware" with "software", we are comparing "software on the controller" with "software on the main cpu".
*** Ok, why can the OS win?
Assume that you have a bunch of memory that you can either put into main memory and have the OS manage the cache, or put on a caching controller. Which one will be better? Let us look at the various cases.
For a cache hit you have:
If the OS does the caching, you just have the OS's cache checking latency.
If the card does the caching, you will have the OS's cache checking latency, plus the I/O setup time, plus the controller's cache checking latency, plus you have to move the data from the card to main memory. If the controller does DMA, it will be taking away from the memory bandwidth that the main CPU needs. If the controller doesn't have DMA, then the main CPU will have to do all the transfers, one word at a time.
For a cache miss, you have:
If the OS does the caching, you have the OS's cache checking latency plus the set up time for the disk I/O, plus the time it takes for the disk to transfer the data (this will be a majority of the time), plus the cost of doing either the DMA or having the CPU move the data into main memory.
The caching controller will have all of the above times, plus it's own cache checking latency.
As you can see, the caching controller adds a lot of overhead no matter what. This overhead can only be offset when you get a cache hit, but since you have the same amount of memory on the controller and the main cpu, you should have the same number of cache hits in either case. Therefore, the caching controller will always give more overhead than an OS managed cache.
*** Yeah, but there is this processor on the controller doing the cache checks, so you really have a multi-processor system. Shouldn't this be faster than a single processor? Doesn't this allow the main cpu to do other things while the controller manages the cache?
Yes, this really is a multi-processor system, but multi-processors are not always faster than uni-processors. In particular, multi-processor systems have communication overhead. In this case, you are communicating with the controller using a protocol that is fairly expensive, with outb instructions and interrupts and such. The overhead of communicating with this other processor is greater than the overhead of just checking the cache on main cpu, even if the main cpu is very slow.
The multi-processor aspect just doesn't help out when you are talking about managing a cache. There is just too much communication overhead and too little processing for it to be a win.
*** Ok, but couldn't the caching controller do a better job of managing the cache?
Both the controller and the OS are going to be executing a piece of software, so in theory there isn't anything that the slower cpu on the controller can do that the OS can't do, but the OS can do things that the controller can't do.
Here are some of the things that the OS can do better:
* When you read a block from a file, the OS can read several more blocks ahead of time. Caching controllers often will read an entire track in order to simulate this file read a head, but the rest of the file isn't always on the same track, only the OS knows where the blocks are really going to be at. This can lead to wasted time and cache memory reading data that will never be used.
* In order to improve file system reliability, some writes _must_ complete immediately, and _must_ complete in the order that they are given. Otherwise, the file system structures may not be left in a coherent state if the system crashes.
Other writes can be completed as time is available, and can be done in any order. The operating system knows the difference between these cases and can do the writes appropriately.
Caching controllers, on the other hand, don't know if the write that it was just given _must_ be written right away, or if it can wait a little bit. If it waits when it shouldn't, you are risking your file system and data.
* Sometimes, you want a large disk cache if you are accessing lots of data off the disk. At other times, you want a small disk cache and more memory left to programs. The operating system can balance these needs dynamically and adjust the amount of disk cache automatically.
If you put the memory on a caching controller, then that memory can _only_ be used for disk caches, and you can _never_ use more. Chances are, you will either have too much or too little memory dedicated to the cache at any give time.
* When a process closes a file, the operating system knows that the blocks associated with that file are not as likely to to be used again as those blocks associated with files that are still open. Only the operating system is going to know when files are closed, the controller won't. Similar things happen with processes.
* In the area of Virtual Memory, the OS does an extremely better job of managing things. When a program accesses a piece of memory, the CPU will do a hardware level check to see if the page is in memory. If the page is in memory, then there will basically be no delay. It is only when the page isn't in memory that the OS gets involved.
Even if all of those extra pages are sitting in the caching controller's memory, they still have to be moved to main memory with all the overhead that that involves.
This is why dynamic caches vs program memory is so important.
*** What is the "Memory Hierarchy" and how does this relate to caching controllers?
The basic idea of a memory hierarchy is to layer various types of memory, so that the fastest memory is closest to the cpu. Faster memory is more expensive, so you can't use only the fastest type and still be cheap. If a piece of data isn't in the highest (fastest) level of the hierarchy, then you have to check the next level down.
In order for a memory hierarchy to work well, you need to make sure that the each level of the hierarchy has much more storage then the level above it, otherwise you wont have a high hit rate.
The hierarchy on a 486 goes something like this:
8 regs << 8k on chip cache << 256k off chip cache << main memory << disk
If you are going to put something between main memory and disk, it needs to be much larger than main memory in order for it to be effective.
*** What about all these neat things that a caching controller can do such as elevator seeking, overlapping seeks with reads and writes, scatter/gather, etc...
These are nice features, but they are all done by either the OS or a good SCSI controller anyway. None of these things are at all related to supporting the cache, so you shouldn't buy a caching controller for just these features.
*** Ok, you have talked about things like Unix, OS/2 and Windows NT, but what about DOS and MS-Windows?
Well, here things get a lot grayer. First, older versions of DOS have notoriously bad disk cache programs. Since neither DOS nor MS-Windows are preemptive multi-tasking systems, it is much harder to do read ahead. Also, since DOS/MS-Windows users are used to being able to power off their computers at any time, doing write behind is much more dangerous. DOS and MS-Windows also can crash much easier than these other OS's, so people might reboot for many reasons.
Caching controllers usually leave the hard disk light on when they have data that hasn't been written out, and people don't usually power their computer off until that light goes out. This lets the controllers do write behind fairly safely. (But you can still loose power, so this isn't risk free.) They also do crude read a heads by prereading entire tracks.
DOS also runs in real mode and real mode can only access 640K of memory. This mean that a disk cache can be real helpful. Unfortunately, to do a software based disk cache, the cpu has to be switched into protected mode in order to access memory beyond the 640K boundary and then you have to switch back into real mode. Intel, however forgot to make it easy to switch back to real mode. All in all, this switching back and forth ends up being real expensive. This _might_ be more expensive than just using a caching controller, I don't know.
So, it is possible that if you configure DOS to not use a cache, and get a caching controller, then you might be a head. I really don't know much about this area. I have not done any real timings of this.
*** So, when would you ever want to buy a caching controller?
The answer is not too often, but there are a few cases that I can think of:
* You have filled up all your SIMM slots on your motherboard and in order to add more memory you would have to throw some out. This is a real shaky reason. You can always sell your old memory, or move it to another computer. The jump from 4 1MB SIMMs to 4 4MB SIMMs is large, but you will be much better off in the long run with more main memory.
* You have maxed out your memory and you need it all for programs and data. If you can't put any more memory on the mother board, then you don't have many choices.
* If you have a bunch of slow (100ns-120ns) memory left over from say a 286 or something and you can't use it on your motherboard because it is too slow, then maybe adding it to a caching controller will help. Be careful however, if your hit rates on the caching controller are too low, then you may be just adding overhead without getting any benefits.
* If you are stuck with a bad OS because that's what your applications run on, then you might be better off with a caching controller.
*** What about those disk drives that come with caches, are they bad too?
Don't confuse caching disk controllers with cache on disk drives. The latter is actually useful. The little cpu on the disk drive has to read every byte that comes off the disk in order to see when the sector that you are interested in has come under the heads and to do any error detection and correction. The disk also has to have buffers in case the bus is busy, and to sync up the speeds of the bus and the heads.
Since all this data is going though the cpu on disk drive and you have to have a buffer anyway, just making the buffer larger and saving the entire track is an easy win. Saving a couple of the most frequent tracks is also a win.
Most of these caches on the disk drives are fairly small (64k-256k), and a single memory chip will give you about that amount of memory anyway, so you aren't wasting many resources. This also allows the OS to always assume that interleaving is not necessary to get full disk throughput, even if it does a fair amount of processing between disk requests.
Continue to:
My Books
