• What is Fibre Channel, and how is it different from SCSI or ATA/IDE?
  • Why are the hard drives so cheap?
  • What controllers do recommend, and why?
  • What do I need to have a complete, bootable FC system on my computer?
  • What are the limitations, as far as OS's that support fibre channel?
  • Why would I want to use Fibre Channel drives, rather than SCSI or ATA/IDE drives?

  • What is Fibre Channel, and how is it different from SCSI or ATA/IDE?

    Fibre channel is a breakthrough in storage technology. It is a new interface that supports very high data rates as well as many more devices than SCSI or ATA/IDE. It is also a serial interface, and uses a loop topology.

    The high bandwidth interface is one of fibre channel's biggest assets. It uses a signaling rate of 1 gigabit per second. New designs for fibre channel HBA's will operate at 2 or 4 gigabits per second! The use of the serial interface is very logical in that it allows data to be transmitted in an efficient way that reflects the way it is actually read of the disks. The IDE/ATA world may be catching up in the near future with serial ATA drives, but the price tag is likely to stop the average user from making serious use of this technology. In addition, it will take a while for the controllers to get support in Linux, Windows 2000, etc...

    Why are the hard drives so cheap?

    For the Windows NT/2000 or Linux/Solaris/BSD user, there is no time like the present to take advantage of a very interesting phenomenon in the computer world. Fibre channel drives that retail for over $300 for 9G are now available for pennies on the dollar through various "asset recovery" companies, as well as various sellers on eBay. Big IT corporations spent thousands and thousands of dollars to buy fibre channel disks and storage arrays when they first came out. Many of them had drive arrays that were loaded with hundreds of 9.1 or 18.2G drives that they have recently pulled out of their storage arrays and replaced with drives that were 36.4G or larger. Many of these companies have also sold their old drives to "asset recovery" companies for as little as $5-10 each. Consequently, these type of companies can afford to sell these drives with a replacement warrantee for as low as $20 each (sometimes even less) and still stay in business. This gives the home user an opportunity to buy 54G or more worth of disks for under $100. ATA/IDE drives are getting cheaper and faster by the day, but there is still something to be said for the performance difference between a 5400rpm Maxtor IDE drive and 5 Seagate Cheetah's (10,000rpm) in a RAID 0 configuration.

    What controllers do recommend, and why?

    It is my belief that the Qlogic family of fibre channel controllers is the best for the home user. Specifically, the QLA 2000 and 2100. These cards are almost identical in function, however the 2000 works in 32-bit mode only, and the 2100 will work in either 32 or 64-bit mode. I have tried Emulex controllers, as well as Adaptec/JNI and the Qlogic is still my favorite. I can use it in Linux and Windows NT/2000 without having to load any special patches or drivers. These cards feature the 266MIPS RISC processor on board, and support sustained transfer rates of 106.3 megabytes per second (MB/s... not to be confused with Mb/s). These controller cards are available on eBay, and by various AR outfits. People's stock changes all the time, but if you e-mail me and tell me what you're looking for, I can probably refer you to someone who is currently selling them (I keep in touch with several eBay sellers on the market).

    What do I need to have a complete, bootable FC system on my computer?

    Now you have a fast hard drive and a fast controller available. Unfortunately, hooking them up is not as easy as buying one simple $5 cable. Fibre channel drives were built anticipating a "hot-swap" environment. Consequently, Single Connection Assembly (SCA) interfaces were used on this drive. These are different from a regular IDE or SCSI cable, because the drives are designed to be removed from their enclosure quickly and easily. If a drive in a storage array fails, you want to be able to replace it quickly without screwing around with a big cable and separate power connector. Therefore, the hard drive's power, data, and ID information are all on one 40-pin connector. In order to use these drives outside a storage array, the appropriate mating connector must be used, and wired up. Devices that do this are known as "Tester Cards". I believe Adaptec made the first ones for a very limited time when the drive first came out. These allowed people to do tests on the drives on a workbench, without having a storage array available, which at that time would have cost up to $5000 or more. Since then, a number of companies have made such devices, for similar testing purposes. I bought one just to mess around with, and made a few interesting discoveries. I now make my own simplified version o f the Tester Card (aka "Tcard") for sale, for those like me who want to play around with this technology without the aggravation of reverse engineering someone else's adapter. Please take note that when you read this page, it is not a claim that I invented the Tcard, but I did find a connector that could be used to mate with the drive, as well as one that allows me to make these devices by hand, reliably. I have used multiple Tcards to develop storage arrays, wire up multiple drives inside one case, and also, build single external storage units for those that don't want to mess around with lots of cabling. If you have ideas for a custom fibre drive setup, I can help make those ideas into a functional system.

    A basic setup consists of a fibre channel hard drive, a Tcard, and a cable from the HBA to the Tcard. When more than one hard drive is to be used, additional Tcards are necessary (one per drive). The use of addition drives and Tcards does not increase the number of cables going to the HBA. If you are using an Adaptec/JNI or Emulex adapter with a copper interface, you need a DB9M-DB9M cable to go from the HBA to the Tcard. This is not a standard serial cable. It must be wired for fibre channel use. It helps to use twisted pair wire, though it is not necessary over extremely short distances. If you are using Qlogic HBA, and HSSDC-DB9 cable is needed. This is because the Qlogic connector uses an HSSDC connector on their cards with a copper interface. Due to the cost of the HSSDC connector itself, these cables are more expensive than a DB9-DB9.

    What are the limitations, as far as OS's that support fibre channel?

    I have been successful booting off fibre channel drives in Windows NT, 2000, Red Hat Linux, and Solaris x86. Connectcom offers Windows 9x drivers for their HBA's. They are the only ones known to offer drivers for Windows 9x. Depending on how your system is set up, the drives may be detected and accessible in these operating systems, however, they will be stuck using MS-DOS compatibility mode. You will not be able to take advantage of most of the HBA's features using these OS's.

    Why would I want to use Fibre Channel drives, rather than SCSI or ATA/IDE drives?

    The point of this page is not to say that fibre channel drives are cheaper to set up and run than IDE/ATA, but it is my belief the they are the most "bang for the buck". There are definite advantages to using fibre channel storage, especially in the area of flexibility. If you want to throw 8 fibre channel drives in a box and use it for storage, it's easy! The drives daisy chain together, and proper cabling allows you to put the "just a bunch of drives" (JBOD) any where you want, within 30M using copper cable, and within 10kM using fibre optic cable. In addition, you can run 126 drives on one controller. This is far superior to the 15 possible with SCSI, and the 2 per channel used on IDE. Running an 8 drive IDE/ATA JBOD would require 2 dual-channel controller cards, and therefore 2 PCI slots and 2 IRQ's. The other option is to find one of those ATA RAID cards with several controllers on one card, but those get seriously expensive.

    High-speed software RAID setups are possible using IDE/ATA dives, but not without a serious cost of valuable CPU time. Like the fibre channel drives, fibre channel host bus adapters (HBA's) can also be purchased for pennies on the dollar compared to their original purchase price. Asset Recovery (AR) outfits occasionally come up with a lot of New Old Stock (NOS), and you can purchase a brand new controller that once cost over $1000 for about 10% of that. Most fibre channel controllers come equipped with one or more onboard Reduced Instruction Set Code (RISC) processors to handle communications to devices on the arbitrated loop (AL) as well as process the data going to and from the drives. These processors are pretty powerful too. HBA's made in '97 or later typically come with a 64-bit 266 Million Instructions Per Second (MIPS) processors. Comparing it to an Intel CPU is definitely an "apples and oranges" comparison, but an overly simplistic analogy would be to think of it as a 266MHz CPU with the capability to do 64-bit arithmetic. (Intel CPU's with the exception of the Pentium5 / Itanium / IA-64 can only do 32-bit arithmetic.) What does all this mean for you and your computer? The capacity to set up high-speed storage using software RAID, with a minimum of CPU utilization.

    The reason for me describing fibre channel storage as being "the most bang for your buck" is the combination of low price and high performance. I have previously discussed how the regular PC owner can buy drives and controller cards for pennies on the dollar. 10,000 and 15,000rpm fibre channel drives are available. The spindle speed of an ATA/IDE drive is rather slow by comparison. They are commonly available in either 5400 or 7200rpm spindle speeds. Fibre channel drives can some with as much as 4-16MB of cache per drive, while I haven't seen and IDE drives with over 2048k. The increased spindle speed also allows the drives to have faster seek times.

    Have you ever considered what spindle speed and seek time do for the performance of your computer? A fast hard drive can make a huge difference in overall system performance. It is so sad for me to see someone who has just bought a new computer with a 1GHz or faster processor, DDR or rambus, and then put a 5400rpm Maxtor hard drive in it! It doesn't matter if the box says it's an ATA/100 drive! While it may be able to burst at that transfer rate under highly unusual circumstances, think of all the operations your 1GHz CPU could have done in the time you were waiting 5 or more milliseconds then you had to for a disk seek and read! What a waste! Its no shock to me that my Celeron 366 system (which boots off a 9G fibre channel drives that is 10, 000 rpm drive and has 4048k of cache) can boot into Windows 2000 faster than a Compaq PIII 550 with a Quantum Bigfoot.