Tuesday, September 22, 2009

SSD and HDD; A Symbiotic Integration of Technologies.

In a recent white paper “Flash and HDD - Symbiosis or Survival of the Fittest” Tom Coughlin presented an interesting case that Flash Storage and HDD technologies are complementary and not competitive, at least from a consumer electronics perspective. The argument he presented suggests that the growth of both Flash and HDD’s are interdependent with each supporting and promoting the growth of the other. However, does this hypothesis hold true in the enterprise class data storage environment?

Over the past few years solid state storage (SSD) has regained its once fashionable standing as a legitimate data storage option encouraging the somewhat familiar refrain by many industry pundits that HDD’s were dead and within the next 5yrs or so will be replaced by solid state storage. Excuse my bluntness but hogwash. Yes SSD’s will become prevalent and yes the technology will command a significant position in the storage hierarchy, “Tier 0” if you will but the assertion that it will eradicate HDD’s is reminiscent of the many online bigots who 20 years ago declared that tape was a member of the technology walking dead. Last time I looked however, the tape market was still a viable storage option. So let me repeat, hogwash!

As a proof point check out a recent white paper that appeared in the Intel Technical Journal Volume 13, Issue 1; “Solid State Drive Applications in Storage and Embedded Systems”. This paper is a collaboration between Dane Nelson of Intel and Sam Siewert of Atrato and presents an interesting approach to intelligent SSD/HDD integration that results in an autonomic application aware architecture that is characterized by exploiting the performance of SSD with the cost balance of HDD’s, orchestrated by the block level, activity and access profile, of data.

Some of the key features of the architecture discussed in the paper include:

  1. A hybrid solution that can scale to petabytes.
  2. Ability to profile I/O access to petabytes of data using megabytes of RAM.
  3. Ability to create an SSD VLUN along with a traditional HDD LUN with the same RAID features so that file-level tiers can be managed by applications.
  4. Ability to create hybrid VLUNs that include HDD capacity and SSD cache with intelligent block management to move most frequent accessed blocks between the tiers.
  5. Ability to create hybrid VLUNs that are composed of HDD capacity and are allocated SLC SSD ingest FIFO capacity to accelerate writes that are not well-formed and/or are not asynchronously and concurrently initiated.
  6. Ability to create hybrid VLUNs that are composed of HDD capacity and allocated RAM egress FIFO capacity so that the black-end can burst sequential data for lower latency sequential read-out.

This I suggest is an example of the intelligent and thoughtful integration of storage technologies that extends Tom’s original consumer electronics based hypothesis to enterprise class storage, that Flash and HDD technology is symbiotic and not competitive.


Thomas said...

Thanks for your comments on my article, written with Jim Handy, based upon a suggestion by Roger Hoyt. Your extension, I think, is well placed. There is no universal memory that will replace all other memories. They each have their place in filling the insatiable demand of humans for more digital storage, more digital content.

If enterprise SSDs and other solid state devices permit faster data creation and use that will drive the need for much less expensive HDD storage as a library or long term repository of that data. Likewise the existence of low cost storage creates a need to share or create that storage more rapidly. This is a very close analogy to the consumer argument in the white paper.

Jim Handy said...

It's interesting to reflect back on other technologies that were going to "Take over the world!" In retrospect, those technologies that succeeded often did so in a way that was accretive to existing products. NAND flash and SSDs are clearly doing this today.

When we compiled the data in our joint report on Storage in Consumer Electronics Tom and I found that flash and HDDs were very complementary, with digital cameras, MP3 players, and other applications driving increased demand for HDDs and with cheap HDD petabytes enabling flash-based devices to grow. Roger suggested for us to highlight that in the white paper.

In another report Objective Analysis found that enterprise SSDs are not so much displacing HDDs in data centers as they are enabling data center performance growth which should blossom into more opportunities for HDDs.

There is a lot of symbiosis between these technologies.