Sunday, July 5, 2009

Buyer beware when vendors talk Storage Utilization.

Words games are not the sole prerogative of the politician. Our marketing colleagues have a tendency to push the envelope with the latest example being some folk’s interpretation of the term Storage Utilization.

When I think of storage utilization I tend to think of the capacity available to the user to perform useful work after all system overhead has been considered. I do not consider the capacity required to protect or manage stored data as user available storage. So when data is being protected with mirroring for example, storage utilization is significantly less than when RAID 5 or RAID 6 is implemented.

Date Protection Scheme...................................Storage Penalty Mirrored...................................................................50%
RAID 5......................................................................17% (5+1 Raid Group)
RAID 6......................................................................33% (4+2 Raid Group)

Except in the case of mirroring the bigger the RAID group the smaller the penalty which is one of the configuration compromises, now add the system OS etc, you get the point.

So when you see eye catching guarantees of Storage Utilization make sure you understand how this utilization is being defined. You may be surprised.


Lucas said...

Hi Bill,
You raise a good point about how marketing terminology can cloud an issue. As you note, an example of this is the term “utilization” and your proposal that “utilization” refers specifically to the space available for use after all storage overhead has been accounted for.

As your detail of data protection schemes’ impact on utilization rates demonstrates, there is significant variation in storage overhead, depending on data protection setting. This makes it challenging to pinpoint a standard “best case” rate for utilization, as it can depend heavily on individual use case, data type, etc.

Some major vendors steer clear of discussing utilization, which isn’t surprising in that utilization rates for traditional storage are notoriously low. Isilon has long been candid about the utilization rates of our storage systems and has gone as far to guarantee at least 80% utilization, which we define as the space available for all data - including data protection settings. This is because variance in data protection schemes makes it difficult to guarantee utilization apart from data protection without necessitating many pages of specifications, guidelines, and legal commitments related to cluster size, protection setting, data type, workload, etc. Therefore, we chose to make the guarantee as simple for customers as possible.

From Isilon’s perspective, not only is 80 percent utilization the minimum rate we guarantee, but because of the minimal storage overhead necessitated by our unique single file system - combining the volume manager, RAID, and file system into a single operating system layer - and N+ data protection scheme, the majority of our customers are operating Isilon systems at just under 90% real usable capacity. Let me explain. Isilon does not employ RAID in the traditional sense, e.g. based on a HW controller and a defined amount of RAID disk, parity drives, and spares. Instead, Isilon sets data protection at the node level, leveraging the intelligence of OneFS to “wide stripe” the data set vertically across all the nodes. For example, take a small Isilon cluster example of five nodes at N+1 (RAID5 equivalent) - protection overhead is 20%. As additional nodes are added the system automatically continues to widen the stripe width and restripe all the data in the background and actually improves the usable capacity. For example, if you scale a cluster from five to 10 nodes at N+1, at 10 nodes your data protection overhead is only 10%, leaving nearly 90% as true “usable storage” or “available capacity” (note: a few percentage points must be reserved for OneFS overhead). While for most traditional vendors their “best day is their first,” Isilon’s usable capacity only increases as you grow system capacity, a significant improvement over traditional utilization rates of 50-60%.

Utilization vs. usable capacity is an important discussion to have and an issue we find customers paying more and more attention to when they evaluate storage strategies. Thanks for bringing it up.

BillM said...

Hi Lucas: The point I was trying to make is that it should be clear to the user community what storage is actually available to solve their storage problem and that vendors do not represent reality in a consistent format. This obligates the user to complete a bit of detective work to understand if the storage he is purchasing matches the data growth he id predicting.

I sense a little smoke and mirrors but it could be my ignorance of your technology but I am not buying the story. Striping vertically or horizontally you still have the parity penalty. Agreed, the larger the strip the less the overhead penalty becomes as a % of available capacity but the downside but you are subjected to a greater chance of disk failures overlapping, longer rebuild times and hence a greater chance of data loss.

Happy to listen if I am off base.