It seems that no matter where you look in IT, you cannot escape the phrase “deduplication” these days.   According to Gartner, 47% of the respondents to a survey conducted over the summer of 2011 ranked data growth in their top three challenges. With spending returning to more normal levels after a couple of down years because of the economy, 62% said they would plan to expand hardware capacity at existing data centers by the end of 2011, while 30% would plan to build entirely new data centers.

Deduplication itself is becoming a critical “must have” for IT professionals, but it must be handled with care.  A bad strategy can quickly cause IT infrastructure costs to increase. With proper planning and implementation, a good deduplication strategy can:

• Greatly reduce cost while improving storage and backup performance
• Reduce backup windows
• Lower data center costs

There are two critical points to keep in mind when determining whether to implement a storage or backup solution that uses deduplication:

• Data that changes often is a bad candidate for deduplication
  Deduplication works by storing data one time.   When the data is changed, only the changed data is stored on disk.  If you attempt to use a deduplication solution with data that is constantly changing, you pay a premium for the deduplication software but lose out on the benefits since all of the changing data still has to be stored.
• Deduplication can consume a large amount of system resources
  There are many different options when it comes to deduplication processing.  It is important to keep CPU load in mind when determining which strategy to implement.   Whether you use your existing servers to deduplicate data or choose a dedicated appliance to handle the deduplication processing, the resource load needs to go somewhere and will require resources.