Data Elasticity Through Database Sharding

Around late 2008, I came across my first real exposure to a customer who wanted to scale their read-write RDBMS.  It was a very interesting problem.  At the time I didn’t evaluate if RDBMS was an appropriate data store, but rather approached it from “let’s make this work”.  A complex PoC was pulled together that relied on a Peer-to-Peer Replication topology, that was super fragile.  Since then I have had several opportunities to work with customers chasing this objective.

In most cases the sought after distributed data store is achieved by leveraging a NoSQL system.  Of course that introduces other considerations.  At one point the team I was working on came up with this sharding pattern, which I’ve blogged about on at least one occasion, however it was predominantly about app and web server scale.  The data tier was simply, how do I say this succinctly, let me try “atrocious”.  It lacked real depth, and real versatility in terms of how it was to be cared for by DBAs, nor did it really consider the SDLC aspects of developers that needed an ELASTIC scale solution for the RDBMS tier.

I spent the next 2 years working on a framework that took the lessons from multiple engagements and placed into a consumable package.  It took a great deal of work, and several failures plus a group of excellent colleagues to produce the package.  In particular Silvano Coriani and Jeff Odell, with some great input in the early days by Josh Handel.  The Channel9 video presentation that can be found at, is where you get to see some good features of the package.  The articles I’m listing below is the outcome of many months of all that goes into content creation.  It is a great supplement to the video presentation.

Successfully achieving elastic scale is no trivial task, don’t be led to believe otherwise.  It is without a doubt the most fun, challenging and exciting journey with HUGE rewards.

Enjoy It!!

This entry was posted in Scalability and tagged , , , , . Bookmark the permalink.

2 Responses to Data Elasticity Through Database Sharding

  1. Pingback: Cloud Migration References | lgit smartblog

  2. Pingback: Liens utiles Azure – IT Consult

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s