Around late 2008, I came across my first real exposure to a customer who wanted to scale their read-write RDBMS. It was a very interesting problem. At the time I didn’t evaluate if RDBMS was an appropriate data store, but rather approached it from “let’s make this work”. A complex PoC was pulled together that relied on a Peer-to-Peer Replication topology, that was super fragile. Since then I have had several opportunities to work with customers chasing this objective.
In most cases the sought after distributed data store is achieved by leveraging a NoSQL system. Of course that introduces other considerations. At one point the team I was working on came up with this sharding pattern, which I’ve blogged about on at least one occasion, however it was predominantly about app and web server scale. The data tier was simply, how do I say this succinctly, let me try “atrocious”. It lacked real depth, and real versatility in terms of how it was to be cared for by DBAs, nor did it really consider the SDLC aspects of developers that needed an ELASTIC scale solution for the RDBMS tier.
I spent the next 2 years working on a framework that took the lessons from multiple engagements and placed into a consumable package. It took a great deal of work, and several failures plus a group of excellent colleagues to produce the package. In particular Silvano Coriani and Jeff Odell, with some great input in the early days by Josh Handel. The Channel9 video presentation that can be found at http://bit.ly/Tju0aB, is where you get to see some good features of the package. The articles I’m listing below is the outcome of many months of all that goes into content creation. It is a great supplement to the video presentation.
Successfully achieving elastic scale is no trivial task, don’t be led to believe otherwise. It is without a doubt the most fun, challenging and exciting journey with HUGE rewards.
- Optimizing Data Elasticity Through Database Sharding – microsoft.com/en-us/library (http://bit.ly/1BiY1YC)
- Database Sharding Basics –microsoft.com/en-us/library (http://bit.ly/1CU8DkL)
- Data Elasticity Principles –microsoft.com/en-us/library (http://bit.ly/1Hhipx0)
- Data Isolation and Container Management –microsoft.com/en-us/library (http://bit.ly/1EM5KUN)
- Data Partitioning Options in Sharding –microsoft.com/en-us/library (http://bit.ly/1I0Uy2V)
- Data Modeling Conventions and Best Practices for Sharding –microsoft.com/en-us/library (http://bit.ly/1Hhilx4)
- Data Modeling for Rapid Sharding –microsoft.com/en-us/library (http://bit.ly/1x6o0TU)
- Sample Sharded Data Model –microsoft.com/en-us/library (http://bit.ly/13UA2E4)
- Data Distribution –microsoft.com/en-us/library (http://bit.ly/1D6pD4C)
- Database Sharding Glossary –microsoft.com/en-us/library (http://bit.ly/1Ad4vsx)