Sharding for Scale: In the App or in the Network?
Sharding has become a popular means of achieving scalability in application
architectures in which read/write data separation is not only possible, but
desirable to achieve new heights of concurrency. The premise is that by
splitting up read and write duties, it is possible to get better overall
performance at the cost of a slight delay in consistency. That is, it takes a
bit of time to replicate changes initiated by a "write" to the read-only
master database. It's eventually consistent, and it's generally considered an
acceptable trade off when searching for higher and higher scalability.
While the most well-known cases of read/write separation and sharding are
based on geography - east coast versus west coast, for example - there are
other cases where localized sharding has also been put into play with great
It's all about that architecture.
There's a lot of things we do to improve the performance of web and mobile
applications. We use caching. We use compression. We offload security (SSL
and TLS) to a proxy with greater compute capacity.
We apply image optimization and minification to content.
We do all that because performance is king. Failure to perform can be, for
many businesses, equivalent to an outage with increased abandonment rates and
angry customers taking to the Internet to express their extreme displeasure.
The recently official HTTP/2 specification takes performance very... (more)
The Importance of Licensing to Equalize Dev and Production
We’re all aware that dev/test != production environments. While the
software stacks upon which applications are deployed may be (and hopefully
are) the same, there still remains a whole lot of “infrastructure”
(that’s everything else) that isn’t the same. Routers, switches, security
devices, load balancers, caches, and other devices dedicated to ensuring the
secure delivery of applications to hungry consumer and corporate users simply
don’t exist in the dev/test environment. That’s particularly true as
organizations cont... (more)
Scaling things seems like such a simple task, doesn’t it? Open a new
checkout line. Call a new teller to the front. Hire another person.
But under the covers in technology land, where networking standards rule the
roost, it really isn’t as simple as just adding another “X”. Oh, we try
to make it look that simple, but it’s not. Over the years we (as in the
industry ‘we’) have come up with all sorts of interesting ways to scale
systems and applications within the constraints that IP networking places
One of those constraints is that physical (L2) and logical (L3) addresses... (more)
Early (very early, in fact) in the rise of SDN there were many discussions
around scalability. Not of the data plane, but of the control (management)
plane. Key to this discussion was the rate at which SDN-enabled network
devices, via OpenFlow, could perform “inserts”. That is, how many times
per second/minute could the management plane make the changes necessary to
adjust to the environment.
It was this measurement that turned out to be problematic, with many
well-respected networking pundits (respected because they are also
professionals) noting that the 1000 inserts per secon... (more)