Laurent posted this in Tech at April 19th, 2012
At Percolate, we make use of data, a lot of data. By itself, data is pretty useless. What adds value is the ability to get answers from it – that is, turning data into information.
A SQL dump alone is worthless. But as soon as you load it on your shiny PostgreSQL server, you can start executing queries and getting results out of your data. This is because the data is structured, relational and the server is optimized to give you an answer in real time – thousands of answers per second.
Living in a perfect world, your server will always be fast enough and available. But we don’t, life is interesting, and hiccups happens.
\
In operations, the first thing you learn by experience is that you never ask yourself if a server will crash, but when. Your job is then to have a plan, automated or not, for what you will do when the crash happens.
Any freshly installed and configured server looks pretty solid, but according to Wikipedia, the uptime record is 6 years. It might be longer, but the reality should not be far from it: Any server will crash some day.
The Percolate team is growing (and we are hiring), but as an early startup we all must do a lot of things. Great! but this also means that we need to make choices on how we will spend the hours (never enough) in our days. Operating a bunch of MySQL servers is something we needed to do, as we wanted to ask questions of our data 24/7.
And we did it for months! Spending precious hours doing what every other sysadmin did. And at some point, we decided that our value was elsewhere: we ended up using AWS RDS as a replacement for our MySQL servers.
Before choosing RDS, we took into consideration a lot of factors.
Price
The cost of an RDS instance is the same as an EC2 instance of the same type. And you can scale up and down any server transparently: increasing capacity or lowering cost.
Time
Configuration, backup, monitoring, and all the work the AWS team did is all included, and that’s time that your engineers can spend on something else.
Independence
RDS are packaged SQL servers. Dump your data and go anywhere you want.
Scalability
We are limited to what’s available in RDS: Vertical scalability and asynchronous master slave replication (by adding read replicas). It’s not a lot but it’s click & play. Anyway, we never thought an SQL server would be scalable, it just gives us some time to scale horizontally.
Visibility
All the features coming with RDS servers exist, are used, and we know we can use those whenever we want. With in-house operations, you need to plan the implementation of those tools. More immediate options means more velocity for your company.
In 2012, a startup won’t succeed because their MySQL cluster is the most optimal ever, but because the project it builds is useful to some. We prefer to focus on what makes us useful.
In the end no one at Percolate wants to go back manually riding those MySQL horses. Of course we can’t optimize those servers as much as the big guys do. But the relational data of our platform is not large enough to be more than what a bunch of MySQL servers can deliver. And when we do get to that point, we’ll stop scaling vertically and jump on the NoSQL bandwagon, using the best of both worlds.
But that’s another story…
Subscribing to the Percolate blog will keep you updated on our insights, trends and what is brewing around the office.