Instagram Engineering Team Slipstreams Web 2.0 Technologies

One of the world’s most popular Smartphone photo sharing apps on the iPhone is Instagram.
With over 15 million downloads , this free app is highly addictive and is features even faster start-up times under Version 2.0.6.
Since we offer highly affordable web hosting on Linux servers we are always interested in the behind-the-scenes engineering that goes into achieving terminal velocity on these mobile apps, especially since we discovered that the Instagram team uses a Ubuntu Linux 11.04 on the Amazon EC2 to power their systems.
The Instagram team only has three engineers and has decided that self-hosting is not yet in their interests. They have thus outsourced their requirements, and, considering this point, so should you. By purchasing dedicated servers or cloud hosting on the WestHost platform you can scale quickly at reduced costs.
The Instagram team works off these three principles:

• Keep it very simple
• Don’t re-invent the wheel
• Go with proven and solid technologies when you can

Their engineering team confirms that each request made to an Instagram server goes through load balancing machines using a DNS Round-robin approach.  While effective it does lead to slower DNS updates and they are now pursuing new options using an Elastic load balancer from Amazon.
They use Django to handle application server requests.  And, they apparently now run nearly 25 of them to handle the increased traffic loads.
Not surprisingly they use PostgeSQL databases to power their back-end. This is becoming an increasingly popular DB back-end for forward thinking websites.
“As a quick tip, we’ve found that ‘vmtouch’ is a fantastic tool for managing what data is in memory, especially when failing over from one machine to another where there is no active memory profile already” said Instagram on their website.
“ Here is the script we use to parse the output of a vmtouch run on one machine and print out the corresponding vmtouch command to run on another system to match its current memory status.”
Instagram said that all their PostgreSQL instances run in a master-replica setup using Streaming Replication. EBS snapshotting allows frequent backups of all systems. XFS drives their file systems, which lets them freeze & unfreeze the RAID arrays when snapshotting, in order to guarantee a consistent snapshot (our original inspiration came from ec2-consistent-snapshot.
“To get streaming replication started, our favorite tool is repmgr by the folks at 2ndQuadrant.”
TechCrunch reports that  over 150 million photos have been uploaded to Instagram and it reached those figures nearly 50% quicker than Flickr did when it first started.
Special note: By the way, notice the Instagram Engineering team is using Tumblr to power their blog.  We wrote about Tumblr in a previous blog.  Tumblr seem to be getting more and more popular every day!