Dropcam is a San Francisco, CA startup that provides video monitoring hardware and software so customers can view high-definition video from iOS and Android devices or the Internet. Consumers and small businesses can use Dropcam’s video platform to monitor homes, offices, or pets. The company offers a free, real-time streaming service as well as Cloud Video Recording (CVR), a subscription service that gives users the ability to review stored footage over a 7 or 30-day period and make clips. Dropcam launched in 2009, and in 2013, received $30 million Series C funding round, led by Institutional Venture Partners (IVP), and with participation by Kleiner Perkins Caufield & Byers, Accel Partners, and Menlo Ventures.
By 2011, Dropcam was storing up to 100 GB of data per customer each month. Initially, the company developed its own storage solution, which proved to be time consuming. “Storage was our biggest issue,” says Greg Nelson, Vice President of Software Engineering. “We had to spend time architecting our storage solution, which made it difficult to scale.” Additionally, Dropcam was renting services from a managed hosting provider and it would take close to 24 hours to onboard a new server.
While Dropcam considered other cloud providers, the company chose Amazon Web Services (AWS) because of cost and the maturity of AWS solutions. “We need to scale horizontally and can’t afford single points of failure. The storage capability of Amazon Simple Storage Service (Amazon S3) and the elasticity of Amazon Elastic Compute Cloud (Amazon EC2) were the key reasons we chose AWS,” says Nelson.
To avoid service interruptions, Dropcam migrated to AWS in stages. In the first phase, Dropcam modified its software for the cloud and ran its platform on both AWS and its other hosted environment. After migrating camera feeds to AWS, Dropcam switched customers over to AWS at the end of each subscription period. Nelson relates, “Overall, the transition took about 1½ months, without any impact to our customers.” Dropcam took advantage of AWS Support, Developer level for help configuring Amazon EC2 instances.
Dropcam runs video streaming and storage servers on Amazon EC2 and Amazon S3, and uses Amazon DynamoDB to scale and maintain throughput. “DynamoDB grows with the number of cameras that are connected to the service,” says Nelson. “Throughput is very steady as cameras come online. Growth in the amount of writes scales linearly with the number of cameras.” Dropcam’s developers don’t have to write queries to horizontally partition databases, and report consistent and low latency with DynamoDB. “With our previous solution, it would take 5 to 10 seconds to load an event on a smart phone or on the web. By using DynamoDB, we reduced delivery time for video events to less than 50 milliseconds,” says Nelson.
For CVR subscribers, the service splits video into blocks that are stored on Amazon S3 and captures metadata for motion detection and image analysis. Dropcam’s engineers used Java and Scala to write in-house software to process and store video on Amazon S3 and to connect smartphones and computers to servers. When a user watches a live feed, the server acts as proxy for footage coming in from the camera and sends it out to the viewer. To watch stored footage, Dropcam reconstructs the video stream from the blocks stored in Amazon S3.
Dropcam uses a Python web framework for Dropcam.com and the API servers used by iPhone and Android applications. The stack includes a Postgres database running on an Amazon EC2 instance with Amazon Elastic Block Store (Amazon EBS). All services run in multiple Availability Zones in the US East (Northern Virginia) Region. Figure 1 demonstrates Dropcam’s environment on AWS.
Figure 1: Dropcam Architecture on AWS
By running on AWS, Dropcam can use scripts to automate AWS services and can scale horizontally by using Amazon S3 to split data into partitions. “We would have to hire an operations staff to scale if we kept using our previous solution,” says Nelson. “We’re a small start-up company focused on building software for the user. Leaving operations, storage, and servers to AWS helps us do that.”
Using Amazon EC2, Dropcam can add compute capacity without a 24-hour delay. “We’re running about 125 instances now and we launch new instances each week,” Nelson relates. “Using AWS, we just run a script and a new server is ready in a couple of minutes.”
Additionally, Dropcam developed a computer vision framework that uses algorithms to process streams of inbound data in real-time. If a change occurs outside of specified parameters, it’s flagged as interesting and alerts the user. For example, a user would receive an alert if there were unexpected movement in the home. The Dropcam team originally used a Postgres database for analysis, but found that the database couldn’t scale to meet their requirements, and switched to DynamoDB and Amazon EC2. “DynamoDB provides a fast and reliable data store that we can rely on to scale with the application,” says Nelson. “DynamoDB is so fast, we don’t have to use memcache to alleviate database load.”
Dropcam has a continuous service model and takes advantage of Amazon EC2 Reserved Instances (RIs) to alleviate costs. “We have a consistent workload and using 3-year Heavy Utilization RIs matches our use case,” says Nelson. “The discount ends up being around 67 percent, which is a big win for us.”By running on AWS, Dropcam can use scripts to automate AWS services and can scale horizontally by using Amazon S3 to split data into partitions. “We would have to hire an operations staff to scale if we kept using our previous solution,” says Nelson. “We’re a small start-up company focused on building software for the user. Leaving operations, storage, and servers to AWS helps us do that.”
To learn more about how AWS can help your web applications needs, visit our Web, Mobile, and Social details page: http://aws.amazon.com/web-mobile-social/.
