Online service provider

Jeff Dean: "Achieving Rapid Response Times in Large Online Services" Keynote - Velocity 2014

• Report rights infringement
• published: 25 Jun 2014
• views: 15173

Jeff Dean's keynote from the 2014 O'Reilly Velocity conference in Santa Clara, CA. Today's large-scale web services provide rapid responses to interactive requests by applying large amounts of computational resources to massive datasets. They typically operate in warehouse-sized datacenters and run on clusters of machines that are shared across many kinds of interactive and batch jobs. As these systems distribute work to ever larger numbers of machines and sub-systems in order to provide interactive response times, it becomes increasingly difficult to tightly control latency variability across these machines, and often the 95%ile and 99%ile response times suffer in an effort to improve average response times. As systems scale up, simply stamping out all sources of variability does not work. Just as fault-tolerant techniques needed to be developed when guaranteeing fault-free operation by design became unfeasible, techniques that deliver predictably low service-level latency in the presence of highly-variable individual components are increasingly important at larger scales. In this talk, I'll describe a collection of techniques and practices lowering response times in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception. Some of the techniques adapt to trends observed over periods of a few minutes, making them effective at dealing with longer-lived interference or resource contention. Others react to latency anomalies within a few milliseconds, making them suitable for mitigating variability within the context of a single interactive request. I'll discuss examples of how these techniques are used in various pieces of Google's systems infrastructure and in various higher-level online services. In this talk, I'll describe a collection of techniques and practices lowering response times in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception. Some of the techniques adapt to trends observed over periods of a few minutes, making them effective at dealing with longer-lived interference or resource contention. Others react to latency anomalies within a few milliseconds, making them suitable for mitigating variability within the context of a single interactive request. I'll discuss examples of how these techniques are used in various pieces of Google's systems infrastructure and in various higher-level online services. This talk presents joint work with Luiz Barroso and a number of other colleagues at Google. About Jeff Dean (Google): Jeff joined Google in 1999 and is currently a Google Senior Fellow in Google's Knowledge Group. He has co-designed/implemented five generations of Google's crawling, indexing, and query serving systems, and co-designed/implemented major pieces of Google's initial advertising and AdSense for Content systems. He is also a co-designer and co-implementor of Google's distributed computing infrastructure, including the MapReduce, BigTable and Spanner systems, protocol buffers, LevelDB, systems infrastructure for statistical machine translation, and a variety of internal and external libraries and developer tools. He is currently working on large-scale distributed systems for training deep neural models for speech, vision, and text understanding. He is a Fellow of the ACM and the AAAS, a member of the U.S. National Academy of Engineering, and a recipient of the ACM-Infosys Foundation Award in the Computing Sciences. Watch more from the 2014 Velocity Conference: http://goo.gl/JoMfvo Find out more about Velocity: http://velocityconf.com/ Don't miss an upload! Subscribe! http://goo.gl/szEauh Stay Connected to O'Reilly Media by Email - http://goo.gl/YZSWbO Follow O'Reilly Media: http://plus.google.com/+oreillymedia https://www.facebook.com/OReilly https://twitter.com/OReillyMedia