Building Large-scale Production Systems for Latency-sensitive Applications
36In today’s era of increased engagement with technology, the myriad interactive and latency-sensitive applications around us necessitate handling large-scale data quickly and efficiently. This talk focuses on designing and developing production-quality systems with particular attention to improving end-to-end latency and building massive-scale solutions. At large scale, providing low latency becomes increasingly challenging, with many issues around distribution of data and computation, providing load balance, handling failures, and continuous scaling. We explore these issues on a wide range of systems, from a large-scale geo-distributed blob storage system that is running in production serving 450 million users (Ambry), to a stateful stream processing system handling 100s of TBs for a single job (Samza), and a real-time edge computing framework transparently running jobs in an edge-cloud environment (Fluid-Edge).
See more at https://www.microsoft.com/en-us/research/video/building-large-scale-production-systems-for-latency-sensitive-applications/