f4: Facebook's Warm BLOB Storage System

Sanjeev Kumar, Facebook

Friday - 10-31-14, 4:00PM-5:00PM
CIC Panther Hollow

Abstract

Facebook's corpus of photos, videos, and other Binary Large OBjects (BLOBs) that need to be reliably stored and quickly accessible is massive and continues to grow. As the footprint of BLOBs increases, storing them in our traditional storage system, Haystack, is becoming increasingly inefficient. To increase our storage efficiency, measured in the effective-replication-factor of BLOBs, we examine the underlying access patterns of BLOBs and identify temperature zones that include hot BLOBs that are accessed frequently and warm BLOBs that are accessed far less often. Our overall BLOB storage system is designed to isolate warm BLOBs and enable us to use a specialized warm BLOB storage system, f4. f4 is a new system that lowers the effective-replication-factor of warm BLOBs while remaining fault tolerant and able to support the lower throughput demands.

Bio

Sanjeev Kumar is a Director of Engineering at Facebook Inc. where he has worked on a wide variety of systems projects. These days, he focuses on three areas: (a) infrastructure for messaging product; (b) exabyte-scale storage systems (for photos and videos, for cold storage, and for data warehouse); and © a set of core systems used to build all the custom services at Facebook. Prior to Facebook, Sanjeev was a Senior Staff Researcher at Intel Corp. where he investigated software, hardware, and applications for many-core architectures. Sanjeev holds a Ph.D. From Princeton University, a M.S. from Indiana University, and a B.Tech from Indian Institute of Technology, Chennai (all in Computer Science).