Lightning Talks

Lightning Talks are back-to-back five-minute presentations on just about anything.

Managing Bare Metal @Spotify

Drew Michel, Spotify

At Spotify, the way we have provisioned and managed servers has seen many iterative improvements. The latest, is a platform that enables squads to fully manage the lifecycle of a server. I’ll go over why this is important for a movement we call "Ops in Squads."

Drew Michel is an SRE at Spotify focusing on providing engineers with a self-service platform for provisioning bare metal hosts.

Don't Fear the Rise of the Machines (or) Why You Can't Scale Ops by Cloning John Connor

Philip Fisher-Ogden, Netflix

We all know what machines are better at than humans and vice versa. Then why do we over-rely on humans to solve problems in production? I think we're afraid of the machines taking over. Once we get over that fear we can move to more scalable approaches to operations. Machines can be programmed to automatically shed load, engage fallbacks, and gracefully degrade. Insight tools can gather forensic data from affected nodes, search for deadlocks and memory leaks, and perform correlation analysis to identify probable causes for the outage. By the time a human gets paged the only thing left to do should be higher-order thinking - why did things fail, not gathering signals and twiddling knobs to try and recover, all while racing the "time to recovery" clock.

John Connor might have been able to fend off Skynet's attempt to end humanity by himself, but I say let's accept the rise of the machines. They’ll do what they do best and we'll do the same. Sure, it might lead to the eventual end of humanity, but until then I'll be able to sleep through a few more pages without production falling over.

Philip Fisher-Ogden, Director of Engineering @ Netflix, ensures "click play" works every time.

Near-line Processing with Apache Samza

Jon Bringhurst, LinkedIn

Apache Samza is a near-line stream processing framework. This lightning talk will briefly cover an example use case. In addition, we'll also talk about how Samza fits into our overall data pipeline here at LinkedIn.

Jon Bringhurst is an SRE for Samza, Kafka, and Zookeeper at LinkedIn.

A Half-Petabyte NAS Using Commodity iSCSI Storage and OpenSource Tools

Benjamin O'Connor, TripAdvisor

At TripAdvisor we built a central storage system for data backups, near-line log aggregation, and general shared storage needs. Clients are mostly NFS and Rsync, and data is aged off to S3 after 5 months. We opted against vendor solutions from EMC/NetApp, etc. and built using commodity hardware and open source solutions. Cheap Dell iSCSI chassis, a couple of linux servers, 10gb ethernet networking, BTRFS, NFS, Rsync and CentOS come together to make the system work. Several challenges with such a huge BTRFS filesystem and making something like this work reliably and consistently without vendor support were encountered. We achieved significant cost savings at the expense of some reliability and administration/maintenance time.

Benjamin O'Connor is currently a Technical Operations Engineer at TripAdvisor with over 15 years experience working on everything from large academic systems (UIUC, MIT) to video game backends (Rock Band, Dance Central, Second Life).

Benchmarking TLS With IPython

Chris Niemira, AOL

The talk is about using IPython as a foundation for running distributed performance tests and analyzing the results. I use TLS benchmarking as an example and show our general method for evaluating the performance of hardware acceleration for different cipher suites. The talk touches on a number of topics, but is fundamentally about using a powerful tool in a novel way. I believe it’s interesting for this audience because I don’t see much about either security or benchmarking on the program.

Chris Niemira is a Senior Site Reliability Engineer at AOL and is responsible for helping maintain the performance and availability of AOL's entire portfolio of products.