System Log Analysis Using BigQuery

Gustavo Franco

Authors:

Abstract:

This talk presents System Log Analysis using an OLA P-based and in-the-cloud system, BigQuery, which is heavily optimized for reads. BigQuery is a Dremel-based SaaS that, when combined with a system logger such as syslog-ng, a tool to convert log entries to CSV, and an uploader, which will be discussed in detail in this talk, allows any DevOp to scale her log analysis needs in an elastic manner.

Gustavo Franco, Google Inc.

Gustavo Franco is the Lead Site Reliability Engineer for the following services, which are part of the Google Cloud Platform: Google Compute Engine, Cloud Storage, and BigQuery. He has been a Debian Developer for more than 10 years. His career spans over 12 years of DevOps-related work including the FIFA World Cup online broadcast, migrating Google to Goobuntu, and more.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX

@conference {255720,
author = {Gustavo Franco},
title = {System Log Analysis Using BigQuery },
year = {2012},
address = {San Diego, CA},
publisher = {{USENIX} Association},
month = dec,
}

Download

View the slides