"Great sessions, great presenters, great community. I feel like I can make up a year of following news and forums on the subject in a week."
LISA14 Attendee
"No matter if you are dealing with the latest tech or trying to maintain something from the dark ages of the ‘90s, there are people at LISA who are experienced with it. Heck, many of the people who _developed_ this tech attend. This is a great place to get answers and ideas."
Lee Damon, University of Washington
"In a world where technology changes rapidly, it's hard to find great resources for in-depth expertise. LISA has a terrific combination of people who create the technology and people who implement it effectively for their companies."
Matt Simmons, Northeastern University
"I like seeing where the industry is heading, how SysAdmins/engineers are evolving, tools they are using, common problems and solutions across the world."
LISA14 Attendee
"LISA is the best mix of training, talks, and networking of any events I've been to. That's the reason I've been to 11 of them.”
LISA14 Attendee
"This was my first LISA. It was great attending a conference that focused on my role without trying to shoehorn every challenge into a specific vendor's solution. The same mix of awesome sessions, speakers, and other attendees will bring me back every year."
LISA14 Attendee
"If you're a sysadmin slaving away in a metaphorical basement and re-inventing the wheel every time your employer's business goals change slightly, LISA will help pull you into the light."
Marc Chiarini, Long-Time IT Admin and Researcher, MarkLogic Corp
"LISA is the home of ops people who do ops correctly. I’m always motivated by the conference content and hallway conversations to be better, and I take home the knowledge needed to move closer to that goal. "
Apache Kafka is a publish/subscribe messaging system that is in use within companies including LinkedIn, Twitter, Netflix, and many others. It is used to build Extract, Transform, and Load (ETL) pipelines, collect metrics and logs, and queue data between applications, often providing the main backbone for moving data within big data infrastructures. This tutorial will focus on how to get started with Kafka, including working with ZooKeeper, which it depends on. We will cover installation, configuring retention and replication, and creating simple applications for producing and consuming messages.
Who should attend:
This tutorial is designed for engineers, both operations and development, who are new to Apache Kafka and publish/subscribe messaging. The only prerequisite knowledge for participating is the ability to install software and execute basic shell commands. The ability to write basic Python programs is helpful, but not required. Full working versions of all scripts used in the tutorial will be provided.
Take back to work:
Participants will leave the tutorial with an understanding of how to set up Apache ZooKeeper and Apache Kafka, and create message producers and consumers, having completed this work on their own systems. This will allow them to set up a publish/subscribe messaging infrastructure that can be used for myriad applications, including monitoring, logging, queuing, and tracking user-generated events.
Topics include:
ZooKeeper
What is ZooKeeper?
What is it NOT?
Standalone Setup
Apache Kafka
Publish/Subscribe Messaging
Kafka Architecture
Installing Kafka
Producing Messages
Message Schema
Using the Console Producer
Producing Inside Applications
Consuming Messages
Using the Console Consumer
Consuming Inside Applications
Limitations of Non-Java Consumers
Kafka Clusters
Adding a Second Broker (partner work)
Replicating Partitions
Creating Multiple Partitions
Message Retention
Retention by Size
Retention by Key (log compacted)
Use Cases
Monitoring
Log Collection
User-generated Events
Queuing
Additional Materials:
The exact requirements are to be determined. Each attendee will be required to bring a laptop, preferably Mac or Linux, on which they can install required software, including a Java development kit (if not already installed), Apache ZooKeeper, and Apache Kafka. More detailed instructions will be provided prior to the conference.
