LISA '10 Interview: Tom Limoncelli

Anyone who has attended LISA in the past few years is undoubtedly familiar with Tom Limoncelli.  Tom's not just a LISA fixture, he's also a widely-respected author of two books (Time Management for System Administrators and The Practice of System and Network Administration) and a contributor to the Everything Sysadmin blog.  Over the weekend, he sat down with me for a few minutes to share his thoughts about LISA '10.

Ben Cotton: You are, quite truly, an expert on everything sysadmin.  How did you reach that status?

Tom Limoncelli:  I'm honored by the question but the name "EverythingSysadmin.com" comes from my co-author (Christine Hogan) and I trying to come up with a domain name that was related to our book, but wasn't really long.  Since the book tried to touch on a little of everything, we came up with EverythingSysadmin.com.

BC:  So would you consider yourself a generalist or do you have a few fields that you feel you're truly an expert in?

TL:  I do consider myself a generalist.  I think that's because when I got started in system administration you had to be.  Now things are different.  Now people tend to specialize in storage, backups, networking, particular operating systems, and so on.  Remember that The Practice of System and Network Administration has three authors; we only know "everything" when all three of us put our brains together.  I guess you'd have to say that my specialty is in always knowing someone that can find an answer for me.

BC:  That's an excellent lesson.  You're scheduled to conduct several training sessions on time management during LISA '10.  What would you say is the biggest lesson to be learned from them?

TL: The biggest lesson is that humans are bad at time management, and that's OK.  The great thing about being human is that we can build tools that let us overcome our problems.  The class that I teach has very little theory. It's mostly a list of techniques people can use to solve specific problems. Use the ones you like, ignore the rest.  The one that most people end up using is finding a good way to manage their to-do list.

BC: If someone's taken your time management training before, what do you have new for them this year?

TL: I have an entirely new class this year.  It's a "part 2" kind of thing, though you don't have to have taken part 1 to take it.  In the morning I'll be teaching "Time Management for System Administrators" which is basically the same half-day class I usually teach.  The afternoon, however, is all new.  It is "Time Management: Team Efficiency".

The thing about Teams is that there are certain things you do that waste time for everyone else.  You might not even realize it.  In this class, I'm going to cover a number of techniques for eliminating those things.  You save time for others, they save time for you.  It's like "time management karma".  What goes around comes around.  For example, meetings are often a terrible waste of time.  I'll talk about some red flags to help you figure out which meetings to skip, and if you run meetings you can figure out if you are creating these red flags.  If you can't fix a badly run meeting, I have some tips on how to negotiate so that you don't have to attend. For example, why send your entire team to someone else's boring meeting?  Send one person to take notes and report back to your team.  If you can't get out of a meeting, I have techniques for avoiding them. For example, when you enter the room tell the facilitator, "I have a conflict for the second half of the meeting.  Can my agenda items be first on the list?" After your item is covered, stand up and leave.  It isn't unethical or dishonest: the "conflict" you had was your urgent need to escape badly run meetings.

BC: You've been a regular fixture at  LISA.  What keeps you coming back?

TL: LISA is like telescope that lets me see into the future.  Every year there are presentations that describe things that the majority of all system administrators won't be exposed to for 2-3 years.  When I come back to work I have more of a "big picture" than my coworkers that didn't attend.  For example, it was at LISA that I first heard of CFEngine, Puppet and other "Configuration Management" (CM) tools.  Lately people talk CM as if it was new.  It's certainly much more popular now, but people that have been attending LISA conferences have been benefiting from CM tools for more than a decade.

90% of what is interesting in system administration relates to scaling: More machines, more RAM, more storage, more speed, more web hits.  Many years ago there was a presentation by a web site that was managing 1 million web hits per day. At the time this was huge achievement.  People that saw that presentation were in a great position a few years later when all big sites scaled to be that big.

BC: What are the big scaling challenges?

TL: Everything we used to know is about to change because of SSD.  Everything I know about designing and scaling systems is based on the fact that CPU caches are about 10x faster than RAM, which is 10x faster than disk, which is about 10x faster than networks.  Over the years this has been basically true: Even as RAM got faster, so did disk.  SSD is about to change that.  The price curve of SSD makes it pretty easy to predict that we're not going to be using spinning magnetic disks to store data soon.  All the old assumptions are going away.  At the same time, CPUs with 16+ and soon 100+ cores make other assumptions change.  Things get worse in some ways.  These are the hot topics that you hear about at a conference like LISA.

Just the other day a very smart coworker said something to me that implied that with the new generation of 100+ core machines we could "just run more processes" and not have to change the way we design things.  I was floored.  That's like saying, "Basketball players seem to be able to jump higher every year.  Why can't we jump to the moon?"

BC: As an avid basketball fan, I find that idea intriguing.   It's obvious attending LISA can be very beneficial. As an experienced attendee, what advice do you have for people who may be going to their first LISA conference?

TL: First: Talk to random people.  When you are on line, introduce yourself to the people next to you.  A big chunk of the learning opportunity is from talking with fellow attendees.  Sysadmins are often introverts, so it is a bit difficult.  Someone once told me that it's always ok to start a conversation with a stranger by sticking out your hand and saying, "Hi!  My name is Joe." (if your name is Joe).  Unlike some conferences where the speakers are corralled into a "green room" and never talk with attendees, at Usenix conferences you can talk to anyone.  At my first Usenix experiences I met Dennis Ritchie, one of the inventors of Unix.

Second: plan your days.  There are activities from 9am until midnight every day.  Read the schedule beforehand and make a grid of what you want to attend.  Saturday night is a session for "first timers" which is a great way to get an overview of the conference.  During the day there are usually 3-4 things going on at any time.  At night there is an entire schedule community-driven events.  You don't want to be picking what to do next at the end of each session.  Also, plan some down-time.  Take breaks. Get plenty of fluids.  It is a full week.

BC: Any other thoughts you'd like to share?

TL: There's also a lot of great security talks, and an entire track of Q&A sessions with experts answering questions about everything from storage to disaster recovery to consulting.  The last thing I'd like to say is, "see you there!"

Registration for LISA '10 is still open at http://www.usenix.org/events/lisa10/.  You can find Tom's training courses on the training page.  He'll also be presenting two technical sessions.