Choosing Among Log Management Tools

When you google log management tools, an interesting thing happens. At the time of this writing, you see no fewer than 4 paid ads, followed by a series of posts. These include, and this is not a joke, a post that lists the top 47.  As a software developer and tools consumer, this drives me insane. It probably does the same for you.

An author named Barry Schwartz coined a term (along with an eponymous book) for this frustration. He called it “the paradox of choice,” and it describes how, while we like to have some choice and autonomy, too much paralyzes us. To understand this in simple, terms, imagine selecting music for a dinner party. If offered two albums from which to choose, you’d make a pretty quick choice. If offered hundreds, you might thumb through them for a long time, trying to consider the likely tastes of all of your guests. And you might actually just give up eventually, and opt for only conversation with no background music at all.

The Paradox of Choice Among Log Management Tools

Back in the DevOps world, you face a similar plight when trying to pick among log management tools. You understand that you need a better way to aggregate and mine your logs than “by hand, using Sublime Text,” so you start to do some research. And then, about two searches in, you find yourself staring at post entitled, “The Top 47 Log Management Tools.” And, if you’re anything like me, you rub your temples and say to yourself, “ugh, never mind, I’ll figure this out tomorrow.”

That, of course, lines up with Schwartz’s findings about human behavior. Beyond having a few options, each additional option presented to a group of people causes fewer people to participate. The higher the number of log management tools in those posts, the fewer people will actually pick any of them at all.

Luckily, there’s a path back to joy. And it’s not even terribly complicated. You just need to dramatically narrow the field.

So today, I’m not going to add to the pile of “pros/cons/features” posts out there comparing dozens of tools. Instead, I’ll speak to heuristics you can employ to help you choose among log management tools. I’m going to help you narrow the field from a paralyzing number of choices that you make you unhappy to a manageable number that empowers you.

Read More

Five Reasons You Need Log Monitoring

You probably regard application logging the way you think of buying auto insurance. You sigh, do it, and hope you never need it. And aren’t you kind of required to do it anyway, or something? Not exactly the scintillating stuff that makes you jump out of bed in the morning.

It feels this way because of how we’ve historically used log files. You dutifully instrument database calls and controller route handlers with information about what’s going on. Maybe you do this by hand, or maybe you use a mature existing tool.  Or maybe you even use something fancy, like aspect-oriented programming (AOP). Whatever your decision, you probably make it early and then further information becomes rote and obligatory.  You forget about it.

At least, you forget about it until, weeks, months, or years later, something happens. Something in production blows up. Hopefully, it’s something innocuous and easily fixed, like your log file getting too big. But more likely some critical and maddeningly intractable production issue has cropped up. And there you sit, scrolling through screens filled with “called WriteEntry() at 2017-04-31 13:54:12,” hoping to pluck the needle of your issue from that haystack.

This represents the iconic use of the log file, dating back decades. And yet it’s an utterly missed opportunity. Your log file can be so much more than just an afterthought and a hail mary for addressing production defects. You just need the right tooling.

Log Monitoring To the Rescue

I’ve talked in the past about one form of upgrade from this logging paradigm: log aggregation. A log aggregation tool brings your log files into one central place, parses them, and allows you to search them rapidly. But you can do even more than that, making use of log monitoring via dashboards.

Read More

What Is Log Aggregation and How Does It Help You?

In order to understand the idea of log aggregation, you need to understand the pain it alleviates. You’ve almost certainly felt this pain, even if you don’t realize it.

Let’s consider a scenario that every programmer has probably experienced. You’re staring at some gigantic, dusty log file, engaging in what I like to think of as “programming archaeology.” And you have a headache.

A Tale of Logger Woe

It started innocently enough. A few users reported occasionally seeing junk text on the account settings screen. It’s not a regular bug, and it’s not particularly important. But it is embarrassing, and it seems like it should be easy enough to track down and fix. So you start trying to do just that.

You start by searching the database for the junk text in their screenshot. You find nothing.  Reasoning that application code must somehow have compiled the text in production, you figure you’ll head for the log files. When you open one up, and it crashes you text editor. Oops. Too big for that editor.

After using a little shell script magic to slice and dice the log file, you open it up and search for the text in question. That takes absolutely forever and yields no results. So you start searching for parts of the text, and eventually you have some luck. There’s a snippet of the text on line 429,012 and then another on line 431,114, with all sorts of indecipherable debug junk in between.

But you can’t find all of the text. And you have a headache. You then realize there’s a second log file for certain parts of the data access layer from before the Big Refactoring of ’15, and the rest of the text is probably in there. Your headache gets worse.

Read More

The DevOps Job Market

DevOps as a profession and discipline just keeps growing in demand. This year alone, more than 100 global conferences are dedicated to DevOps — and even the “best of” lists are staggering. Hundreds of companies (including Scalyr) are developing tools specifically for the DevOps field, and blogs dedicated to everything from current news and trends to making light of DevOps daily frustrations abound (RIP DevOps Reactions). Even a casual survey of the trending-up of “DevOps” in Google search over the past five years makes it pretty clear that demand for professionals in this space will only continue to rise.

Point. Made. (via Google Trends)

Read More

Company Culture: Actions, not just words

I recently joined the marketing team at Scalyr. I left my previous role last fall and took some time off. After a bit of travel, I spent the last few months exploring what I wanted to do next. In my next opportunity, I wanted a product first company and a culture that aligned with my values. Throughout my search, I met with several dozen people and companies, some casual catch-ups and others more formal interviews. I wanted to share some of the lessons I learned in my process that ultimately made me believe Scalyr was the right place for me.

Read More

CareerBuilder Resolves Customer Issues 5x Faster with Scalyr

We are excited to have CareerBuilder as a customer here at Scalyr. I recently sat down with Leon Chapman, Director of Cloud Operations at CareerBuilder, to learn more about their decision to use Scalyr and the impact the product is having on their teams and customer experience.

 

 

CareerBuilder chose Scalyr as their log management tool. After moving to the cloud two years ago, the team was looking to consolidate tools across their 250 person engineering organization. As a customer facing product, being able to identify issues quickly helps CareerBuilder deliver a better customer experience for the millions of people who use their products and services each day.

In particular, CareerBuilder found that when comparing Scalyr to other products, Scalyr beat the competition in:

  • Speed
  • Performance
  • Ability to scale without needing to manage infrastructure

Read More

Support Driven Development: Listen now so you don’t hear it later

Here at Scalyr, we’re big fans of Complaint-Driven Development, which I’ll summarize as “focus engineering effort on fixing the things users actually complain about.” We especially focus on issues that generate support requests, with such success that, as CEO, I’m still able to personally handle the majority of frontline support – even as we head toward eight-digit annual revenue.

An important consideration is that support requests cost money even if they aren’t your (product’s) fault. In this post, I’ll explore five common sources of support requests relating to the first piece of Scalyr software most users touch – our log collection agent and how we’ve sometimes had to think outside the box to address them. None of these were bugs, exactly. (We’ve had those as well, but you don’t need to read a blog post to know it’s a good idea to fix bugs.)

Arguably, none of these issues were “our fault.” But they generated a significant fraction of our support tickets. By eliminating them, we’ve reduced support costs significantly. Even more important, we’ve increased the probability that a user’s first experience with Scalyr is positive, especially for those users (a majority!) who will bounce off of a new product at the first sign of trouble, without bothering to ask for help.

Read More

In DevOps Incident Response, Plans Are Worthless, But Planning Is Everything

So said President Dwight D. “Ike” Eisenhower (more or less). His battles were fought in the trenches, not the technology stacks—but for DevOps teams, the principle holds. No plan survives contact with the enemy.

A plan is a set of instructions you can follow when you understand what needs to be done. Handy when the enemy is one you know, and can plan for. But the Enemy You Know isn’t what (literally) keeps DevOps engineers up at night.

Good DevOps teams competently respond to incidents, outages, and just plain weird stuff happening in their technology stack. Great teams go further—they see around corners, developing the instincts and skills to prepare for the unexpected. At Scalyr we start by reducing the risk of the dreaded 3 am call as much as possible. But we’d be foolish to stop there.

Read More

A Tale of Siri and My Home’s Energy Usage

Full disclosure: I’m a Scalyr DevOps engineer, but I’d be geeking out over the sheer number of possible uses for Scalyr even if I wasn’t. It’s more than a log analysis tool—it’s a platform. Scalyr now monitors the temperature inside my house, as well as the history of my thermostat and HVAC system usage. I’m one of very few homeowners in the world with real-time access to information about my HVAC system’s energy usage.

What compelled me to do this? Siri.

And a desire to harness home automation to improve my house’s energy efficiency. Here’s the story.

Read More

451 Research Covers Scalyr

“For Scalyr, it’s about speed, scale, and simplicity in log management,” is the title of 451 Research’s recent report on Scalyr. Nancy reached out to us to discuss our product a few weeks ago. We gave her a demo and spoke a number of times in detail about Scalyr. I was excited when I saw that she had decided to write about us and I’m even more excited to share the content of that report with the greater Scalyr family.

Click here to read Nancy’s report. 

If you’re looking for more information on Scalyr, don’t hesitate to contact us directly or to sign up for a trial.