Common Ways People Destroy Their Log Files

For this article, I’m going to set up a hypothetical scenario (but based on reality) that needs logging. We’re writing an application that automates part of a steel factory. In our application, we need to calculate the temperature to which the steel must be heated. This is the responsibility of the TemperatureCalculator class.

The class is fed a lot of parameters that come from external sensors (like current temperature of the furnace, weight of the steel, chemical composition of the steel, etc.). The sensors sometimes provide invalid values, forcing us to be creative. The engineers said that, in such a case, we should use the previous value. This isn’t something that crashes our application, but we do want to log such an event.

So the team has set up a simple logging system, and the following line is appended to the log file:

An invalid value was provided. Using previous value.

Let’s explore how this well-meant log message doesn’t actually help. In fact, combined with similar messages in our log file, the log file ends up being a giant, useless mess.

 

Trash Fire Depicting Way People Destroy Log Files

 

Read More

Get Started Quickly With Python Logging

Picking up from the previous logging articles on how to get started logging with C# and Java, today we’ll be looking at how to get up and running quickly with logging in Python.

Even if you’ve already read the previous articles, this post is worth a read. It will cover new ground, like the basics around application logging in Python and a few other things, such as

  • Configuring the logging module.
  • What to log and why.
  • The security implications of logging.

So what are you waiting for? Keep reading, and let’s get a simple project set up to begin working with.

Python Scalyr Colors with LogRead More

Sexy But Useless DevOps Trends

What’s sexy but useless? A Ferrari in a traffic jam. It’s beautiful, but all that power means nothing. When trapped in traffic, it can’t live up to its full potential.

Same with DevOps. While there are some critical DevOps functions that you absolutely need, there are some sexy but useless DevOps trends that are good to be aware of. Truth be told, there’s no recipe that will tell you how to succeed in DevOps. Everyone will have different opinions, and what worked for others might not work for you. But you can trust one thing: there are some actions that will guide you directly to frustration with DevOps.

With the amount of information out there about DevOps, you might get overwhelmed and think it’s not for you. You also might think the learning curve is too steep—that you need to change too many things before you get started. Maybe you’ll need a new team, new tools, more metrics, more time… you name it.

My advice is this: don’t get distracted by all things that people say about DevOps. These things I’m going to talk about here, for instance, are all style and no substance.

 

Like this Ferrari if it were stuck in a traffic jam, some DevOps trends are sexy but useless.

Read More

5 Critical DevOps Practices

DevOps is like pizza. We can’t think of pizza without considering critical ingredients: dough, sauce, cheese, and your preferred choice for vegetables and proteins. Everyone likes different toppings. In my case, I can’t think about pizza without extra cheese and meat. You might choose differently, but I think we can agree there are some ingredients that are critical for this food to be called pizza. Quality and ingredients will vary, but some things will always remain true.

Well, it’s the same with DevOps practices. There are some critical practices, and you can’t think about DevOps without considering them. Everyone will have preferred choices regarding the tools and the process, but the practice will remain and each practice complements the other.

Every critical DevOps practice takes time to get down, but the end result will be magnificent. So, let’s discuss what they are and how to implement them.

Pizza with Scalyr Colors

Read More

A Detailed Introduction to the Apache Access Log

What is the Apache access log?  Well, at the broadest level, it’s a source of information about who is accessing your website and how.

But as you might expect, a lot more goes into it than just that.  After all, people visiting your website aren’t like guests at your wedding, politely signing a registry to record their presence.  They’ll visit for a whole host of reasons, stay for seconds or hours, and do all sorts of interesting and improbable things.  And some of them will passively (or even actively) thwart information capture.

So, the Apache access log has a bit of nuance to it.  And it’s also a little…complicated at first glance.

But don’t worry — demystifying it is the purpose of this post.

Apache Access Log: the Why

I remember starting my first blog years and years ago.  I paid for hosting and then installed a (much younger) version of WordPress on it.

For a while, I blogged into the void with nobody really paying attention.  Then I started to get some comments: a trickle at first, and then a flood.  I was excited until I realized that they were all suspiciously vague and often non-sequiturs.  “Super pro info site you have here, oPPS, I HITTED THE CAPSLOCK KEY.”  And these comments tended to link back to what I’ll gently say weren’t the finest sites the internet had to offer.

Yep.  Comment spam.

Somewhere between manually deleting these comments and eventually installing a WordPress plugin to help, I started to wonder where these comments were all coming from.  They all seemed to magically appear in the middle of the night and they were spammy, but I was interested in patterns beyond that.

This is a perfect use case for the Apache access log.  You can use it to examine a detailed log of who has been to your website.  The information about visitors can include their IP address, their browser, the actual HTTP request itself, the response, and plenty more.

An apache feather, representing our look at the apache access log.Read More

Get Started Quickly With Java Logging

You’ve already seen how to get started with C# logging as quickly as possible.  But what if you’re more of a Java guy or gal? Well, then we’ve got your back, too: today’s post will get you up to speed with logging using C#’s older cousin.

As in the previous post in this series, we’ll not only provide a quick guide but also go into more detail about logging, diving particularly into the what and why of logging.

The Simplest Possible Java Logging

For this simple demo, I’m going to use the free community version of IntelliJ IDEA. I’m also assuming that you have the Java JDK installed on your machine.

Read More

Log Appender: What Is It and Why Would You Use It?

If you’re not familiar with application logging, I can understand there being some confusion when you hear the term “log appender.”  What do you mean, “append”?  I want to write stuff to a file.  Isn’t that kinda the whole deal with log files?

So let’s demystify things a little.  A log appender is a specific part of the general logging process.  Meaning, yes, logging is often about writing to files, and a log appender can help you with that.  But it can actually help you with a whole lot more besides.

The appender is the part of a logging system that’s responsible for sending the log messages to some destination or medium.  It answers the question “where do you want to store this stuff?”

Image of Storage Cabinet

Anatomy of a Logging Operation

If all you’ve ever done with logging is dump messages to a file, this might smack of over-engineering.  “What do you mean ‘part of a logging system’?  You call the file API and write stuff to a file.”

Well, that certainly works for simple and small cases.  But as you expand your logging operation, you might need to get a little more sophisticated since simple file writes will start to create conflicts and other problems. You might even adopt a first-class logging framework. (In fact, you should.)  When you do this, logging becomes a more involved proposition, and it’s one that you can split into three main concerns:

  • Message recording and formatting.  This is where you decide what should go in the log and how to format it.
  • Log appender.  This is, as I’ve already mentioned, the part of the operation that decides where the messages go and how they get there.
  • Log consumption.  This can range from someone simply inspecting the logs to sophisticated search, intelligence, and visualization.

Even in the simplest logging implementation, all of these things actually take place.  For instance, consider this pseudocode:

public void keepTrackOfSomething() {
     _file.write("This method doesn't seem super-useful", "log.txt", File.Append);
}

Let’s see how our three concerns apply.

  • Recording and formatting is just creating the string “This method doesn’t seem super-useful.”
  • The appending is a simple file write in append mode to “log.txt”
  • Consumption happens later when someone scans through log.txt and reads the message.

Read More

What Goes Into Log Analysis?

I’ve talked here before about log management in some detail.  And I’ve talked about log analysis in high-level terms when making the case for its ROI.  But I haven’t gone into a ton of detail about log analysis.  Let’s do that today.

At the surface level, this might seem a little indulgent.  What’s so hard?  You take a log file and you analyze it, right?

Well, sure, but what does that mean, exactly?  Do you, as a human, SSH into some server, open a gigantic server log file, and start thumbing through it like a newspaper?  If I had to guess, I’d say probably not.  It’s going to be some interleaving of tooling, human intelligence, and heuristics.  So let’s get a little more specific about what that looks like, exactly.

Log Analysis, In the Broadest Terms

In the rest of this post, I’ll explain some of the most important elements of log analysis.  But, before I do that, I want to give you a very broad working definition.

Log analysis is the process of turning your log files into data and then making intelligent decisions based on that data.

It sounds simple in principle.  But it’s pretty involved in practice.  Your production operations generate all sorts of logs: server logs, OS logs, application logs, etc.  You need to take these things, gather them up, treat them as data, and make sense of them somehow.  And it doesn’t help matters any that log files have some of the most unstructured and noisy data imaginable in them.

So log analysis takes you from “unstructured and noisy” to “ready to make good decisions.”  Let’s see how that happens.

 

Read More

Search Your Files with Grep and Regex

How do you search through a file?  On the surface, this might seem like sort of a silly question.  But somewhere between the common-sense answer for many (“double click it and start reading!”) and the heavily technical (“command line text grep regex”) lies an interesting set of questions.

  • Where does this file reside?
  • What kind of file is it?
  • How big is the file?
  • What, exactly, are you looking for in the file?

Today, we’re going to look at one of the most versatile ways to search a file: using grep and regex (short for regular expression).  Using this combination of tools, you can search files of any sort and size.  You can also search with extremely limited access to your environment, and if you get creative, you can find just about anything.

But with that versatility comes a bit of a learning curve.  So let’s look at how to take the edge off of that and get you familiar with this file search technique.  To do that, I’ll walk through a hypothetical example of trying to extract some information.  First, though, let’s cover a bit of background.

Magnifying Glass Aimed at Search Box on Screen

Read More

Logging Levels: What They Are and How They Help You

Logging levels probably aren’t the most exciting thing in this world.  But then again, neither is banking.  And yet both things are fundamental to the people who use them as a tool.

Application logging is one of the most important things you can do in your code when it comes to facilitating production support.  Your log files serve as a sort of archaeological record of what on earth your codebase did in production.  Each entry in a log file has important information, including a time stamp, contextual information, and a message.  Oh—and generally, something called a logging level.

So what are logging levels?

Well, put as simply as possible, they’re simply a means of categorizing the entries in your log file.  But they categorize in a very specific way—by urgency.  At a glance, the logging level lets you separate the following kinds of information:

  • Hey, someone might find this interesting: we just got our fourth user named Bill.
  • OH NO SOMEONE GET A FIRE EXTINGUISHER SERIOUSLY RIGHT NOW.

Logging levels can help you distinguish whether you need the pictured fire extinguisher or whether the message just contains information.

For the most part, this distinction helps in two ways.  First, you can filter your log files this way during search.  And second, you can control the amount of information that you log.  But we’ll get to that in a bit.

Logging Levels: Why Do We Do It?

When it comes to logging, you have two essential and opposing forces, if you will.  On the one hand, you want to capture every last detail you can because this might prove useful during troubleshooting or auditing your system.  But on the other hand, all of that logging consumes resources. You can eat up disk space, overload people reading the logs, and even start to slow down your production code if you go overboard.

So logging requires either a balance or a way to get both the proverbial signal and the noise.  And logging levels look to help with this by making your situation configurable on the fly.

Here’s a helpful metaphor, perhaps.  Imagine you have an android friend and you ask him about what he did today.

“Well, I woke up, inhaled, moved my nose three inches to the right—”

“No!  Too much information.  Go from granularity 10 down to 3.”

“I woke up, had breakfast, caught a taxi…”

“THAT’s better.”

Usually, you don’t need to know everything the android did.  But if he starts malfunctioning, you might want to go back to “granularity 10” to zoom in on the problem as narrowly as possible.

Logging levels work this way.  Often you’ll only want to know if there are problems or warning conditions.  But sometimes you’ll really want to dial up the information for troubleshooting purposes.

Read More