A Detailed Introduction to the Apache Access Log

What is the Apache access log?  Well, at the broadest level, it’s a source of information about who is accessing your website and how.

But as you might expect, a lot more goes into it than just that.  After all, people visiting your website aren’t like guests at your wedding, politely signing a registry to record their presence.  They’ll visit for a whole host of reasons, stay for seconds or hours, and do all sorts of interesting and improbable things.  And some of them will passively (or even actively) thwart information capture.

So, the Apache access log has a bit of nuance to it.  And it’s also a little…complicated at first glance.

But don’t worry — demystifying it is the purpose of this post.

Apache Access Log: the Why

I remember starting my first blog years and years ago.  I paid for hosting and then installed a (much younger) version of WordPress on it.

For a while, I blogged into the void with nobody really paying attention.  Then I started to get some comments: a trickle at first, and then a flood.  I was excited until I realized that they were all suspiciously vague and often non-sequiturs.  “Super pro info site you have here, oPPS, I HITTED THE CAPSLOCK KEY.”  And these comments tended to link back to what I’ll gently say weren’t the finest sites the internet had to offer.

Yep.  Comment spam.

Somewhere between manually deleting these comments and eventually installing a WordPress plugin to help, I started to wonder where these comments were all coming from.  They all seemed to magically appear in the middle of the night and they were spammy, but I was interested in patterns beyond that.

This is a perfect use case for the Apache access log.  You can use it to examine a detailed log of who has been to your website.  The information about visitors can include their IP address, their browser, the actual HTTP request itself, the response, and plenty more.

An apache feather, representing our look at the apache access log.Read More

Get Started Quickly With Java Logging

You’ve already seen how to get started with C# logging as quickly as possible.  But what if you’re more of a Java guy or gal? Well, then we’ve got your back, too: today’s post will get you up to speed with logging using C#’s older cousin.

As in the previous post in this series, we’ll not only provide a quick guide but also go into more detail about logging, diving particularly into the what and why of logging.

The Simplest Possible Java Logging

For this simple demo, I’m going to use the free community version of IntelliJ IDEA. I’m also assuming that you have the Java JDK installed on your machine.

Read More

Log Appender: What Is It and Why Would You Use It?

If you’re not familiar with application logging, I can understand there being some confusion when you hear the term “log appender.”  What do you mean, “append”?  I want to write stuff to a file.  Isn’t that kinda the whole deal with log files?

So let’s demystify things a little.  A log appender is a specific part of the general logging process.  Meaning, yes, logging is often about writing to files, and a log appender can help you with that.  But it can actually help you with a whole lot more besides.

The appender is the part of a logging system that’s responsible for sending the log messages to some destination or medium.  It answers the question “where do you want to store this stuff?”

Image of Storage Cabinet

Anatomy of a Logging Operation

If all you’ve ever done with logging is dump messages to a file, this might smack of over-engineering.  “What do you mean ‘part of a logging system’?  You call the file API and write stuff to a file.”

Well, that certainly works for simple and small cases.  But as you expand your logging operation, you might need to get a little more sophisticated since simple file writes will start to create conflicts and other problems. You might even adopt a first-class logging framework. (In fact, you should.)  When you do this, logging becomes a more involved proposition, and it’s one that you can split into three main concerns:

  • Message recording and formatting.  This is where you decide what should go in the log and how to format it.
  • Log appender.  This is, as I’ve already mentioned, the part of the operation that decides where the messages go and how they get there.
  • Log consumption.  This can range from someone simply inspecting the logs to sophisticated search, intelligence, and visualization.

Even in the simplest logging implementation, all of these things actually take place.  For instance, consider this pseudocode:

public void keepTrackOfSomething() {
     _file.write("This method doesn't seem super-useful", "log.txt", File.Append);
}

Let’s see how our three concerns apply.

  • Recording and formatting is just creating the string “This method doesn’t seem super-useful.”
  • The appending is a simple file write in append mode to “log.txt”
  • Consumption happens later when someone scans through log.txt and reads the message.

Read More

What Goes Into Log Analysis?

I’ve talked here before about log management in some detail.  And I’ve talked about log analysis in high-level terms when making the case for its ROI.  But I haven’t gone into a ton of detail about log analysis.  Let’s do that today.

At the surface level, this might seem a little indulgent.  What’s so hard?  You take a log file and you analyze it, right?

Well, sure, but what does that mean, exactly?  Do you, as a human, SSH into some server, open a gigantic server log file, and start thumbing through it like a newspaper?  If I had to guess, I’d say probably not.  It’s going to be some interleaving of tooling, human intelligence, and heuristics.  So let’s get a little more specific about what that looks like, exactly.

Log Analysis, In the Broadest Terms

In the rest of this post, I’ll explain some of the most important elements of log analysis.  But, before I do that, I want to give you a very broad working definition.

Log analysis is the process of turning your log files into data and then making intelligent decisions based on that data.

It sounds simple in principle.  But it’s pretty involved in practice.  Your production operations generate all sorts of logs: server logs, OS logs, application logs, etc.  You need to take these things, gather them up, treat them as data, and make sense of them somehow.  And it doesn’t help matters any that log files have some of the most unstructured and noisy data imaginable in them.

So log analysis takes you from “unstructured and noisy” to “ready to make good decisions.”  Let’s see how that happens.

 

Read More

Search Your Files with Grep and Regex

How do you search through a file?  On the surface, this might seem like sort of a silly question.  But somewhere between the common-sense answer for many (“double click it and start reading!”) and the heavily technical (“command line text grep regex”) lies an interesting set of questions.

  • Where does this file reside?
  • What kind of file is it?
  • How big is the file?
  • What, exactly, are you looking for in the file?

Today, we’re going to look at one of the most versatile ways to search a file: using grep and regex (short for regular expression).  Using this combination of tools, you can search files of any sort and size.  You can also search with extremely limited access to your environment, and if you get creative, you can find just about anything.

But with that versatility comes a bit of a learning curve.  So let’s look at how to take the edge off of that and get you familiar with this file search technique.  To do that, I’ll walk through a hypothetical example of trying to extract some information.  First, though, let’s cover a bit of background.

Magnifying Glass Aimed at Search Box on Screen

Read More

Logging Levels: What They Are and How They Help You

Logging levels probably aren’t the most exciting thing in this world.  But then again, neither is banking.  And yet both things are fundamental to the people who use them as a tool.

Application logging is one of the most important things you can do in your code when it comes to facilitating production support.  Your log files serve as a sort of archaeological record of what on earth your codebase did in production.  Each entry in a log file has important information, including a time stamp, contextual information, and a message.  Oh—and generally, something called a logging level.

So what are logging levels?

Well, put as simply as possible, they’re simply a means of categorizing the entries in your log file.  But they categorize in a very specific way—by urgency.  At a glance, the logging level lets you separate the following kinds of information:

  • Hey, someone might find this interesting: we just got our fourth user named Bill.
  • OH NO SOMEONE GET A FIRE EXTINGUISHER SERIOUSLY RIGHT NOW.

Logging levels can help you distinguish whether you need the pictured fire extinguisher or whether the message just contains information.

For the most part, this distinction helps in two ways.  First, you can filter your log files this way during search.  And second, you can control the amount of information that you log.  But we’ll get to that in a bit.

Logging Levels: Why Do We Do It?

When it comes to logging, you have two essential and opposing forces, if you will.  On the one hand, you want to capture every last detail you can because this might prove useful during troubleshooting or auditing your system.  But on the other hand, all of that logging consumes resources. You can eat up disk space, overload people reading the logs, and even start to slow down your production code if you go overboard.

So logging requires either a balance or a way to get both the proverbial signal and the noise.  And logging levels look to help with this by making your situation configurable on the fly.

Here’s a helpful metaphor, perhaps.  Imagine you have an android friend and you ask him about what he did today.

“Well, I woke up, inhaled, moved my nose three inches to the right—”

“No!  Too much information.  Go from granularity 10 down to 3.”

“I woke up, had breakfast, caught a taxi…”

“THAT’s better.”

Usually, you don’t need to know everything the android did.  But if he starts malfunctioning, you might want to go back to “granularity 10” to zoom in on the problem as narrowly as possible.

Logging levels work this way.  Often you’ll only want to know if there are problems or warning conditions.  But sometimes you’ll really want to dial up the information for troubleshooting purposes.

Read More

Get Started Quickly With C# Logging

If you’re interested in getting started with C# logging as quickly as humanly possible, you’ve come to the right place.  Today, we’ll look at just how to do that.  Don’t worry, we’ll get into comprehensive detail too, covering a number of things.  Some of those include

  • What is logging?
  • Why would you want to log?

But for now, let’s get to the action.  How can you start logging in the absolute quickest and simplest way possible?  Let’s take a look.

Getting started quickly with C# logging

The Simplest C# Logging That Could Possibly Work

Let’s start with a clean slate for the sake of easy demonstration.  First, open Visual Studio and create a new console project by selecting File->New->Project, as shown here.

To get started with C# logging, create a console project, as shown here.

I’ve called my new project ConsoleProject and opted to put it into a new solution called LoggerPlaypen.  No need to congratulate me on my creativity, though.

This will open the Program.cs file containing your Program class.  Let’s now add a few lines of code to that class, as follows:

static void Main(string[] args)
{
    var displayMessage = "Hello, user!";

    Console.WriteLine(displayMessage);
    Console.ReadLine();
}

We now have an application that successfully defines a greeting, displays it, and waits for an enter keystroke to dismiss it.  When you hit F5 to run, you’ll see the following message before pressing enter:

Here is the console output, "Hello, user!"

But let’s say that we wanted not only to display this message but also to log it to a file.  How might we do that?  With something fancy, like a logging framework, inversion of control, or aspect-oriented programming?  Nah, let’s not get ahead of ourselves.  Try this instead, making sure to add “using System.IO;” at the top with the other using statements:

static void Main(string[] args)
{
    var displayMessage = "Hello, user!";

    File.WriteAllText("log.txt", displayMessage);

    Console.WriteLine(displayMessage);
    Console.ReadLine();
}

Now, when you hit F5, you’ll see your greeting message once again.  But that extra line you’ve added will create a log file for you.  Go to your project directory, and then navigate to the Bin->Debug folder. You’ll see a file hanging out there in addition to your console executable.

Here's the file in your Debug directory, plus a shot of the file's contents, "Hello, user!"

Is That Really It?

And…that’s it.  You just successfully implemented your first C# logger!

Is that all you need to know about logging?  Of course not.  Will this logging scheme be sufficient for any but the most trivial applications?  Nope.  But have you learned the barest essential backbone of C# logging?  Absolutely.

I told you it’d be a quick start.  But I strongly recommend that you read on to really get a handle on this logging thing.

Read More

Surprising Use Cases for Log Visualization

People commonly say that a picture is worth a thousand words.  So I wonder if log visualization is worth a thousand log entries.  The math equivalency might be a little hard to prove, but the idea is worth exploring.

You’re recording all sorts of information in your log files, but are you visualizing that information?

Do you have dashboards and graphs that help you picture production behavior?  Or does the information sit buried within digital mountains of arcane strings?  The proverbial needle in the haystack?  Does anyone who wants to use it need to engage in laborious searches?

If you’re not visualizing your logs, you’re missing out.  But I don’t necessarily want to make the case for visualization today.  Instead, I’d like to offer some ideas for visualization that you perhaps hadn’t considered.  Let’s take a look at some use cases for log visualization that you might not have considered.

Prerequisites for Log Visualization

First things first, though.  Before I can take you through the use cases, you need to have a setup that allows log visualization.  Specifically, you need modern log file management, which includes the following things that concern us:

  • Log aggregation (gathering and storing log files).
  • Meaningful parsing of the data contained in the log files.
  • Powerful search capabilities.
  • Log visualization tools.

Don’t get me wrong.  You could implement all of this stuff yourself.  But then again, you could also implement your own source control and text editor.  Just because you can doesn’t mean you should.

If you want to get serious about visualizing log files without burning tons of time, you need tooling and infrastructure in place to help you.  Once you have that, though, let’s take a look at some of the things you might do.

Read More

Creating an Audit Trail for Your Business

No matter what you do, there will be aspects of your job that you absolutely love.  And then you’ll have the things that you tolerate out of necessity.  I’m guessing that, for almost everyone reading, “audit trail” sounds like something that fits squarely into the “tolerate” bucket.

Even if you don’t know what it is, it probably sounds equal parts intimidating and boring.  The closest word association you’ll likely have with “audit” is that it’s what the IRS does to you when it simultaneously takes a fine-toothed comb to your life and demands more money from you.  And looking to avoid angering the IRS is probably not what you dreamed of on career day as a child.

But building and maintaining an audit trail for your business doesn’t have to be onerous.  Far from it.

Magnifying glass looking at a graph

What Is an Audit Trail, Anyway?

I’ve thrown the word around a few times, but let’s get a little more precise to set the stage for a post.  What is an audit trail?

To get a good working definition of “audit trail,” consider the definition of “audit.”

An official examination and verification of accounts and records, especially of financial accounts.

It has official overtones to it, and it involves taking a detailed look at relevant records.  So when you commission an audit, you ask someone to come in, on the record, and take a detailed look at what you’re doing.

Building and maintaining an audit trail for your business doesn't have to be onerous.

An audit trail, then, is what you do to facilitate this activity.  You make sure to dutifully document and capture anything that an auditor might need.  What’s the reasoning for this?  Generally speaking, you do this to demonstrate that you operate with a high degree of transparency and that your activities are all ethical, responsible, and legal.

Take the aforementioned case of the IRS mandating an audit for you.  This will tend to go much better for you if you’ve made sure to create an audit trail: saving receipts, noting business expenses, keeping careful track of all income, etc.

 

Read More

Log File Too Big — What Should I Do?

You have a problem. But you don’t just have an ordinary problem. You have one of the most frustrating kinds of problems in the technical world. In the most basic terms, you’re trying to open a log file that’s too big to open. But “log file too big” doesn’t fully capture the frustration or the problem.

You need something out of your log file, so you go to open it. Then you wait. And wait. And wait.

After some amount of time, your text editor just crashes. Hoping it’s a fluke, you try again, waiting 15 minutes before another crash. So you’re half an hour in and not only have you not solved your actual problem — you haven’t even successfully taken what should be the simplest imaginable step toward solving it.

This combination of a long feedback loop with a non-deterministic outcome is what makes this so maddeningly frustrating. But fear not. Let’s take a look at how you can solve this, starting with the quickest and most superficial route and working toward root cause analysis.Log file too big? It can seem as though the thing is crushing you.

Pick a Different Tool at Your Disposal

If opening this log file is crashing an editor, like, say, Notepad, then your easiest step is to use a different editor. At least that way you can know that fate will reward your waiting with an opened file rather than with a crash.

Your path of least resistance here is to use something you already have installed. So consider the following utilities for each application.

  • For Windows, you can use WordPad. If you have enough memory to cover the size of the file you want to edit, WordPad will load it. So these days, that’s quite likely to apply to files even topping a gig in size.
  • For Mac, use Vim. It should be able to handle as big a file as you have memory, and with good search besides.
  • There are a lot of different flavors of Linux out there, so it’s a little harder to talk about default installations. But if you have it, you can also use Vim here. If not, you can install it easily, and you can use tail -X at the command line, where X is the number of lines you’d like to see.

That should at least get you started. You should be able to see your file without needing to wait for something to maybe or maybe not crash.

Download and Use a Text Editor Meant for This

If you have a little more patience, you should ask yourself whether your current need is a one-time-only situation or if you’ll be viewing and editing a lot of large files. If the latter, you’ll want to get more deliberate about the tools in your toolbox. I might suggest this even if you think this is a one-time need. Familiarizing yourself with a new, powerful text editor can’t hurt anything.

The number of text editors available to you is FAR too large to enumerate here. But Wikipedia has an extensive page on them, including specific information about file size.

If the problem solved by opening your large file isn’t too pressing, you could always engage in some yak-shaving. But you should probably solve that problem first, using a tool at your disposal. Then come back and spend some time evaluating your text editor options. Find one that can open large files and that has other features you like besides. I’d say even try out a few of them.

If large files figure to be part of your life going forward, you should have a plan of attack for them.

Read More