The Different Types of Server Monitoring Software

If you landed here from a google search for “server monitoring software,” then I don’t envy you. You’re probably searching for a specific answer to a specific problem. And you’ve no doubt learned that there are roughly 8,498 companies that offer server monitoring software. But the only thing they have in common? All of them can totally monitor everything that you might ever need monitored.

E-commerce has grown steadily in the last two decades, eventually coming to completely dominate our commercial world. So a lot of people have a lot of money invested in a lot of stuff running on the internet. This, in turn, makes a lot of people interested in monitoring that stuff, seeking to defend their livelihoods and gain competitive advantages.

Thus when we talk about server monitoring software, we’re casting a pretty wide net. Too wide, in fact. Servers handle lots of things. Searching for server monitoring software will cause you to smack up against the paradox of choice. You have too many options.

You could narrow the field in a number of ways. How much does the tool cost? What server operating system? Is it open source? You get the idea.

But today, I’d like to help you focus the scope of your search by acknowledging that servers are extremely complex, resulting in many different potential things to monitor. Let’s take a look at the different types of server monitoring software as a function of what, exactly, they monitor. Get a better focus there, and you’ll know what to look for.

Read More

Flyclops Accelerates Development by Using Scalyr

We are thrilled to have Flyclops as a customer at Scalyr. I recently chatted with co-owner Dave Martorana to learn more about how they chose Scalyr and how it is helping the engineering team be more effective.

Flyclops was evaluating log management tools when they came across Scalyr in a newsletter. After signing up for a trial, the team was blown away by the speed of the product. Searches went from minutes to seconds, which allowed them to save valuable development time. Using Scalyr has improved their ability to provide support and rapidly resolve issues, which has led to increased player happiness and let the team sleep at night knowing that they had the right tool in place to help them monitor activity on their servers.

About Flyclops

Flyclops is a independent mobile games studio located in Philadelphia, PA, specializing in casual multi-player games, both asynchronous turn-based, and real-time. Flyclops’s games have been played by millions across the globe.

Evaluation Process

When Flyclops was looking for a logging solution, they evaluated several tools. Other tools they were experimenting with made searching logs a painful task. Flyclops had a few things they were looking for in a log management tool, including:

  • Ease of logging
  • Speed of collection
  • Ability to pull metrics out of log data
  • Ability to parse custom log formats
  • Ability to diagnose issues quickly

Dave Martorana, co-owner of Flyclops, discovered Scalyr via a Google Go newsletter as a featured log management tool. Given that a lot of the Flyclops backend was written in Go, they decided to give it a try.

Results of using Scalyr

When the Flyclops team started using Scalyr, they immediately took notice of the speed and performance of the tool. Searches went from tens of seconds and minutes in other tools to almost instantaneous with Scalyr. 

According to Dave,

“Scalyr has been the single best tool I’ve added to our stack in years”.

Flyclops has 500,000 unique players per month. By using Scalyr, they are able to save significant time investigating issues, which gives more time for development. Scalyr allowed them to diagnose most problems substantially faster than with other tools they had tried. They were able to replace whole suites of monitoring tools with something that can answer questions they don’t know they’re going to have in the future.

The team liked that they had the ability to write their own parsers and didn’t have to conform to a certain pattern when writing data to logs. On the client side of things, they’ve gone through a number of third party crash reporting tools. They started logging client exceptions and wrote some custom parsers in order to parse thru the stack traces and proactively look at what’s unique to their products. They were able to turn Scalyr into the best stack analysis tool they had used.

The team sleeps a lot better knowing that Scalyr is watching their servers. The ability to answer questions they didn’t anticipate allows them to be more proactive. They are able to define custom variables and query them. When launching a new feature with a staged rollout, they are able to use Scalyr to validate that they are rolling out at the speed they expected with just a little bit of graphing. All of this allows them to better support their players, and be sure their customers are having a high-quality experience at all times.

Features of a Good Log File Viewer

When you think of a log file viewer, what do you think of? Come on, be honest. Vim or Emacs?  Notepad++ or Sublime? Do you just invoke “tail” from the command line? Please say you’re not just using Notepad in Windows. No judgment, but if you’re going that route, you’re putting yourself through a lot of unneeded pain.

I ask about these tools — these text editors — because that’s how we conceive of a log file viewer. Logs are text files. So when we go to view them, we use a tool meant for viewing (and editing) text. This is completely understandable, and it’s also served us well as an industry since some programmer half a century ago first had the idea to output runtime information to a file.

Or, at least, it has sufficed. Viewing log files with a text editor supplies us with the basics for troubleshooting. We can look at the information contained in the file and we can do text search, with varying degrees of sophistication. And, if necessary, we can copy and make modifications to the text.

But surviving isn’t thriving. When it comes to employing a log file viewer, we can ask for so much more. This really shouldn’t surprise in the year 2017. Software is “eating the world” and the DevOps movement has brought us an explosion of SaaS and tools to help software shops. Should it really surprise anyone that the modern log file viewer can do some awesome stuff?

Let’s take a look at some of what you should expect when picking a tool to help you view your logs. What are features of a good log file viewer in this day and age?

Read More

What To Look for in a Logging Framework

If you’ve spent any amount of time in the software industry, you’ve probably bumped up against logging. Maybe you first encountered it as a way to debug your program as you worked, printing “you are here” messages. Go much beyond that, especially in 2017, and your logging efforts likely graduate to the use of a logging framework.

What Is a Logging Framework? Let’s Get Precise.

So what marks this distinction? When do you go from newbie printing out “got into the Calculate() method” to user of a logging framework? To understand that, let’s define logging framework.

A logging framework is a utility specifically designed to standardize the process of logging in your application. This can come in the form of a third party tool, such as log4j or its .NET cousin, log4net. But a lot of organizations also roll their own.

You have to go beyond just standardizing, however, to qualify as a framework. After all, you could simply “standardize” that logging meant invoking the file system API to write to a file called log.txt. A logging framework has to standardize the solution by taking it care of the logging for you, exposing a standard API.

To get more specific, we can conceive of a logging framework encapsulating three main concerns: recording, formatting, and appending. When you want to capture runtime information about your application, you start by signaling what to record. Then, you determine how to format that information. And, finally, you append it to something. Classically, this is a file, but you can also append to RDBMS tables, document databases, or really anywhere capable of receiving data.

If you have a piece of code dedicated to solving these problems, you have yourself a logging framework.

Should You Use an Existing Logging Framework or Build Your Own?

Now that we have precision around the definition, let’s examine your options for making use of a logging framework. Notice that I frame the question in terms of which one you should pick, rather than whether or not you should use one. I do that because production applications should most definitely make use of logging frameworks. Without them, you’re flying blind about what your application does in the wild.

The question then becomes whether you should use an existing one or write your own. Use an existing one — full stop. It’s 2017, and this is a well solved problem in every tech stack.  You shouldn’t write a logging framework any more than you should write your own source control tool or build a bug tracker. Others have built these, and you can have them cheaply or freely. Stick with solving problems in your own domain, rather than reinventing wheels.

So you need a logging framework, and you should use an existing one. The question then becomes “which one?” Which logging framework should you use? I won’t answer that outright, since it will depend on your stack and your needs. Instead, I’ll offer guidance for how to choose.

Read More

What Is Log Aggregation and How Does It Help You?

In order to understand the idea of log aggregation, you need to understand the pain it alleviates. You’ve almost certainly felt this pain, even if you don’t realize it.

Let’s consider a scenario that every programmer has probably experienced. You’re staring at some gigantic, dusty log file, engaging in what I like to think of as “programming archaeology.” And you have a headache.

A Tale of Logger Woe

It started innocently enough. A few users reported occasionally seeing junk text on the account settings screen. It’s not a regular bug, and it’s not particularly important. But it is embarrassing, and it seems like it should be easy enough to track down and fix. So you start trying to do just that.

You start by searching the database for the junk text in their screenshot. You find nothing.  Reasoning that application code must somehow have compiled the text in production, you figure you’ll head for the log files. When you open one up, and it crashes you text editor. Oops. Too big for that editor.

After using a little shell script magic to slice and dice the log file, you open it up and search for the text in question. That takes absolutely forever and yields no results. So you start searching for parts of the text, and eventually you have some luck. There’s a snippet of the text on line 429,012 and then another on line 431,114, with all sorts of indecipherable debug junk in between.

But you can’t find all of the text. And you have a headache. You then realize there’s a second log file for certain parts of the data access layer from before the Big Refactoring of ’15, and the rest of the text is probably in there. Your headache gets worse.

Read More

Company Culture: Actions, not just words

I recently joined the marketing team at Scalyr. I left my previous role last fall and took some time off. After a bit of travel, I spent the last few months exploring what I wanted to do next. In my next opportunity, I wanted a product first company and a culture that aligned with my values. Throughout my search, I met with several dozen people and companies, some casual catch-ups and others more formal interviews. I wanted to share some of the lessons I learned in my process that ultimately made me believe Scalyr was the right place for me.

Read More

CareerBuilder Resolves Customer Issues 5x Faster with Scalyr

We are excited to have CareerBuilder as a customer here at Scalyr. I recently sat down with Leon Chapman, Director of Cloud Operations at CareerBuilder, to learn more about their decision to use Scalyr and the impact the product is having on their teams and customer experience.

 

 

CareerBuilder chose Scalyr as their log management tool. After moving to the cloud two years ago, the team was looking to consolidate tools across their 250 person engineering organization. As a customer facing product, being able to identify issues quickly helps CareerBuilder deliver a better customer experience for the millions of people who use their products and services each day.

In particular, CareerBuilder found that when comparing Scalyr to other products, Scalyr beat the competition in:

  • Speed
  • Performance
  • Ability to scale without needing to manage infrastructure

Read More

Support Driven Development: Listen now so you don’t hear it later

Here at Scalyr, we’re big fans of Complaint-Driven Development, which I’ll summarize as “focus engineering effort on fixing the things users actually complain about.” We especially focus on issues that generate support requests, with such success that, as CEO, I’m still able to personally handle the majority of frontline support – even as we head toward eight-digit annual revenue.

An important consideration is that support requests cost money even if they aren’t your (product’s) fault. In this post, I’ll explore five common sources of support requests relating to the first piece of Scalyr software most users touch – our log collection agent and how we’ve sometimes had to think outside the box to address them. None of these were bugs, exactly. (We’ve had those as well, but you don’t need to read a blog post to know it’s a good idea to fix bugs.)

Arguably, none of these issues were “our fault.” But they generated a significant fraction of our support tickets. By eliminating them, we’ve reduced support costs significantly. Even more important, we’ve increased the probability that a user’s first experience with Scalyr is positive, especially for those users (a majority!) who will bounce off of a new product at the first sign of trouble, without bothering to ask for help.

Read More

451 Research Covers Scalyr

“For Scalyr, it’s about speed, scale, and simplicity in log management,” is the title of 451 Research’s recent report on Scalyr. Nancy reached out to us to discuss our product a few weeks ago. We gave her a demo and spoke a number of times in detail about Scalyr. I was excited when I saw that she had decided to write about us and I’m even more excited to share the content of that report with the greater Scalyr family.

Click here to read Nancy’s report. 

If you’re looking for more information on Scalyr, don’t hesitate to contact us directly or to sign up for a trial.

Please Review Scalyr (And Get Free Stuff)

To help get the word out about Scalyr, we’re looking for people to write honest reviews of our product on GetApp and Capterra. If you take the time to write a review on one of those sites we’ll send you a $20 gift card for either Starbucks or Amazon (please see details below).

To write a review on GetApp you need to have a LinkedIn account and be willing to sign into GetApp with it, but your review can be anonymous once posted. For Capterra you need to give your email address and the name of the company where you used the product that you are reviewing, and finished reviews will include your name and company name.

We are only looking for reviews from current Scalyr customers or Scalyr trial users who have uploaded data and spent more than two hours using Scalyr. In order to qualify for the gift cards, please send the following information to contact@scalyr.com:

  • a link to the review in question
  • your name and the email address you use to log into scalyr
  • the email address where you’d like us to send the gift card code

We’ll take honest reviews anytime, but this particular offer of a gift card when you review us expires on February 17th, 2017. All gift cards will be issued for USD $20 from the US Starbucks or US Amazon website.

To write a GetApp review, please start here.

For Capterra, please click here. 

Thanks in advance!