Getting Started With the Rails Logger

Let’s continue our ongoing series on getting starting with loggers for different languages and platforms. Back in March, we covered logging with Ruby; now it’s time to take a look at the platform most often associated with that language, Rails.

We’ll start with a simple application with scaffolding for CRUD operations on a single record. We’ll look at Rails’ default logging configuration and how to use logging in an application. Then we’ll look at how logging can be improved and why you might want to improve it.

This tutorial uses Ruby v2.5.1 and Rails 5.2.0. You’ll need to have them installed to follow along. These instructions will use the command line to create and configure the application and will not rely on a specific IDE or editor. We’ll let Rails use SQLite for the backend database.rails_logger_scalyr

Read More

Kubernetes: The Next VMware?

It’s been almost 10 years since VMware started selling ESX version 4.0. This set the path for VMware to dominate more than 75% of the virtualization market in 2017. Gartner considers this market “matured” since most of its revenue comes from maintenance instead of new licensing. Many companies have consolidated their workloads with virtualization, but there are new problems to solve.

Delivering, testing, deploying, and scaling applications are among these challenges. Teams that implement microservices also need to automate as much as possible to make them manageable. Kubernetes, Marathon, Swarm, and Nomad compose a new breed of tools that respond to these needs through orchestration. If you host on-premises or in the cloud, consider them to help your business more quickly deliver code to production.

Companies evolving towards data-driven decision-making often implement machine learning and business intelligence tools, looking for an edge in their markets. As information technology professionals, it’s our responsibility to make sure our businesses select tools that

  • perform in a reliable way;
  • allow quick deployment of new features;
  • scale properly in response to user demand; and
  • deploy new software in a safe and reproducible way.

In this article, I explain why I think Kubernetes is a market leader in the orchestration space and how it might steal VMware’s thunder in the not-so-distant future.

kubernetes_the_next-vmware_scalyr

Read More

Getting Started Quickly With C++ Logging

Since publishing my article Get Started Quickly With Python Logging, I’ve been working on a couple of C++ projects where I’ve found a need for more robust logging solutions than a simple time stamp and message written to a file. Since we also have articles on logging for C# and Java, it made sense to continue the series with an article on C++ logging with Boost.Log. Specifically in this article we are going to:

  • Create a simple Visual Studio 2017 project.
  • Install the Boost libraries using the NuGet package manager.
  • Learn how to configure log output formatting.
  • Add custom attributes to our logger.

This article is a big one, so strap in and let’s get started.

getting_started_quickly_with_c++_logging_scalyr

Read More

Modern software is great, but when it comes to observability, gird your loins!

This won’t come as a shock, but engineers who are on the hook to develop revenue-generating software are quickly moving away from traditional, monolithic architectures and delivering code faster than ever. This trend is especially evident among Scalyr customers, but we wanted to understand how it is generally unfolding and how it affects DevOps observability. We surveyed 155 software development practitioners in DevOps-focused organizations over the last couple of months, and just released our State of DevOps Observability Report. You can download the full report here or check out its summary in this infographic, but in this blog I’ll give you the Cliff’s Notes version of what we found.

Croped infographic on DevOps Observability

We confirmed that organizations really are shifting away from traditional, monolithic architectures. Three-quarters of survey respondents say they deliver at least some of their applications, and more than one-third deliver most of their applications, as microservices. They are also delivering software rapidly, with 71 percent of engineers pushing code into production at least weekly and nearly one-third doing so at least daily. Looking at all of our survey findings through the lens of these two trends, we realize that modern software delivery is putting pressure on DevOps observability.

Here are some of our findings:

Companies are delivering software in a modern way.

  • Three-quarters of respondents deliver some and more than one-third deliver most of their applications as microservices.
  • 71 percent of engineers push code at least once per week, and nearly one-third push code at least once per day.

Read More

Containers and Kubernetes vs VMs vs Config Management

In today’s DevOps-centered world, it’s often easy to be taken in by the infrastructure solution of the hour—VMs for everyone! Or wait, now containers are where it’s at! Who needs config management anyway?

Those of us that are primarily developers probably haven’t had to think about infrastructure decisions as much as our friends on the operations side. Since we haven’t had to make those decisions in the past, it can be hard to figure out what’s out there and why you’d want to use things like VMs or containers. We need to consider what actual problem these solutions are trying to solve. So let’s start simply.

containers_kubernetes_vs_config_management

Read More

Why Do Engineers Care About Logging?

Logging and log analysis are critical to an engineer’s skill set. Engineers rely on logs to track how their systems are running. Sometimes engineers fail to log important events. They often regret not having them in the logs.

See, engineers use logs for error resolution, security, and even for making improvements to the system. Having certain events in the logs and the tools to properly analyze them is what enables engineers to do these things. Let’s look at these ways in which logs are valuable in order to understand more about why engineers care so much about logging.

Scaylr_Engineers_Care_About_Logging_Keyboard

Read More

Getting Started Quickly With Go Logging

It’s time to talk about how to get started with logging again. The languages we’ve covered so far are C#, Java, Python, Ruby, Node.js, and JavaScript. Today, we’re going to be talking about the Go programming language, also known as Golang.

Go is a statically compiled, open-source programming language created by Google in 2009. It runs on Windows, Linux, and Mac and has been increasingly adopted in the last couple of years because of its simplicity and concurrency mechanisms. And in case you didn’t know, Docker and Kubernetes were written in Go.

Go is quite an interesting language when we talk about logging. Why? For one thing, it includes a really simple native library for logging. But the most interesting thing is that in Go there are no exceptions—only errors. That means when you want to log application errors, things get interesting. We’ll cover that later in this post.

But let’s start by taking a look at how easily you can get started logging in Go.

Go Beaver With Scalyr Colors

Read More

Log Analyzer: What It Is and How It Can Help You

A log analyzer allows you to quickly search a specific set of data in a haystack of records. Think of it like bookkeeping. Accounting books tell a story for each transaction made in a period of time. To search through those transactions, you need criteria like the date and amount paid. You find the transaction log and it has more information: payment method, recipient, plus other details.

That’s what a log analyzer does: it helps you find events in a collection so you can study them and fix problems. Here’s an ideal case:

  1. An incident report provides data for log analysis.
  2. Someone reproduces this issue based on your log data.
  3. Your development team proposes a solution in a new fixed app.
  4. Someone reproduces the incident again, this time using your new app.
  5. Log analysis and testing show whether this fix was effective or not.

In this article, I’ll share insights from my experience with log analysis. Read on, draw parallels with your organization, and infer your own conclusions.

Graph with a magnifying glass over it

Read More

Five To-Dos When Monitoring Your Kubernetes Environment

If you’re on the DevOps front line, Kubernetes is fast becoming an essential element of your production cloud environment. Since container orchestration is critical to deploying, scaling, and managing your containerized applications, monitoring Kubernetes needs to be a big part of your monitoring strategy.

Container environments don’t operate like traditional ones. So, if you are monitoring your applications and infrastructure, you need to be thoughtful about how you monitor your container environment in which they are running. Here are five best practices to inform your strategy:

  1. Centralize your logs and metrics. Orchestrating your containerized services and workloads through Kubernetes brings order to the chaos, but remember that your environment is still decentralized. You will give yourself a fighting chance if you centralize your logs and metrics.
  2. Account for ephemeral containers. The beauty of container orchestration is it’s easy to start, stop, kill, and clean up your containers in short order. However, monitoring them may not be so easy. You still need to debug problems and monitor cluster activity, even when services are coming and going. The trick is to grab the logs and metrics before they’re gone. If you don’t, your metrics will look more like the graph on the left than the one on the right.
    log files examples for transient containers
  3. Simplify, simplify, simplify. With all of the moving pieces in your container environment (services, APIs, containers, orchestration tool), you need to monitor without introducing unneeded complexity. Rather than bloating your container with various monitoring agents, each requiring updates on unique schedules, abstract your monitoring and management tools from what you’re monitoring and managing. This will also help your engineers focus on building and delivering software, not operating the delivery platform.
  4. Monitor each layer explicitly. You will need to collect logs and monitor for errors, failures, and performance issues at each layer – the pod, the container, and the controller manager – of your environment. For example, you’ll need to be able to troubleshoot pod issues, ensure the container is working, and collect runtime metrics in the controller manager.
  5. Ensure data consistency across layers. For fast, accurate debugging, you need to ensure data consistency across all the layers in your container environment. Things like accurate timestamps, consistent units of measurement (such as milliseconds vs. seconds), and collecting a common set of metrics and logs across applications and components will help you troubleshoot and debug quickly and accurately across all of your layers.

One best practice for accomplishing these to-dos in a simple, straightforward manner is to monitor the containers in your Kubernetes environment without touching your application containers. Do this by introducing a DaemonSet, or alternatively a sidecar, into your Kubernetes environment(s) that sits alongside your containerized services and includes your logging and metrics collection agent. Deploying in this method will ensure consistent data collection, minimize the changes required to your application containers, and most importantly, eliminate the possibility of selective blindness in your production environment.

A few ways to implement this include:

  • Introduce a DaemonSet with the Fluentd logging agent (this will give you logging but not metrics). If you already have an ELK cluster configured, this is probably the option for you. Learn more here.
  • Introduce a DaemonSet or sidecar with the Prometheus metrics agent (CoreOS has done an excellent job of integrating Prometheus and Kubernetes). Running Prometheus on your Kubernetes cluster will give you metrics instrumentation, querying, and alerting. Learn more here.
  • A variety of metrics and performance monitoring tools, including Heapster, DataDog, cAdvisor, New Relic, Weave/VMware, and several others also offer a DaemonSet or sidecar options for Kubernetes monitoring.
  • Scalyr, log management for the DevOps front line, has a preconfigured DaemonSet containing the open source Scalyr agent available for download and use. The Scalyr DaemonSet natively supports both Kubernetes logging and metrics. You can download the YAML file for deploying the containerized Scalyr agent from GitHub here. Note that you also can download the full open-source Scalyr agent from GitHub here.