Introducing deeper support for Kubernetes, collaboration features and stack tracing for easier troubleshooting

As the move to the cloud and containers continues to make software delivery faster and easier, environments are getting more complex. At Scalyr, we believe that observability solutions need to help engineering and operations teams get access to all the data they need, fast. Along those lines, we are announcing new features to help teams support the latest container and orchestration technologies, improve collaboration, and streamline workflows for faster and easier issue identification and resolution.

Kubernetes Cluster-Level Logging

Our new Kubernetes cluster-level logging enables engineering teams to effectively monitor and troubleshoot Kubernetes environments by centralizing and visualizing logs by deployment. Regardless of source, Scalyr intelligently ingests, parses and organizes logs to give developers an application-level view for faster issue resolution in complex container environments.

As shown in the screenshot above, users are presented with log summaries by deployment  rather than by each individual container. The automatic grouping of logs pertaining to a deployment gives developers a more holistic view of each application or service that may be running on multiple containers and pods. Insight into individual pods and nodes is also available but that level of detail is abstracted by default so developers can focus on their application right away.

Chart Annotations

For many of the complex software issues faced by engineers, increased collaboration is key to finding solutions quickly.  To improve collaboration between engineering and operations and between different engineering teams, our new chart annotations provide a way for users to call attention to specific points or windows of time with markers and customizable notes. Annotations can be manually added to dashboard charts to highlight potential issues and shared with any team member. This improves communication and productivity among engineering team members, giving them additional context to more quickly hone in on the specific logs related to the problem at hand.

Slack Integration

We’ve extended our integration with Slack to provide more native interaction within the Scalyr UI.  When viewing a chart, users can select “Share with Slack” from the Share dropdown menu and immediately send the chart to another user or channel in Slack.

Stack Trace Linking

Scalyr now makes it possible to jump directly from log events with stack traces into the reference source code in your repositories. This streamlines your debugging workflow, making it faster and simpler to get under the hood, make tweaks, test hypothesis, and ultimately solve problems. Investigating exceptions is now as easy as one mouse-click. Scalyr supports any web-accessible repository such as GitHub.

CloudWatch Support

We’ve improved our support for AWS CloudWatch to provide a simpler and more reliable way to import AWS logs. By importing your AWS logs into Scalyr, you’ll get a centralized and more holistic view of all of your services, including serverless AWS Lambda functions and other AWS services.

We will be releasing these new features in the coming weeks. If you’ll be at AWS re:Invent 2018, be sure to swing by Booth 1014 to catch a demo.

Get Started Quickly With Scala Logging

We’ve covered how to log in eight different languages so far: C#, Java, Python, Ruby, Go, JavaScript, PHP, and Swift We’ve also included a few libraries and platforms, like Log4J, Node.js, Spring Boot, and Rails. Now, it’s time to show how you can get started quickly with Scala logging.

First, I’ll show you with a quick example of manual logging with Scala. I’ll use IntelliJ IDEA to create and run a Scala project, using sbt to build the code. So, you can use the application to get started on any platform that supports Scala and Java.

Then, I’ll discuss details of how and why logging matters. Finally, I’ll move on to using the Scala Logging wrapper in an application and how it can improve your ability to monitor your applications and issues.

Let’s get started!

Scala logging logo with Scalyr colors
Read More

Everyone has a part in making the planes fly

Waking up at 0445 to the sound of Reveille blaring across the Lackland Air Force Base Giant Voice System, rushing down the dormitory stairs just so my training instructors could bark orders at myself and the other Airmen while we ran miles around the outside track until the sun came up, I wondered if I was going to be able to make it. I wondered what the Air Force was going to be like. I wondered how things were going to change.

I thought basic training would be the most difficult part of the military because that’s what movies had depicted, but that was not the reality. After graduation, the real work began. Life was filled with on-the-job training, thick volumes of professional development courses that I had to memorize and be tested on, not to mention having to perform my job 10 hours each day, maintain physical fitness standards, go to college on my personal time, and somewhere in between all of that, find the time to sleep. Every day was insanely busy and challenging, but still rewarding. The folks I was lucky enough to work with were (and still are) some of the most inspiring people I have ever met and made even the most challenging days seem easy. It was all worth it. To say that I am fortunate having spent 10 years in the United States Air Force is an understatement; I am extremely grateful. All of these experiences have given me the tools necessary to navigate the civilian world, especially my first job as the Office Manager here at Scalyr, but how do these experiences translate?

A Sense of Urgency

Both the military and tech environments require a sense of urgency, regardless of what your job title is. In my previous profession, a sense of urgency revolved around things like performing complex data analysis, drafting hundreds of pay orders for folks traveling all over the world, or even providing critical support to our Crisis Action Team. Nowadays, my sense of urgency has shifted to planning fun events that continue building upon our company’s culture, making sure our employees have the necessary equipment to perform their jobs effectively and alerting the office when our daily lunch delivery might be running late! My job duties may have changed, but without a sense of urgency, nothing would be accomplished.

Service Before Self

Even though this is one of the Air Force’s core values, I believe this to be a universal value which everyone benefits from. Having the ability to put personal desires aside and function for the greater good of the team should be something that everyone works towards in a professional setting. Just like each position is vital to the success of the mission regardless of how big or small, I have discovered that the same applies to working at Scalyr (or anywhere else). Whether it’s a recruiting coordinator scheduling candidate onsite interviews, or an engineer fixing a high priority bug, every position matters; “everyone has a part in making the planes fly.”

Change

Being flexible and accepting of change was as important in the military as it is in the tech world and more specifically, at a tech startup. “Aren’t you bored?” is a question I am asked frequently. A lot of people may think that I had more work to do in the military compared to where I am now, but the answer to their question is no. I still have a never-ending list of things I need to do, every day is impossible to plan because there is always something that inevitably comes up which alters the course of my day, but all of that is good. Change is good. Expecting things to change and not getting comfortable is a part of life.

So what does all of this have to do with log management? Absolutely nothing! I am just here to give a different perspective and hope that this Veterans Day people not only thank veterans for their service, but also see the importance of hiring them. Yes, it can be tricky trying to navigate a veteran’s resume since the language and certifications may not translate at first glance, but that should not be a deterrent from hiring these well qualified, hard-working people who are extremely eager to apply their skills and contribute to a different kind of team and mission. Will they be bored in a job outside of the military? Not at all!

Microservices Communication: How to Share Data Between Microservices

For some, the ideal picture of a modern application is a collection of microservices that stand alone. The design isolates each service with a unique set of messages and operations. They have a discrete code base, an independent release schedule, and no overlapping dependencies.

As far as I know, this type of system is rare, if it exists at all. It might seem ideal from an architectural perspective, but clients might not feel that way. There’s no guarantee that an application made up of independently developed services will share a cohesive API. Regardless of how you think about Microservices vs. SOA, services should share a standard grammar and microservices communication is not always a design flaw.

The fact is, in most systems you need to share data to a certain degree. In an online store, billing and authentication services need user profile data. The order entry and portfolio services in an online trading system both need market data. Without some degree of sharing, you end up duplicating data and effort. This creates a risk of race conditions and data consistency issues.

At the same time, how do you share data without building a distributed monolithic service instead of a micro? What’s the effective and safe way to implement microservice communication? Let’s take a look at a few different mechanisms.

First, we’ll go over different sharing scenarios. Depending on how you use the data, you can share it via events, feeds, or request/response mechanisms. We’ll take a look at the implications of each scenario.

Then, we’ll cover several different mechanisms for microservice communication, along with an overview of how to use them.

Microservices talking to one another illusrating microservices communication
Read More

How to Redirect Docker Logs to a Single File

Sometimes, when troubleshooting or monitoring a Docker container, we need to see the application’s output streams. Containerized applications generate standard output (stdout) and standard error (stderr) like any other software. The Docker daemon merges these streams and directs them to one of several locations, depending on which logging driver is installed. The default driver makes the container output easy to access, and if we want to copy the information to another location, we can redirect docker logs to a file.

Let’s take a look at how we can manipulate logs generated by the default json-file log driver. This driver places the log output in a system directory in JSON formatted files but provides a command line tool for displaying log contents in its original format.

Docker whale leaping into a single file with Scalyr colors signifying a way to redirect docker logs to file
Read More

Top 4 Ways to Make Your Microservices Not Actually Microservices

Microservices have gained such a level of popularity these days that they’re often touted as “the way” to build software. That’s all well and good, except there are a lot of microservice anti-patterns flying around.

The vast majority of microservice systems I’ve seen aren’t really microservices at all. They’re separately deployable artifacts, sure. But they’re built in a way that, for developers, causes more pain than solves problems.

So let’s dive a little more into this. First, we’ll talk about what a microservice was intended to be. Then we’ll get into some of these anti-patterns.

Disguise_face_in_Scalyr_colors
Read More

Getting Started Quickly With Swift Logging

We’ve covered how to log in seven different languages so far: C#, Java, Python, Ruby, Go, JavaScript, and PHP. We’ve also included a few libraries and platforms, like Log4J, Node.js, Spring Boot, and Rails.

Now, it’s time to talk about Apple’s Swift language. Swift has been slowly gaining in popularity, especially with since its open source release.

I’ll start with a quick example of manual logging in Swift. Then I’ll discuss details of how and why logging matters. Finally, I’ll move on to using Apple’s Unified Logger in a Swift application and how it can improve your ability to monitor applications and track down issues

The code example will be for MacOS, but you can easily adapt it for any Apple platform.

Let’s get to work!

Swift_Logging_Swift_Bird_Scalyr
Read More

Microservices Vs. SOA: What’s the Difference?

What’s the difference between microservices and SOA? The two design paradigms have a lot in common.

That makes sense when you consider that microservices are an offshoot of the SOA movement. But there are essential differences between the two systems. Let’s take a look at the two different approaches to architecture and highlight where they differ.

Before we dive in, it’s important to note that neither architecture has a universally accepted definition. So, you could spend as much time debating the details of what microservices or SOA are as you could arguing their differences.

Read More

How to Merge Log Files

You have log files from two or more applications, and you need to see them together. Viewing the data together in proper sequence will make it easier to correlate events, and listing them side-by-side in windows or tabs isn’t cutting it.

You need to merge log files by timestamps.

But just merging them by timestamp isn’t the only thing you need. Many log files have entries with more than one line, and not all of those lines have timestamps on them.

merge_log_files
Read More

API vs. Microservices: A Microservice Is More Than Just an API

When writing software, consider both the implementation and the architecture of the code. The software you write is most effective when written in a way that logically makes sense. In addition to being architecturally sound, software should also consider the interaction the user will have with it and the interface the user will experience.

Both the concept of an API and the concept of a microservice involve the structure and interactions of software. A microservice can be misconstrued as simply an endpoint to provide an API. But microservices have much more flexibility and capabilities than that. This article will speak on the differences between APIs and microservices, plus detail some of the benefits a microservice can provide.

To get started, let’s define our terms.

api_vs_microservices_scalyr
Read More