You have log files from two or more applications, and you need to see them together. Viewing the data together in proper sequence will make it easier to correlate events, and listing them side-by-side in windows or tabs isn’t cutting it.
You need to merge log files by timestamps.
But just merging them by timestamp isn’t the only thing you need. Many log files have entries with more than one line, and not all of those lines have timestamps on them.
When writing software, consider both the implementation and the architecture of the code. The software you write is most effective when written in a way that logically makes sense. In addition to being architecturally sound, software should also consider the interaction the user will have with it and the interface the user will experience.
Both the concept of an API and the concept of a microservice involve the structure and interactions of software. A microservice can be misconstrued as simply an endpoint to provide an API. But microservices have much more flexibility and capabilities than that. This article will speak on the differences between APIs and microservices, plus detail some of the benefits a microservice can provide.
I’ll open with a quick example of manual logging in PHP. Then we’ll revisit the details of why logging matters and what your logs should show. And lastly, I’ll show you how to set up and use the most popular PHP logging framework.
In this article, I’ll provide step-by-step instructions on how to create a Docker container, modify its internal state, and then save the container as an image. This is really handy when you are working out how an image should be constructed, because you can just keep tweaking a running container until it works like you want it to. When you’re done, just save it as an image.
Containers are hot stuff right now, so it’s natural that you’re here wondering what the business case and benefits of containers could be.
If this is you—if you’re looking to assess whether containers would make sense for your company—then you’re in just the right place. Because by the end of this article, you’ll not only have a good understanding of what containers are and what they’re good (and not so good) at, but you’ll also have some decision making criteria to help you decide whether they’ll work for you in your unique situation.
We’ve got quite a bit of ground to cover, so let’s get to it.
As an engineer, you already have enough responsibilities when developing software. Adding more tasks–say, DevOps-related ones—to your workday activities might not sound very appealing. With DevOps, not only are you responsible for producing working software, but now you also need to automate the building, testing, and deployment phases of the software. That’s a lot to take care of! But the extra work aside, maybe you’re just tired of the DevOps movement, and all the hype surrounding it is causing DevOps fatigue.
As a former developer, I can identify with that feeling of fatigue. I’ve also seen some colleagues reach a certain level of frustration with DevOps. There are times when we make the mistake of taking on everything, even the releases. This is especially common if we’re perfectionists and don’t like to deliver software with bugs. We could even get to the point of releasing our code to production. (Although now that you’re “doing” DevOps, that might become your responsibility anyway.) After all, if we code it, we know the things that could go wrong and how to fix it if there are problems.
Even though now I’m more on the operations side of things—dealing with servers and helping companies implement DevOps—let me share some thoughts with you about why I think engineers are getting DevOps fatigue.
Well, microservices sure are getting a lot of attention these days. It’s almost uncool to like them, as they are so mainstream. When you have all these separate modules doing their own things, the question inevitably comes up: How do we stitch them together?
The answer? Very carefully. Here are some tips you can use if you find yourself in the position of needing to orchestrate your microservices.
At Scalyr, we’ve optimized our pipeline from data ingestion to query execution so our customers can search logs at TBs per second. In this post, we’ll discuss a specific part of that pipeline: regular expression (regex) parsing during Bloom filter creation. Read on to learn how we captured the huge query latency reduction enabled by Bloom filters with a custom-built regex parser, and how much speed we gained as a result.
A little background
Scalyr organizes incoming log data in a columnar format: each text line is split into multiple “columns” that can be queried independently. We also partition the data into multiple “epochs,” where each epoch represents a fixed time range. For example, a typical HTTP access log in our database is organized like this:
Last month, we focused on usability improvements in our Scalyr Platform releases. These include better alerts, tweaks to the graph legend, live tail improvements, adding scrolling to the search fields sidebar and improving times and dates in search results.
We covered basic logging for Java applications a while back. In that tutorial, we used log4j version 2, a logging framework from the Apache project. Let’s go one step further with Java application logging and look at log4j2 configuration.
Log4j’s capabilities have made it one of Java’s most popular logging frameworks. It can be configured for multiple logging destinations and a variety of log file formats. Log messages can be filtered and directed at the individual class level, giving developers and operations personnel granular control over application messages.
Let’s examine these mechanisms by configuring log4j with a command line Java application.