The Summer of #SmoothScalin

Welcome to the summer of #SmoothScalin.

Hi, I’m Brett, a rising junior in college, I’ve taken surprisingly few classes that really take a deep dive into my majors, Information Systems, and Marketing. With graduation on the horizon, this used to concern me because I felt unprepared to enter the working world as graduation looms on the horizon. However, this summer I was fortunate enough to land a marketing operations internship at Scalyr. I’ve gained experience, confidence, a network, and a few really cool t-shirts.

After being at Scalyr for six weeks, I can confidently say I’ve learned more about marketing and what it’s like to work at a fast-paced startup than I have in two years of undergrad business classes. While I may be an intern, I’ve had a chance to work in depth on many projects ranging from designing and building landing pages to creative brainstorming for projects and engagements. One of the projects I’ve been working on is #SmoothScalin and I’m pleased to share it with you now.

What is #SmoothScalin?

#SmoothScalin is a program benefiting engineers and DevOps teams at startups. Early days for a startup can be like rough waters (believe us, we know!), especially as your team grows and – especially if you’re delivering software -as you start really ramping development. So, to help you meet the challenges that that growth can bring, we want to give you access to Scalyr. We use Scalyr to monitor our own system, which helps a ton because we’re moving super fast (don’t worry, we have a backup to help us avoid recursive errors). But enough about that; let me get to the offer!

In short, we’re offering one free year (from now until October 31, 2019) of Scalyr’s blazing-fast log management platform to startups that have received their A or B series funding within the last year.

Why should you sign up for #SmoothScalin?

Plain and simple: You get a log management platform that will help your entire team aggregate, monitor, manage, and search all of your log files. We promise you will be blown away by our speed, simplicity, and shareability. And when you’re on the front line of software delivery, that means something. Our ability to search 1.5 TBs of logs per second will totally reduce the amount of time you spend troubleshooting and debugging time. What that means to you is the same query that takes you 15 minutes in your traditional log management tool will take you less than one second in Scalyr. Besides being wicked fast, we’re also simple to use. How many hours have you spent futzing around with the query language in those traditional tools (or waiting in line for the PhD guru gatekeeper who knows how to do it – ugh!). Finally, Scalyr is shareable. We don’t mean we have a share button (okay, we do have a share button), but that the whole team can use it. Traditional tools either charge by the user, slow way down, or the company will throttle your usage. We don’t. The whole darn team can pound the crap out of Scalyr and you’ll be fine. We’re architected for this sort of thing.

What does your free year look like?

You’ll receive all the features of the Scalyr platform, along with the ability to add as many users as you want. We will also offer support and setup assistance for those who would like it. Also, you can upload up to 5GB/day of log volume on a 15-day retention. (Note that anything over the 5GB/day amount will be charged at our per GB list price of $40 per month, so it’ll still be a steal for you if you’re lucky (and good!) enough to grow fast and generate a ton of data!)

Pretty straightforward, right?

So what do you need to do?

Start by checking out the details on #SmoothScalin.
Then submit your application and we’ll take it from there.

And if you want to know more about Scalyr, check out the information on our homepage.

Not a startup who’s raised an A or B round, but still dogged by slow, complex, or non-shareable log management? Check us out anyway because we’re still a great deal. Check out our customer love on G2 Crowd, or if you’re more of a starched-shirt type, read why Gartner named us a Cool Vendor.

And may your summer (and the entire year) be #SmoothScalin.

Kubernetes: The Next VMware?

It’s been almost 10 years since VMware started selling ESX version 4.0. This set the path for VMware to dominate more than 75% of the virtualization market in 2017. Gartner considers this market “matured” since most of its revenue comes from maintenance instead of new licensing. Many companies have consolidated their workloads with virtualization, but there are new problems to solve.

Delivering, testing, deploying, and scaling applications are among these challenges. Teams that implement microservices also need to automate as much as possible to make them manageable. Kubernetes, Marathon, Swarm, and Nomad compose a new breed of tools that respond to these needs through orchestration. If you host on-premises or in the cloud, consider them to help your business more quickly deliver code to production.

Companies evolving towards data-driven decision-making often implement machine learning and business intelligence tools, looking for an edge in their markets. As information technology professionals, it’s our responsibility to make sure our businesses select tools that

  • perform in a reliable way;
  • allow quick deployment of new features;
  • scale properly in response to user demand; and
  • deploy new software in a safe and reproducible way.

In this article, I explain why I think Kubernetes is a market leader in the orchestration space and how it might steal VMware’s thunder in the not-so-distant future.

kubernetes_the_next-vmware_scalyr

Read More

Getting Started Quickly With C++ Logging

Since publishing my article Get Started Quickly With Python Logging, I’ve been working on a couple of C++ projects where I’ve found a need for more robust logging solutions than a simple time stamp and message written to a file. Since we also have articles on logging for C# and Java, it made sense to continue the series with an article on C++ logging with Boost.Log. Specifically in this article we are going to:

  • Create a simple Visual Studio 2017 project.
  • Install the Boost libraries using the NuGet package manager.
  • Learn how to configure log output formatting.
  • Add custom attributes to our logger.

This article is a big one, so strap in and let’s get started.

getting_started_quickly_with_c++_logging_scalyr

Read More

Modern software is great, but when it comes to observability, gird your loins!

This won’t come as a shock, but engineers who are on the hook to develop revenue-generating software are quickly moving away from traditional, monolithic architectures and delivering code faster than ever. This trend is especially evident among Scalyr customers, but we wanted to understand how it is generally unfolding and how it affects DevOps observability. We surveyed 155 software development practitioners in DevOps-focused organizations over the last couple of months, and just released our State of DevOps Observability Report. You can download the full report here or check out its summary in this infographic, but in this blog I’ll give you the Cliff’s Notes version of what we found.

Croped infographic on DevOps Observability

We confirmed that organizations really are shifting away from traditional, monolithic architectures. Three-quarters of survey respondents say they deliver at least some of their applications, and more than one-third deliver most of their applications, as microservices. They are also delivering software rapidly, with 71 percent of engineers pushing code into production at least weekly and nearly one-third doing so at least daily. Looking at all of our survey findings through the lens of these two trends, we realize that modern software delivery is putting pressure on DevOps observability.

Here are some of our findings:

Companies are delivering software in a modern way.

  • Three-quarters of respondents deliver some and more than one-third deliver most of their applications as microservices.
  • 71 percent of engineers push code at least once per week, and nearly one-third push code at least once per day.

Read More

Containers and Kubernetes vs VMs vs Config Management

In today’s DevOps-centered world, it’s often easy to be taken in by the infrastructure solution of the hour—VMs for everyone! Or wait, now containers are where it’s at! Who needs config management anyway?

Those of us that are primarily developers probably haven’t had to think about infrastructure decisions as much as our friends on the operations side. Since we haven’t had to make those decisions in the past, it can be hard to figure out what’s out there and why you’d want to use things like VMs or containers. We need to consider what actual problem these solutions are trying to solve. So let’s start simply.

containers_kubernetes_vs_config_management

Read More

How Serverless will Change DevOps

This post on Serverless is from guest author Limor Wainstein, email is limor@agileseo.co.il.

During the past decade, both DevOps and serverless have become quite popular for software development. These not only challenged the way software is being developed but also affected the entire software development lifecycle. First of all, let’s try to understand serverless and DevOps are.

Serverless and DevOps

Serverless is commonly used in conjunction with terms such as architecture, computer, and services. In brief, serverless architecture refers to the solution architecture of software applications built using fully managed third-party services (serverless services) as core dependencies. At its core, serverless computing provides runtimes to execute code, which is also known as function as a service (FaaS) platforms.

DevOps is a collection of practices and tools that increase the ability to deliver (build, test and deploy) applications and services more efficiently. DevOps requires more automation and eliminating overhead in setting up the infrastructure as well as in the build and deployment of code. DevOps does not only affect the technology; DevOps is also targeted at people and team structures.

Continuous Integration (CI) and Continuous Delivery (CD) tools are used to build, test and deploy code and are often done using build automation  Tools that provide fast resolution of errors and bugs are equally crucial.

In addition to using FaaS, fully-managed Docker containers as a service are also evolving. For example, technologies such as AWS Fargate allow to run these Docker containers and scale them without needing to provision and manage any underlying infrastructure. This is useful for DevOps since the containers can also be used to build, test and deploy applications.

Fully Managed DevOps Services

Fully managed DevOps services are beginning to emerge for CI and CD where these services are offered in software as a service (SaaS) models, or hosted services like Travis, Circle CI where the respective third-party service providers manage the underlying infrastructure and services. These services could range from semi-managed to fully managed serverless services that are used for the core of DevOps.

Docker containers underly many of these managed services, and these containers are used to allow isolated and customized build environments. Some services such as AWS Code Build enable defining the Docker image together with the required libraries and tools. This allows direct execution of the build job without needing to reinstall for each build. Various orchestration tools are used by these managed services, such as Kubernetes, Docker Swarm, AWS ECS, to control the fleets of containers.

DevOps Tools with Serverless.

There is also a tendency towards implementing DevOps using serverless architectures. The ability to implement unique and efficient CI/CD tailored to the application requirements, consumption-based cost models, supportive services (e.g., AWS Code Build, AWS Code Pipeline, etc.) create attraction towards this approach. By using serverless services, it is possible to implement an entire build, test and deploy pipeline by writing the glue code using serverless services, without using any hosted solutions.

Open Source and Serverless

Over the past few years, many open source DevOps tools started emerging. One of the most popular open source DevOps tools is the Serverless Framework.

The Serverless Framework allows simplification by:

  • Declaring the microservices infrastructure, endpoints and mapping it with execution logic (for example, with a Lambda function).
  • Building artifacts, archiving and deploying to infrastructure.
  • Extending DevOps capabilities using plugins.

Cloud providers are also beginning to develop similar tools. For example, AWS recently introduced its AWS Serverless Application Model (SAM) which could potentially replace the Serverless Framework functionality when it comes to declaring the serverless APIs. Contributions by the open source community do keep the Serverless Framework supporting newly-introduced AWS services as they appear.

In addition, open source security is becoming a challenge when using third-party libraries. On the other hand, vulnerability scanning is also becoming a commodity, especially with the introduction of the NPM scanning process. This has helped to reduce the development risks in building applications.

Infrastructure as Code

With the adoption of serverless architectures, infrastructure provisioning has become a significant part of the deployment process. Applications are using different serverless services as building blocks, so it is crucial to be sure that these services are correctly configured and connected. One of the best practices is to define the infrastructure in the code using declarative tools and languages like Terraform. This enables versioning of the infrastructure along with the code thus reducing the manual work on the DevOps process.

Immutable Infrastructure

Another effect of serverless for software application development is how we look at the serverless services modifications. It is often possible to provision new instances of serverless services while keeping the existing instances running. This means you can deploy new instances of serverless services and switch to the more recent version upon success. Serverless has made Blue-Green and Canary deployments both more practical and affordable for various applications.

Distributed DevOps

Using serverless technologies for application development means that DevOps has to deal with distributed systems. While it’s possible to use different serverless services from different providers, configuring a CI/CD pipeline involves coordinating these operations. This adds further complexity to the DevOps processes. It is important to implement more visibility into each serverless service with detailed monitoring when these services are provisioned once a change had been made.

Summary

Overall, we’ve outlined a few ways in which serverless is changing DevOps practices used for software development. To wrap up, let’s look at several best practices for serverless and DevOps.

  • Implement automation for CI/CD.
  • When implementing tests such as code style, unit tests, integration tests, UI tests, make them a part of development routine checks.
  • Upon committing the code, try to run the code style checking using git lifecycle hooks if possible.
  • Automate as needed to support building the artifacts, running all the test cases and send the results back to Pull Request using a CI tool.
  • Rerun the tests before deployments, preferably through an appropriate deployment trigger.
  • Include the infrastructure in the code.
  • Use managed services for CI/CD whenever possible to reduce the overhead of infrastructure management.
  • Design an application for rollback support for both previous versions and database migrations.
  • If it’s a database change, create scripts that can both apply and revert the changes.
  • Create version and store deployments in a durable place.
  • Allow deploying specific versions of application deployment.
  • Choose tools that help streamline the debug and monitoring cycle.

Separating the deployment cycles of serverless projects allows teams to make changes without redeploying the entire application. It remains vital to learn and understand the changes introduced by DevOps and serverless process and technologies and to adapt accordingly so that the DevOps processes are aligned to support serverless applications.

Bio:
Limor is a technical writer and editor at Agile SEO, a boutique digital marketing agency focused on technology and SaaS markets. She has over 10 years’ experience writing technical articles and documentation for various audiences, including technical on-site content, software documentation, and dev guides. She specializes in big data analytics, computer/network security, middleware, software development and APIs.
Twitter: @LimiMaayan

Why Do Engineers Care About Logging?

Logging and log analysis are critical to an engineer’s skill set. Engineers rely on logs to track how their systems are running. Sometimes engineers fail to log important events. They often regret not having them in the logs.

See, engineers use logs for error resolution, security, and even for making improvements to the system. Having certain events in the logs and the tools to properly analyze them is what enables engineers to do these things. Let’s look at these ways in which logs are valuable in order to understand more about why engineers care so much about logging.

Scaylr_Engineers_Care_About_Logging_Keyboard

Read More

Scalyr Platform: Batch Log Export, Alerting, and UI

In the Scalyr platform, releases over the past month include batch log export to Amazon S3 and alerting and UI improvements.

Batch Log Export to Amazon S3

We will make batch log export available on June 11, allowing you to filter logs and export them to Amazon S3. This lets you archive log data for compliance purposes, store them for future use, or include them in your CRM or ticketing system. Moreover, the exports run in the background in Scalyr. Users receive email notifications when they are completed. You can read more at Batch Export to Amazon S3.

Scalyr Alert Email
Scalyr Alert Email

Alerting Improvements

As part of our effort to improve alerting, we’re making email notifications easier to use. We have grouped alerts into “new,” “ongoing,” and “resolved” sections, making new issues stand out more clearly. Additionally, each alert now provides direct, deep links to the exact spot in Scalyr search or graphs for the alert time range (or the current time).

User Interface Improvements

We have upgraded the UI, including adding expandable dashboard legends so you can make better sense of what you’re seeing.

Scalyr Dashboard Expandable Legend
Scalyr Dashboard Expandable Legend

Going Forward

We are developing Scalyr for the engineering front line with a focus on our three value pillars: fast, simple, and shareable.

We will continue to revamp common workflows to refine the user experience. Right now, we are focusing on alerts. Soon, we will add improved alert management, including bulk actions and better silencing options, plus a more useful alert landing page.

Feedback

Your product (or any) feedback is always welcome. Please reach out to us at support@scalyr.com.