Top 10 DevOps Tools to Watch in 2023
In the coming year, we could want to adopt some new (relatively) DevOps tools because they could significantly increase engineering productivity.
So without further ado, let's begin.
Let's start with Infrastructure, which is the foundation of DevOps.
First things first: like Terraform, AWS CDK, CDK for Terraform, etc., Pulumi is an Infrastructure as Code (IaC) tool.
Even if Terraform is arguably the most popular option for IaC today, there are several cons:
- You'd need to learn HCL as a new "language" (HashiCorp Config Language).
- HCL isn't really a "language" in the traditional sense, or at least not one that is very potent. For instance, a for loop on a module won't be usable until late 2020.
Bring on Pulumi.
So what is it?
It's simple for you to comprehend if you are familiar with the AWS CDK: it is that. Except that it works with all clouds and is ubiquitous (or at least, it attempts to be).
If you are unfamiliar with AWS CDK, consider this: With Pulumi, you can manage your infrastructure using programming languages that you are already familiar with, saving you the time and effort of having to learn yet another configuration language.
For whom is Pulumi? Great inquiry.
If you already know how to write in a language like TypeScript, Python, Go, C#, Java, etc. and don't really want to learn HCL, Pulumi might be the right choice for you. Technically, if you're using AWS, you could also use AWS CDK, however Pulumi makes more sense if you want to orchestrate a hybrid cloud architecture.
You might also give Pulumi a go if you already use Terraform extensively but are sick of HCL's restrictions and don't enjoy utilising built-in functions like "count" that make your infrastructure code harder to read.
This utility is no longer "new"; on GitHub, it has received more than 14k stars. Yet it is more recent than Terraform. Give it a shot if it fixes a specific issue you're having.
The open-source text file editor SOPS, which stands for Secrets OPerations, automatically encrypts and decrypts files.
The text editor, encryption, and automation are highlighted.
Typically, you use these steps to encrypt a text file:
- To write, modify, and manipulate the text data, use your preferred editor; then, save the changes as a file.
- Encrypt the entire file using an encryption/decryption tool.
When the encrypted file needs to be read:
- Using an encryption/decryption tool, you must first decrypt the file.
- With your preferred text editor, open the decrypted file (which is now a standard text file).
This "standard" procedure has one glaring flaw: using two tools (an editor and an encryption/decryption tool) for a single task.
Hopefully, you can see where I'm heading with this: SOPS is for that.
In other words, utilizing a git repo to store secrets is possible and simple for collaboration because it can be coupled with numerous encryption services (such HashiCorp Vault, AWS KMS, etc.) to encrypt your secret files automatically.
These days, when you want to build or deploy an app, your first thoughts are probably about containerization and 12-factor apps. Our cloud-native workload mainly relies on container images, so container image security is becoming increasingly important. Any container built from an image inherits all of its properties, including security flaws, incorrect configurations, and even malware.
A security scanner is Trivy. It operates anywhere you need it, is dependable, quick, and simple. The most well-known application of Trivy's numerous scanners, which seek for distinct security flaws, is for scanning container images for known vulnerabilities (CVEs).
Before publishing it to a container registry or deploying your application, you can use it locally as a CLI tool to scan your local container image and other artefacts.
Trivy is also made to be used in CI and is simple to connect with your CI pipelines, making it the ideal fit for the "continuous everything" DevOps philosophy.
A Kubernetes subsidiary project called lustrous API is dedicated to offering declarative APIs and tooling to make it easier to set up, maintain, and upgrade several Kubernetes clusters.
The Kubernetes Special Interest Group (SIG) Cluster Lifecycle launched the Cluster API project, which automates cluster lifecycle management for platform operators using Kubernetes-style APIs and patterns. The Kubernetes cluster configuration and the accompanying infrastructure, such as virtual machines, networks, load balancers, and virtual private clouds (VPCs), are all established in the same manner as how application developers deploy and manage their workloads. This makes it possible to install clusters consistently and repeatedly in a range of infrastructure scenarios.
If the official definition is unclear to you, consider that you can construct a K8s cluster by using just one kubectl apply command. This command is compatible with AWS, Azure, DigitalOcean, Docker, GCP, OpenStack, and other services.
For K8s clusters, kubectl apply is all that is required; there is no need to write Terraform modules (or, worse, try to figure out all the parameters of someone else's modules), and there is no need to learn how to use eksctl for AWS and something else for another cloud. Wow, that's impressive. I know. It is thus listed among the top ten tools to keep an eye on.
According to reports, Linkerd is the lightest and quickest service mesh in the world. A service mesh is what? A service mesh is an infrastructural layer that is specifically designed to make service-to-service communication quick, secure, and reliable.
Linkerd excels in being simple to use. It only takes one line of command to install it. This sentence comes to a close. It's so straightforward that I'm at a loss for words.
Let's chat more, though.
Quick setup is used. Since even Docker images are small, they are pulled more quickly.
There aren't any major differences in the architecture. There are two planes: a control plane and a data plane. The control plane consists of services that handle telemetry, API, control data for the data plane proxies, etc. The data plane also has proxies that operate next to each service instance. For further information, see the official document here.
The open-source envoy proxy, a high-performance C++ distributed proxy intended for single services and applications, is used by Istio and AWS App Mesh. It is a sophisticated all-purpose proxy. On the other side, Linkerd uses a specially created Rust-written proxy to be as compact, light, and secure as possible.I'm not here to weigh in on whether Rust or C++ is the best and safest language, but as a contemporary language with a unique approach to memory management (ownership instead of garbage collection), Rust definitely has an advantage.
Unlike Istio, Linkerd uses a service mirroring method for multi-cluster administration. The system is almost identical to a single-cluster arrangement in terms of simplicity, with the addition of two multi-cluster setups and a multi-cluster control plane.
In conclusion, Linkerd is a unique form of service mesh that is incredibly lightweight, straightforward, and powerful. Without adding complexity, Linkerd enhances the security, observability, and dependability of Kubernetes. Even though it isn't really a new tool, you should try it if the features suit your needs and you value simplicity.
Another CI is GitHub Actions.
So why GitHub Actions?
We kind of have to take a close look at it because, among other things, it is on the CNCF tech radar (and is currently at the "assess" stage, making it a "new" tool).
Another reason is that GitHub Actions naturally connect with your GitHub repositories and CI frequently interacts with your code. No more issues connecting your CI to your code repositories.
Another advantage for startups is that GitHub Actions includes some free quota, which may be more than enough when you've just launched a new product, making it absolutely free.In addition to saving money on running some VMs in a cloud for your own infrastructure just for the CI component, you are probably not going to need to register any additional self-hosted runners for quite some time.
Tekton is an additional CI (I know, I copy-pasted this line from the previous section).
Key characteristics include:
- It can be utilised by a K8s cluster.
- You kubectl apply the pipelines after defining them as a native K8s resource.
- It currently has a dashboard and CLI.
Additionally, you may create, test, and deploy using Tekton in a variety of contexts, including serverless or virtual machines. Tekton pipelines can be used to deploy across several cloud providers or hybrid environments.
Should you use it? My take is, if:
- you must "own" your CI system (for instance, you may not be able to use the free quota of GitHub Actions for some reason);
- you now employ K8s;
- you enjoy interacting with K8s in this way;
then attempt Tekton.
It is easy to install, and you may quickly get it working.
Harness is a CI, but it goes beyond that.
It comes from the company HashiCorp, whose name we already know, and it combines several elements into one:
- CI
- CD/GitOps
- feature flags
- cloud costs
To run your builds, Harness provides hosted virtual machines (VMs). You may build your code worry-free on the infrastructure that Harness offers with Harness Cloud. You may concentrate on creating excellent software by putting less time and effort into maintaining infrastructure.
Pipelines and Stages are used to model Continuous Delivery in Harness. You specify what to deploy using Services, where to deploy it using Environments, and how to deploy it using Execution steps in each stage.
You may carry out GitOps deployments in Harness using Harness GitOps. In your Git manifest, you specify the desired state of the service you wish to deploy, and Harness GitOps uses your live Kubernetes cluster to sync the state.
With the help of the feature management tool Harness Feature Flags (FF), you may alter the functionality of your software without writing new code. It enables you to conceal code or behaviour without releasing updated software. A feature flag is comparable to an effective "if" statement.
In conclusion, this is the SaaS CI/CD/FeatureFlags solution to consider if you want them all in one location.
Let's talk about Prometheus' local storage first:
Even if we can use storage.tsdb.retention to set a lengthy data retention duration, such as years, the issue of scalability and planning still needs to be addressed. Long queries might take a lot of memory to process when there have been a number of high-resolution probes used. A rate() function over a year with a 15-second scrape interval, for instance, takes 2.1 million samples, or roughly 2.6MiB of data. And that only applies to one metric.
The present TSDB implementation is perfectly capable of handling changing the retention time to years if you have a small infrastructure. Consider using a larger distributed TSDB for larger applications.
And Thanos, an open-source, highly available Prometheus setup with long-term storage capabilities, is a solution that addresses this issue. It focuses on long-term storage. Try Thanos if you've already experienced problems with Prometheus storage.
Let's discuss about Sentinel now.
Sentinel is HashiCorp's implementation of the policy-as-code methodology, in which policies are created, revised, shared, and enforced using code.
Sentinel works effectively with other HashiCorp products because it is a HashiCorp product. Sentinel is the ideal tool for you if you frequently use Terraform, Vault, Consul, or Nomad and want to explore Policy-as-Code.
Here are a few specific instances of what Sentinel policies can accomplish:
- Don't let Terraform be used to provide Cloud resources without tags.
- Make sure that only authorised system administrators with active MFA are able to modify critical Vault data.
- Allow Nomad to only run Docker workloads.
- Consul key updates are only possible during regular business hours.
a sample short code:
import “tfplan/v2” as tfplan
aws_instances = filter tfplan.resource_changes as _, rc {
rc.mode is “managed” and
rc.type is “aws_instance” and
rc.change.actions is not “delete”
}
main = rule {
all aws_instances as _, instance {
(instance.change.after.tags else {}) is not empty
}
}
I think it's very self-explanatory. Get the AWS instances from a Terraform plan; after the update, tags cannot be empty (unless you are trying to delete the instance).
Keep checking back because I'll shortly post an introduction to Policy-as-Code if you're interested.
- A brief classification of all the tools described in this article is as follows:
- Pulumi, Infrastructure as Code
- Security: Trivy and SOPS
- K8s/multi-cluster: Linkerd, Cluster API
- GitHub Actions, Tekton, and Harness are examples of CI/CD.
- Keep an eye on Thanos
- HashiCorp Sentinel policy as code
Read the full article
