Go for Development Operations & Site Reliability Engineering (SRE)
Go helps enterprises automate and scale
Development Operations (DevOps) teams help engineering organizations automate tasks and improve their continuous integration and continuous delivery and deployment (CI/CD) process. DevOps can topple developmental silos and implement tooling and automation to enhance software development, deployment, and support.
Site Reliability Engineering (SRE) was born at Google to make the company’s “large-scale sites more reliable, efficient, and scalable,” writes Silvia Fressard, an independent DevOps consultant. “And the practices they developed responded so well to Google’s needs that other big tech companies, such as Amazon and Netflix, also adopted them.” SRE requires a mix of development and operations skills, and “empowers software developers to own the ongoing daily operation of their applications in production.”
Go serves both siblings, DevOps and SRE, from its fast build times and lean syntax to its security and reliability support. Go’s concurrency and networking features also make it ideal for tools that manage cloud deployment—readily supporting automation while scaling for speed and code maintainability as development infrastructure grows over time.
Featured Go users & projects
Docker is a software-as-a-service (SaaS) product, written in Go, that DevOps/SRE teams leverage to “drive secure automation and deployment at massive scale,” supporting their CI/CD efforts.
Drone is a Continuous Delivery system built on container technology, written in Go, that uses a simple YAML configuration file, a superset of docker-compose, to define and execute Pipelines inside Docker containers.
etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines, and its written in Go.
IBM’s DevOps teams use Go through Docker and Kubernetes, plus other DevOps and CI/CD tools written in Go. The company also supports connection to it’s messaging middleware through a Go-specific API.
Microsoft uses Go in Azure Red Hat OpenShift services. This Microsoft solution provides DevOps teams with OpenShift clusters to maintain regulatory compliance and focus on application development.
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. It supports a number of cloud providers such as AWS, IBM Cloud, GCP, and Microsoft Azure - and its written in Go.
Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Most Prometheus components are written in Go, making them easy to build and deploy as static binaries.
YouTube uses Go with Vitess (now part of PlanetScale), its database clustering system for horizontal scaling of MySQL through generalized sharding. Since 2011 it’s been a core component of YouTube’s database infrastructure, and has grown to encompass tens of thousands of MySQL nodes.
How to use Go for DevOps & SRE
Go has been enthusiastically adopted by the DevOps and SRE communities. As previously noted, many underpinnings of the modern cloud environment are themselves written in Go—including Docker, Etcd, Istio, Kubernetes, Prometheus, Terraform, and many others.
DevOps/SRE teams write software ranging from small scripts, to command-line interfaces (CLI), to complex automation and services, and Go’s feature set has benefits for every situation.
For small scripts: Go’s fast build and startup times. Go’s extensive standard library—including packages for common needs like HTTP, file I/O, time, regular expressions, exec, and JSON/CSV formats—lets DevOps/SREs get right into their business logic. Plus, Go’s static type system and explicit error handling make even small scripts more robust.
For CLIs: every site reliability engineer has written “one-time use” scripts that turned into CLIs used by dozens of other engineers every day. And small deployment automation scripts turn into rollout management services. With Go, DevOps/SREs are in a great position to be successful when software scope inevitably creeps. Starting with Go puts you in a great position to be successful when that happens.
For larger applications: Go’s garbage collector means DevOps/SRE teams don’t have to worry about memory management. And Go’s automatic documentation generator (godoc) makes code self-documenting.
Go books on DevOps & SRE
Go Programming for Network Operations
Go Programming Blueprints
Go in Action
The Go Programming Language
Monitoring and tracing
- opentracing/opentracing-go, vendor-neutral APIs and instrumentation for distributed tracing
- jaegertracing/jaeger-client-go, and open source distributed tracing system developed by Uber
- grafana/grafana, an open-source platform for monitoring and observability
istio/istio, an open-source service mesh and integratable platform
- spf13/cobra, a library for creating powerful modern CLI applications and a program to generate applications and CLI applications in Go
- spf13/viper, a complete configuration solution for Go applications, designed to work within an app to handle configuration needs and formats
- urfave/cli, a minimal framework for creating and organizing command line Go applications
- golang-migrate/migrate, database migration tool written in Go