1. Documentation
  2. Go Telemetry

Go Telemetry

Table of Contents:

Background
Overview
Configuration
Counters
Reporting and Uploading
Charts
IDE Prompting
Frequently Asked Questions

Background

Go telemetry is a way for Go toolchain programs to collect data about their performance and usage. Here “Go toolchain” means developer tools maintained by the Go team, including the go command and supplemental tools such as the Go language server gopls or Go security tool govulncheck. Go telemetry is only intended for use in programs maintained by the Go team.

By default, telemetry data is kept only on the local computer, but users may opt in to uploading an approved subset of telemetry data to telemetry.go.dev. Uploaded data helps the Go team improve the Go language and its tools, by helping us understand usage and breakages.

The word “telemetry” has acquired negative connotations in the world of open source software, in many cases deservedly so. Yet measuring the user experience is an important element of modern software engineering, and data sources such as GitHub issues or annual surveys are coarse and lagging indicators, insufficient for the types of questions the Go team needs to be able to answer. Go telemetry is designed to help programs in the toolchain collect useful data about their reliability, performance, and usage, while maintaining the transparency and privacy that users expect from the Go project. To learn more about the design process and motivation for telemetry, please see the telemetry blog posts. To learn more about telemetry and privacy, please see the telemetry privacy policy.

This page explains how Go telemetry works, in some detail. For quick answers to frequently asked questions, see the FAQ.

Overview

Go telemetry uses three core data types:

All local Go telemetry data and configuration is stored in the directory os.UserConfigDir()/go/telemetry directory. Below, we’ll refer to this directory as <gotelemetry>.

The diagram below illustrates this data flow.

In the rest of this document, we’ll explore the components of this diagram. But first, let’s learn more about the configuration that controls it.

Configuration

The behavior of Go telemetry is controlled by a single value: the telemetry mode. The possible values for mode are local (the default), on, or off:

The gotelemetry command configures the telemetry mode and manages local telemetry data. Use this command to install it:

go install golang.org/x/telemetry/cmd/gotelemetry@latest

The following commands interact with the telemetry mode:

For the complete usage information of the gotelemetry command line tool, see its package documentation.

Telemetry may also be enabled by accepting an IDE prompt.

Counters

As mentioned above, Go telemetry is instrumented via counters. Counters come in two variants: basic counters and stack counters.

Basic counters

A basic counter is an incrementable value with a name that describes the event that it counts. For example, the gopls/client:vscode counter records the number of times a gopls session is initiated by VS Code. Alongside this counter we may have gopls/client:neovim, gopls/client:eglot, and so on, to record sessions with different editors or language clients. If you used multiple editors throughout the week, you might record the following counter data:

gopls/client:vscode 8
gopls/client:neovim 5
gopls/client:eglot  2

When counters are related in this way, we sometimes refer to the part before the : the chart name (gopls/client in this case), and the part after : as the bucket name (vscode). We’ll see why this matters when we discuss charts.

Basic counters can also represent a histogram. For example, the gopls/completion/latency:<50ms counter records the number of times an autocompletion takes less than 50ms.

gopls/completion/latency:<10ms
gopls/completion/latency:<50ms
gopls/completion/latency:<100ms
...

This pattern for recording histogram data is a convention: there’s nothing special about the <50ms bucket name. These types of counters are commonly used to measure performance.

Stack counters

A stack counter is a counter that also records the current call stack of the Go toolchain program when the count is incremented. For example, the crash/crash stack counter records the call stack when a toolchain program crashes:

crash/crash
golang.org/x/tools/gopls/internal/golang.hoverBuiltin:+22
golang.org/x/tools/gopls/internal/golang.Hover:+94
golang.org/x/tools/gopls/internal/server.Hover:+42
...

Stack counters typically measure events where program invariants are violated. The most common example of this is a crash, but another example is the gopls/bug stack counter, which counts unusual situations identified in advance by the programmer, such as a recovered panic or an error that “can’t happen”. Stack counters include only the names and line numbers of functions within Go toolchain programs. They don’t include any information about user inputs, such as the names or contents of a user’s source code.

Stack counters can help track down rare or tricky bugs that don’t get reported by other means. Since introducing the gopls/bug counter, we’ve found dozens of instances of “unreachable” code that was reached in practice, and tracking down these exceptions has led to the discovery (and fix) of many user-visible bugs that were either not obvious to the user or too difficult to report. Especially with prerelease testing, stack counters can help us improve the product more efficiently than we could without automation.

Counter files

All counter data is written to the <gotelemetry>/local directory, in files named according to the following schema:

[program name]@[program version]-[go version]-[GOOS]-[GOARCH]-[date].v1.count

These files are memory mapped into each running instance of the instrumented programs. The use of a memory-mapped file means that even if the program immediately crashes, or several copies of instrumented tools are running simultaneously, the counters are recorded safely.

Reporting and uploading

Approximately once a week, counter data gets aggregated into reports named <date>.json in the <gotelemetry>/local directory. These reports sum all of counts for the previous week, grouped by the same program identifiers used for the counter file (program name, program version, go version, GOOS, and GOARCH).

Local reports can be viewed as charts with the gotelemetry view command. Here’s an example summary of the gopls/completion/latency counter:

Uploading

If telemetry uploading is enabled, the weekly reporting process will also generate reports containing the subset of counters present in the upload config. These counters must be approved by the public review process described in the next section. After it has been successfully uploaded, a copy of the uploaded reports are stored in the <gotelemetry>/upload directory.

Once enough users opt in to uploading telemetry data, the upload process will randomly skip uploading for a fraction of reports, to reduce collection amounts and increase privacy while maintaining statistical significance.

The telemetry proposal process

Counters may be added to the upload configuration only through the telemetry proposal process, which proceeds as follows:

  1. The proposer files a proposal to upload new data. This is expressed in the form of a specific chart that will be displayed on telemetry.go.dev.
  2. Once discussion on the issue resolves, the proposal is approved or declined by a member of the Go team.
  3. The proposer sends a CL modifying the internal chart config to include the new chart.
  4. An automatic process regenerates the upload config to allow uploading of the counters required for the new chart. This process will also regularly add new versions of the relevant programs to the upload config as they are released.

In order to be approved, new charts cannot carry sensitive user information, and additionally must be both useful and feasible. In order to be useful, charts must serve a specific purpose, with actionable outcomes, that can’t be served by other means. For example, in order to collect a counter that measures the latency of a given operation, it must be shown that this latency can’t reasonably be measured via benchmarking, and that knowing the latency distribution will help meaningfully improve future versions of the program in question. In order to be feasible, it must be possible to reliably collect the requisite data, and the resulting measurements must be statistically significant. To demonstrate feasibility, the proposer may be asked to instrument the target program with counters and collect them locally first.

The full set of such proposals is available at the proposal project on GitHub.

Charts

In addition to accepting uploads, the telemetry.go.dev website makes uploaded data publicly available. Each day, uploaded reports are processed into two outputs, which are available on the telemetry.go.dev homepage.

Charts are specified in the chart config format. For example, here’s the chart config for the gopls/client chart.

title: Editor Distribution
counter: gopls/client:{vscode,vscodium,vscode-insiders,code-server,eglot,govim,neovim,coc.nvim,sublimetext,other}
description: measure editor distribution for gopls users.
type: partition
issue: https://go.dev/issue/61038
issue: https://go.dev/issue/62214 # add vscode-insiders
program: golang.org/x/tools/gopls
version: v0.13.0 # temporarily back-version to demonstrate config generation.

This configuration describes the chart to be produced, enumerates the set of counters to be aggregated, and specifies the program versions to which the chart applies. Additionally, the proposal process requires that an accepted proposal be associated with the chart. Here’s the chart resulting from that config:

IDE Prompting

For telemetry to answer the types of questions we want to ask of it, the set of users opting in to uploading need not be large–approximately 16,000 participants would allow for statistically significant measurements at the desired level of granularity. However, there is still a cost to assembling this healthy sample: we need to ask a large number of Go developers if they want to opt in.

Furthermore, even if a large number of users choose to opt in now (perhaps after reading a Go blog post), those users may be skewed toward experienced Go developers, and over time that initial sample will grow even more skewed. Also, as people replace their computers, they must actively choose to opt in again. In the telemetry blog post series, this is referred to as the “campaign cost” of the opt-in model.

To help keep the sample of participating users fresh, the Go language server gopls supports a prompt that asks users to opt in to Go telemetry. Here’s what that looks like from VS Code:

If users choose “Yes”, their telemetry mode will be set to on, just as if they had run gotelemetry on. In this way, opting in is as easy as possible, and we can continually reach a large and stratified sample of Go developers.

Frequently Asked Question

Q: How do I enable or disable Go telemetry?

A: Use the gotelemetry command, which can be installed with go install golang.org/x/telemetry/cmd/gotelemetry@latest. Run gotelemetry off to disable everything, even local collection. Run gotelemetry on to enable everything, including uploading approved counters to telemetry.go.dev. See the Configuration section for more info.

Q: Where does local data get stored?

A: In the os.UserConfigDir()/go/telemetry directory.

Q: How often does data get uploaded, if I opt in?

A: Approximately once a week.

Q: What data gets uploaded, if I opt in?

A: Only counters that are listed in the upload config may be uploaded. This is generated from the chart config, which may be more readable.

Q: How do counters get added to the upload config?

A: Through the public proposal process.

Q: Where can I see telemetry data that has been uploaded?

A: Uploaded data is available as charts or merged summaries at telemetry.go.dev.

Q: Where is the source code for Go telemetry?

A: At golang.org/x/telemetry.

go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.