Gopls: Implementation

Last major update: Jan 16 2024

This doc presents a high-level overview of the structure of gopls to help new contributors find their way. It is not intended to be a complete description of the implementation, nor even of any key components; for that, the package documentation (linked below) and other comments within the code are a better guide.

The diagram below shows selected components of the gopls module and their relationship to each other according to the Go import graph. Tests and test infrastructure are not shown, nor are utility packages, nor packages from the x/tools module. For brevity, packages are referred to by their last segment, which is usually unambiguous.

The height of each blob corresponds loosely to its technical depth. Some blocks are wide and shallow, such as protocol, which declares Go types for the entire LSP protocol. Others are deep, such as cache and golang, as they contain a lot of dense logic and algorithms.

Starting from the bottom, we’ll describe the various components.

The lowest layer defines the request and response types of the Language Server Protocol:

• The protocol package defines the standard protocol; it is mostly generated mechanically from the schema definition provided by Microsoft. The most important type is DocumentURI, which represents a file: URL that identifies a client editor document. It also provides Mapper, which maps between the different coordinate systems used for source positions: UTF-8, UTF-16, and token.Pos.

• The command package defines Gopls’s non-standard commands, which are all invoked through the workspace/executeCommand extension mechanism. These commands are typically returned by the server as continuations of Code Actions or Code Lenses; most clients do not construct calls to them directly.

The next layer defines a number of important and very widely used data structures:

• The file package defines the primary abstractions of a client file: its Identity (URI and content hash), and its Handle (which additionally provides the version and content of a particular snapshot of the file.

• The parsego package defines File, the parsed form of a Go source file, including its content, syntax tree, and coordinary mappings (Mapper and token.File). The package performs various kinds of tree repair to work around error-recovery shortcomings of the Go parser.

• The metadata package defines Package, an abstraction of the metadata of a Go package, similar to the output of go list -json. Metadata is produced from go/packages, which takes care of invoking go list. (Users report that it works to some extent with a GOPACKAGESDRIVER for Bazel, though we maintain no tests for this scenario.)

  The package also provides Graph, the complete import graph for a workspace; each graph node is a Package.

The settings layer defines the data structure (effectively a large tree) for gopls configuration options, along with its JSON encoding.

The cache layer is the largest and most complex component of gopls. It is concerned with state management, dependency analysis, and invalidation: the Session of communication with the client; the Folders that the client has opened; the View of a particular workspace tree with particular build options; the Snapshot of the state of all files in the workspace after a particular edit operation; the contents of all files, whether saved to disk (DiskFile) or edited and unsaved (Overlay); the Cache of in-memory memoized computations, such as parsing go.mod files or build the symbol index; and the Package, which holds the results of type checking a package from Go syntax.

The cache layer depends on various auxiliary packages, including:

• The filecache package, which manages gopls’ persistent, transactional, file-based key/value store.

• The xrefs, methodsets, and typerefs packages define algorithms for constructing indexes of information derived from type-checking, and for encoding and decoding these serializable indexes in the file cache.

  Together these packages enable the fast restart, reduced memory consumption, and synergy across processes that were delivered by the v0.12 redesign and described in “Scaling gopls for the growing Go ecosystem”.

The cache also defines gopls’s go/analysis driver, which runs modular analysis (similar to go vet) across the workspace. Gopls also includes a number of analysis passes that are not part of vet.

The next layer defines four packages, each for handling files in a particular language: mod for go.mod files; work for go.work files; template for files in text/template syntax; and golang, for files in Go itself. This package, by far the largest, provides the main features of gopls: navigation, analysis, and refactoring of Go code. As most users imagine it, this package is gopls.

The server package defines the LSP service implementation, with one handler method per LSP request type. Each handler switches on the type of the file and dispatches to one of the four language-specific packages.

The lsprpc package connects the service interface to our jsonrpc2 server.

Bear in mind that the diagram is a dependency graph, a “static” viewpoint of the program’s structure. A more dynamic viewpoint would order the packages based on the sequence in which they are encountered during processing of a particular request; in such a view, the bottom layer would represent the “wire” (protocol and command), the next layer up would hold the RPC-related packages (lsprpc and server), and features (e.g. golang, mod, work, template) would be at the top.

The cmd package defines the command-line interface of the gopls command, around which gopls’s main package is just a trivial wrapper. It is usually run without arguments, causing it to start a server and listen indefinitely. It also provides a number of subcommands that start a server, make a single request to it, and exit, providing traditional batch-command access to server functionality. These subcommands are primarily provided as a debugging aid; but see https://go.dev/issue/63693.

The source files for this documentation can be found beneath golang.org/x/tools/gopls/doc.