Go Wiki: SlowBots

The Go build system supports “SlowBots”, which are a way to configure the TryBots (pre-submit builders) to add additional builders into the set of build configurations that TryBots normally run.

Normally TryBots only run things that are fast and elastically provisioned. That is, TryBots run tests for ports that are available on Google Cloud where we have tons of capacity and can spin up many VMs at will, sharding out test execution widely so the TryBots complete in 5-10 minutes.

But sometimes that’s not enough. SlowBots let you say that you’re cool waiting a long time until some specific set of builders becomes available. (There is often only one physical machine for some configurations, and often backlogged with work, and that builder might be slow too.)

Using SlowBots

Click “Choose Tryjobs” under the commit message and a dialog will appear.

A red box indicating the location of the “Choose Tryjobs” button under the commit message.

The dialog will ask you to click checkboxes for the builds you would like to run against your CL. A typical CL to the main Go repo usually wants builders starting with gotip-. See below for more details about the options.

An example of the Choose Tryjobs dialog.

Once you select the builds you would like to run, there are two ways to trigger the test runs:

An example of how to use Cq-Include-Trybots

Reviewer workflow

As a reviewer, you cannot edit commit messages. If a CL you are reviewing should run SlowBots, we recommend the following workflow:

  1. Select the desired builds in the “Choose Tryjobs” dialog.
  2. Click “Add” to immediately start the builds.
  3. Add an unresolved comment to the commit message asking the owner to add the exact Cq-Include-Trybots line from the dialog to the commit message.

(2) will provide immediate feedback from the results of the tests without waiting for the owner to upload a new patch set, while (3) will ensure that the tests continue running on future patch sets and block submission.

Note: https://crbug.com/40287467 tracks improvements to this process in LUCI to reduce toil.

SlowBot names

Each build’s name roughly indicates what it will do, but below is some more detail:

There are currently a lot more possible builds listed than what’s actually supported or valid.

Here are some general guidelines for which SlowBots will work as expected:

TODO: Apply these guidelines as filters automatically.

Pre-LUCI SlowBots

We’re currently in the middle of a migration to a new open-source CI system created by the Chromium project called LUCI. The above instructions describe how to run SlowBots on LUCI, but not all ports have been migrated to LUCI yet. In the interim, these ports are still available on the old infrastructure. Below are instructions on how to use SlowBots on the old infrastructure.

TRY=ppc64le, freebsd, netbsd-386, ios, linux-arm64-packet

… where the terms after TRY= are either:

For the main Go repository, the terms after TRY= can also be:

When running TryBots again later, the most recent TRY= comment on the current patchset is used. To turn it off set TRY= with an empty string after the equals sign. If the current patchset doesn’t have a TRY= comment, the most recent TRY= comment is used.

Pitfalls with Pre-LUCI SlowBots

This content is part of the Go Wiki.