Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: Either gopls cache always invalidates on 2nd save for M1 mac (or darwin, not sure) or vs code extension host fails, same config works perfectly on linux. #54007

Closed
rdavis552 opened this issue Jul 16, 2022 · 4 comments
Assignees
Labels
gopls Issues related to the Go language server, gopls. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@rdavis552
Copy link

rdavis552 commented Jul 16, 2022

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
    • <1.18.4>
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
    • <0.9.1>
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
    • <1.69.1>
  • Check your installed extensions to get the version of the VS Code Go extension
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
GOBIN: undefined
toolsGopath: /Users/richardd/.vscode/godev
gopath: /Users/richardd/go
GOROOT: /usr/local/go
PATH: /Users/richardd/.nvm/versions/node/v16.2.0/bin:/opt/dropbox-override/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/whoseisit/bin:/usr/local/engtools/darwin/bin:/usr/local/engtools/common/bin:/usr/local/go/bin:/opt/homebrew/bin:/usr/local/arcanist/bin:/Library/Apple/usr/bin:/Users/richardd/go/bin:/usr/local/go/bin

	go:	/usr/local/go/bin/go: go version go1.18.4 darwin/arm64

	gotests:	not installed
	gomodifytags:	not installed
	impl:	not installed
	goplay:	not installed
	staticcheck:	/Users/richardd/.vscode/godev/bin/staticcheck	(version: v0.3.2 built with go: go1.18.4)
	gopls:	/Users/richardd/.vscode/godev/bin/gopls	(version: v0.9.1 built with go: go1.18.4)

go env
Workspace Folder (rSERVER): /Users/richardd/src/server
	GO111MODULE=""
	GOARCH="arm64"
	GOBIN=""
	GOCACHE="/Users/richardd/Library/Caches/go-build"
	GOENV="/Users/richardd/Library/Application Support/go/env"
	GOEXE=""
	GOEXPERIMENT=""
	GOFLAGS=""
	GOHOSTARCH="arm64"
	GOHOSTOS="darwin"
	GOINSECURE=""
	GOMODCACHE="/Users/richardd/go/pkg/mod"
	GONOPROXY=""
	GONOSUMDB=""
	GOOS="darwin"
	GOPATH="/Users/richardd/go"
	GOPRIVATE=""
	GOPROXY="https://proxy.golang.org,direct"
	GOROOT="/usr/local/go"
	GOSUMDB="sum.golang.org"
	GOTMPDIR=""
	GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
	GOVCS=""
	GOVERSION="go1.18.4"
	GCCGO="gccgo"
	AR="ar"
	CC="clang"
	CXX="clang++"
	CGO_ENABLED="1"
	GOMOD="/dev/null"
	GOWORK="/Users/richardd/src/server/go.work"
	CGO_CFLAGS="-g -O2"
	CGO_CPPFLAGS=""
	CGO_CXXFLAGS="-g -O2"
	CGO_FFLAGS="-g -O2"
	CGO_LDFLAGS="-g -O2"
	PKG_CONFIG="pkg-config"
	GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_8/bycsrbvd2dv2pptr3nqnyyqh0000gn/T/go-build3856595532=/tmp/go-build -gno-record-gcc-switches -fno-common">

Share the Go related settings you have added/edited

Run Preferences: Open Settings (JSON) command to open your settings.json file.
Share all the settings with the go. or ["go"] or gopls prefixes.

I can't show all, namely directory filters, for work reasons. I have also changed these configuration settings every which way trying to get a viable solution running over the past week. [Below are just snippets in case someone is curious about formatting].

    "go.useLanguageServer": true,
    "editor.maxTokenizationLineLength": 50000,
    "go.toolsGopath": "~/.vscode/godev",
    "go.trace.server": "verbose",
        "go.lintOnSave": "off",
    "go.vetOnSave": "off",
    "go.buildOnSave": "off",
    "gopls": {
      "verboseOutput": true,
      "build.memoryMode": "DegradeClosed", //Same situation with this off or on
      "ui.diagnostic.staticcheck": false,
      "build.experimentalUseInvalidMetadata": true,    
      "ui.navigation.importShortcut": "Definition",
      "ui.documentation.linksInHover": false,
      "ui.codelenses": {
        "generate": true,
        "regenerate_cgo": false,
        "tidy": false, 
        "upgrade_dependency": false,
      },    

Describe the bug

A clear and concise description of what the bug.
We recently transitioned a very large 70gb+ monorepo from GOPATH to go.work/go.mod. Use of the language server has definitely been a bumpy transition for local M1 mac devs.

On workspace open from configuration file, there is an initial ~1 minute workspace load time (no problem at all, it is a large repo). Once things have loaded, and a specific go file is opened, everything is initially populated great. All inlay hover actions work great, errors are populated if they exist before hand, and even with normal memory mode on, all directory wide errors are populated. You can even make an edit to the code and save once with inlay hovers continuing to work.

However, once you run a second save, there seems to be some cache invalidation as you experience the classic "waiting for code actions from Go" pop-up and inlay hovers just read "loading". After about 2 minutes, the loading/waiting is done and the full code base is completely usable by gopls. Everything is linted as you need it on save, all hover actions work, everything works fine.

This occurs every time you load the workspace. It's almost as if the initial load is populating cache somewhere else or something. Verbose gopls server traces show it's always the second fixImports call on save.

A couple of findings:
1.) I added quite a bit of logging to the go extensions goMain.js file. In most of the logs, you can see that it completed registering the language provider/document symbols after the hang up ( see the jump from 11:49 to 11:52 in image 1). In others, it reports that the extension host has started profiling the connection and attached the debugger with document symbols returning after a ~1 minute jump (image 2).

image

Screenshot by Dropbox Capture

This is confirmed every single time this occurs by vs code extension logs reporting something along the lines of the second the hang up occurs:
[2022-07-15 11:22:42.390] [main] [info] Enabling inspect port on extension host with pid 47681

2.) I then switched over to adding a lot of logging to the gopls bin. This reports that it was able to find and return the document symbols in a timely manner.

3.) This identical configuration works perfectly on our linux remote hosts. There is an initial ~1 minute load and then things work great. It seems like it's actually able to call and retrieve the data quick as well. Sometimes there is a momentary hang up with "loading" on hover but the data is returned and hover inlays populated in ~1-2 seconds.

Some thoughts:
1.) Potential culprit is a few +linux .go files we have that are reported when gopls is initial building the workspace. You receive a lot of No dep handle found for {+linux file name}. I have tried to remove these package directories via the build.DirectoryFilters but they are always processed both when the workspace loads and when fixImports occurs.
When I add a:

      "build.buildFlags": [
        "-tags=linux",

or use test as the build tag, these errors are not reported but the hang up still occurs.
I had a thought that maybe gopls was building these with the linux flag and storing them in modcache as such so have since removed them.

2.) One interesting thing I noticed is sometimes if you completely remove all go tools, modcache, and the go vs code extension, then reinstall them, and load the workspace, things work fine: you have the initial workspace load, traverse to a go file, and things run snappy. This kind of reinforced my thoughts on removing the linux build tag.

3.) I had another hunch that this was potentially dlv related. It doesn't appear as though dlv is available for vscode with an m1, it didn't pop up as an installable tool until I set:

    "go.toolsEnvVars": {
      "GOOS": "linux",
      }

dlv is of course immediately available on our linux builds. When I saw the Debugger attached log from Extension Host while viewing the goMain.js logs, I thought maybe that was dlv attaching itself to gopls. I'm not sure on this one at all, just worth noting.

4.) Running  gopls -rpc.trace -v check path/to/file.go takes about ~30 seconds, performs a full sweep of our go workspace form the looks of it and reports the same no dep handle for and c compiler issues. It just runs faster from the looks of it.

A clear and concise description of what you expected to happen.
Go through the initial ~1 minute workspace load then have gopls be usable for the entire stack, or at least module, on an M1 mac.

Steps to reproduce the behavior:

  • This might be tricky, I'd be happy to dm as it would most likely involve disclosing some private information. I believe y'all have been working with a colleague on creating a map for the cache.
  1. Go to '...'
  2. Click on '....'
  3. See error

Screenshots or recordings

If applicable, add screenshots or recordings to help explain your problem.

@rdavis552 rdavis552 changed the title Either gopls cache always invalidates on 2nd save for M1 mac (or darwin, not sure) or vs code extension host fails, same config works perfectly on linux (most likely +linux no op in this file) Either gopls cache always invalidates on 2nd save for M1 mac (or darwin, not sure) or vs code extension host fails, same config works perfectly on linux. Jul 16, 2022
@hyangah hyangah added the gopls Issues related to the Go language server, gopls. label Jul 20, 2022
@jamalc jamalc transferred this issue from golang/vscode-go Jul 22, 2022
@jamalc jamalc added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 22, 2022
@jamalc jamalc added this to the gopls/later milestone Jul 22, 2022
@jamalc jamalc changed the title Either gopls cache always invalidates on 2nd save for M1 mac (or darwin, not sure) or vs code extension host fails, same config works perfectly on linux. x/tools/gopls: Either gopls cache always invalidates on 2nd save for M1 mac (or darwin, not sure) or vs code extension host fails, same config works perfectly on linux. Jul 22, 2022
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Jul 22, 2022
@findleyr
Copy link
Contributor

Sorry for the slow response on this (several folks have been on vacation here), and thank very much for the detailed analysis!

I'd like to try investigating, if you're still able to help.

One starting point would be to try installing gopls@master. We're working very actively on cache invalidation and performance, and so it's worth confirming that the issue persists at tip.

git clone https://go.googlesource.com/tools
cd tools/gopls
go install .

Just to be clear: you are saving a .go file, right? In general that should be a no-op (from a state change perspective), but saving a go.work or go.mod file can trigger reloading.

Naively, I can think of a couple potential culprits:

  • fixImports is asking for a bunch of state that wasn't initialized by the initial workspace load, because it is misconfigured and getting cache misses.
  • the goimports module resolver (which is somewhat out-of-band of gopls' cache) is just much slower to initialize on Darwin, due to e.g. different syscall performance.

Needless to say a repro would be extremely helpful here, but I think you've also given us enough information to start investigating.

@findleyr findleyr modified the milestones: gopls/later, gopls/v0.9.2 Jul 28, 2022
@findleyr findleyr self-assigned this Aug 9, 2022
@findleyr
Copy link
Contributor

findleyr commented Aug 9, 2022

I'm trying to understand this, and have a few additional questions:

No dep handle found for {+linux file name}

This doesn't make sense to me. That dep should be a package ID, not a file name. Can you share more details about this error message?

c compiler issues

Can you tell me more about the c compiler issues you experience?

One interesting thing I noticed is sometimes if you completely remove all go tools, modcache, and the go vs code extension, then reinstall them, and load the workspace, things work fine

Hmm. Can you try just clearing your modcache? (and then not running a subsequent build)? Do the problems go away in that case? (perhaps the slowness is experienced during module cache scanning). On that note: do you have any other external processes that may populate the module cache on the first save?

Finally, can you try installing the next gopls version to confirm that the problem persists? There are a large number of changes related to loading in this release, and it would be good to confirm that we still see the same behavior.

go install golang.org/x/tools/gopls@v0.9.2-pre.2

@findleyr
Copy link
Contributor

I believe the root cause of this issue may be https://go.dev/issue/59216.

That issue is slated for v0.12.0, but this issue is not otherwise actionable. Moving this issue to v0.13.0 where we should have more data about the performance of gopls' new architecture.

@findleyr
Copy link
Contributor

We encountered a 100% reproducer for this bug in #60089, and were able to understand it: the hanging appears to be client-side, and is related to the size of the file watching patterns gopls was serving to VS Code. It is only reproducible with two saves in short order.

We fixed by transposing our watch patterns (more, smaller patterns).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Issues related to the Go language server, gopls. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

6 participants