Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: get standard library building with -d=checkptr #34972

Closed
mdempsky opened this issue Oct 17, 2019 · 72 comments
Closed

all: get standard library building with -d=checkptr #34972

mdempsky opened this issue Oct 17, 2019 · 72 comments
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@mdempsky
Copy link
Member

As a prerequisite for #34964, the standard library's tests need to all pass when -d=checkptr is enabled:

$ go test -a -short -gcflags=all=-d=checkptr std cmd

Currently, that's not the case.

  1. There are also some panics because of (*[Big]T)(p)[:n] expressions, like in package reflect. This code pattern should be recognized by cmd/compile.

  2. There are a few tests that fail because CL 201781 leads them to heap allocate when before things would only stack allocate; e.g., context.TestAllocs or database/sql.TestRawBytesAllocs. Not sure how to handle this; maybe for now the escape analysis change should only be enabled for -d=checkptr=2, and -race/-msan only enable -d=checkptr=1.

  3. In sync/atomic.hammerStoreLoadPointer, there's (in effect):

     new := uintptr(LoadPointer(addr))
     new += offset
     StorePointer(addr, unsafe.Pointer(new))
    

    This violates package unsafe's pointer rules: pointers have to be converted to uintptr and back to unsafe.Pointer in a single expression without being stored in a uintptr-typed variable, as is being done with new. (This just needs someone to grok exactly what the test is doing, and fix it to follow pointer rules correctly.)

There seem to be other failures still that need to be diagnosed. If folks want to try running the command above, claiming a failure, and then investigating what's going on, that would be great.

@mdempsky mdempsky added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 17, 2019
@mdempsky mdempsky added this to the Go1.14 milestone Oct 17, 2019
@mdempsky
Copy link
Member Author

/cc @odeke-em @cuonglm

@mdempsky
Copy link
Member Author

This code in package reflect is currently failing in checkptrAlignment, and seems to violate the safety rules because of the conversion to *[16]byte (sometimes gcdata points to <16 bytes):

go/src/reflect/type.go

Lines 2184 to 2192 in 8c6876e

kmask := (*[16]byte)(unsafe.Pointer(ktyp.gcdata))
for i := uintptr(0); i < ktyp.ptrdata/ptrSize; i++ {
if (kmask[i/8]>>(i%8))&1 != 0 {
for j := uintptr(0); j < bucketSize; j++ {
word := base + j*ktyp.size/ptrSize + i
mask[word/8] |= 1 << (word % 8)
}
}
}

(There's similar code for etype just below.)

@odeke-em
Copy link
Member

I'll cover the sync/atomic.hammer* tests fixup :)

@mdempsky
Copy link
Member Author

This failure looks genuinely suspicious to me:

$ go test -a -short -gcflags=all=-d=checkptr testing/quick
runtime: pointer 0xc00022a3e0 to unallocated span span.base()=0xc000222000 span.limit=0xc00023a000 span.state=3
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

@gopherbot
Copy link

Change https://golang.org/cl/201841 mentions this issue: all: disable tests that fail under -d=checkptr

@gopherbot
Copy link

Change https://golang.org/cl/201839 mentions this issue: cmd/compile: recognize (*[Big]T)(ptr)[:n:m] pattern for -d=checkptr

@mdempsky
Copy link
Member Author

Uploaded a couple CLs to improve -d=checkptr, and address some issues.

CL 201841 enumerates all of the remaining std cmd test failures that I'm aware of, but haven't investigated yet.

@odeke-em
Copy link
Member

Hey @mdempsky in regards to the sync/atomic.HammerLoad* tests that are flagged, what they do is that for 8 goroutines, run pairwise increments of the high and low parts of an int* or *uint and panic if high != low. To save the value at the end of the function, we invoke

StorePointer(addr, newValue)

where newValue is a raw uint64 value such as 0xc0000000c0 and that value is being written to addr.

checkptr tries to find the object associated with newValue but unfortunately that's a raw value that's not attached to any object and perhaps beyond Go's address space. Writing such code is a valid usecase and here is a minimal repro to test that edge case

package main

import (
	"fmt"
	"sync/atomic"
	"unsafe"
)

func main() {
	val := uint64(820338753727)
	addr := (*unsafe.Pointer)(unsafe.Pointer(&val))
	v := uintptr(atomic.LoadPointer(addr))
	atomic.StorePointer(addr, unsafe.Pointer(v+1+1<<32))

	xs := uintptr(atomic.LoadPointer(addr))
	fmt.Printf("xs: %#x\n", xs)
}
$ go run -gcflags=all=-d=checkptr main.go 
panic: (runtime.ptrArith) (0x10c09c0,0xc00000c080)

goroutine 1 [running]:
main.main()
	/Users/emmanuelodeke/Desktop/openSrc/bugs/golang/34972/main.go:13 +0x91
exit status 2

Trying to recognize cases that do plain arithmetic is going to be a tricky one :)

@gopherbot
Copy link

Change https://golang.org/cl/201877 mentions this issue: syscall: fix wrong unsafe.Pointer align in anyToSockaddr

@cuonglm
Copy link
Member

cuonglm commented Oct 18, 2019

Another instance in reflect:

return (*[1 << 20]*rtype)(add(unsafe.Pointer(t), uadd, "t.inCount > 0"))[:t.inCount]

and some lines below:

return (*[1 << 20]*rtype)(add(unsafe.Pointer(t), uadd, "outCount > 0"))[t.inCount : t.inCount+outCount]

@mdempsky
Copy link
Member Author

@odeke-em Thanks for looking into it.

Writing such code is a valid usecase

Regardless of the merit of the use case, if the example violates package unsafe's safety rules, then it's not valid Go.

Your repro case can be simplified to just:

package main

import (
	"fmt"
	"unsafe"
)

func main() {
	fmt.Println(unsafe.Pointer(uintptr(0xc000000000)))
}

This is failing because 0xc000000000 is an offset into the Go heap (at least where Go tries to allocate it by default on 64-bit OSes), and package unsafe's safety rules don't allow you to fabricate a pointer into the heap out of nothing.

To fix the test, it should either:

  1. Exclusively use pointers outside of the Go heap. (This is challenging, because you don't know where the Go heap is, and allocating non-Go-heap memory in a portable way is tricky.)

  2. Alternatively, allocate a large chunk of memory, find an adequately aligned pointer within that chunk of memory, and then restrict to modifying the lower bits of the address. For example:

     buf := make([]byte, 1 << 21) // allocate 2MB
     p := unsafe.Pointer(&buf[0])
     p = unsafe.Pointer((uintptr(p) + 1<<20 - 1) &^ (1<<20 - 1))  // align p to 1MB boundary
    

    p is now 20-bit aligned, and you can change any of the lower 20 bits and be guaranteed it still points into buf.

@bcmills
Copy link
Contributor

bcmills commented Oct 18, 2019

There are also some panics because of (*[Big]T)(p)[:n] expressions, like in package reflect. This code pattern should be recognized by cmd/compile.

I saw that one coming in #13656 (comment)!

One could easily envision a compiler that, say, sanity-checks that all pointers to arrays point to addresses that are mapped in the program's address space.

@bcmills
Copy link
Contributor

bcmills commented Oct 18, 2019

There are a few tests that fail because CL 201781 leads them to heap allocate when before things would only stack allocate

If so, those tests are overspecified: the language spec does not make any guarantees about allocations or escapes. They should be updated to skip (or use different thresholds) when they are not using a specific compiler in a specific configuration.

gopherbot pushed a commit that referenced this issue Oct 18, 2019
Caught with:

	go test -a -short -gcflags=all=-d=checkptr log/syslog

and:

	grep -rE '\*\[([^2]|.{2,})\].*\)\(unsafe.Pointer' syscall

Updates #34972

Change-Id: Iafd199b3a34beb7cc3e88484bf2fbae45183f951
Reviewed-on: https://go-review.googlesource.com/c/go/+/201877
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
@mdempsky
Copy link
Member Author

They should be updated to skip (or use different thresholds) when they are not using a specific compiler in a specific configuration.

Agreed, though I don't think currently there's a way for tests to check what compiler flags were used. (Keep in mind too that different compiler flags can be used for different packages.)

@cuonglm
Copy link
Member

cuonglm commented Oct 19, 2019

@mdempsky on OSX, I got a lot of:

fatal error: stack growth after fork

runtime stack:
runtime.throw(0x12e77be, 0x17)
	/Users/cuonglm/sources/go/src/runtime/panic.go:774 +0x72
runtime.newstack()
	/Users/cuonglm/sources/go/src/runtime/stack.go:928 +0xd04
runtime.morestack()
	/Users/cuonglm/sources/go/src/runtime/asm_amd64.s:449 +0x8f

goroutine 19 [running]:
runtime.checkptrAlignment(0xc00008a9e8, 0x126e840)
	/Users/cuonglm/sources/go/src/runtime/checkptr.go:14 +0x137 fp=0xc00008a8e8 sp=0xc00008a8e0 pc=0x1006177
syscall.funcPC(...)
	/Users/cuonglm/sources/go/src/syscall/syscall_darwin.go:469
syscall.forkAndExecInChild(0xc00008e120, 0xc000082ba0, 0x5, 0x5, 0xc00006a900, 0x2e, 0x2e, 0x0, 0x0, 0xc00008ac00, ...)
	/Users/cuonglm/sources/go/src/syscall/exec_darwin.go:65 +0xe6 fp=0xc00008aa60 sp=0xc00008a8e8 pc=0x10703d6
syscall.forkExec(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ac00, 0x1140b6b, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/syscall/exec_unix.go:201 +0x35b fp=0xc00008ab70 sp=0xc00008aa60 pc=0x10718eb
syscall.StartProcess(...)
	/Users/cuonglm/sources/go/src/syscall/exec_unix.go:248
os.startProcess(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ad98, 0x0, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/os/exec_posix.go:52 +0x2c0 fp=0xc00008ac58 sp=0xc00008ab70 pc=0x1093300
os.StartProcess(0xc0000c6160, 0x20, 0xc000094a40, 0x4, 0x4, 0xc00008ad98, 0x2d, 0x0, 0x0)
	/Users/cuonglm/sources/go/src/os/exec.go:102 +0x7c fp=0xc00008acb0 sp=0xc00008ac58 pc=0x1092d2c
os/exec.(*Cmd).Start(0xc0000bc2c0, 0xc00008af01, 0xc000082b70)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:416 +0x50c fp=0xc00008adf0 sp=0xc00008acb0 pc=0x116cbac
os/exec.(*Cmd).Run(0xc0000bc2c0, 0xc000082b70, 0xc0000bc2c0)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:338 +0x2b fp=0xc00008ae18 sp=0xc00008adf0 pc=0x116c63b
os/exec.(*Cmd).CombinedOutput(0xc0000bc2c0, 0x20, 0xc00008af30, 0x3, 0x3, 0xc0000bc2c0)
	/Users/cuonglm/sources/go/src/os/exec/exec.go:561 +0x91 fp=0xc00008ae48 sp=0xc00008ae18 pc=0x116d7a1
go/importer.TestForCompiler(0xc0000f0300)
	/Users/cuonglm/sources/go/src/go/importer/importer_test.go:23 +0x11d fp=0xc00008af70 sp=0xc00008ae48 pc=0x124dfed
testing.tRunner(0xc0000f0300, 0x12f26a0)
	/Users/cuonglm/sources/go/src/testing/testing.go:909 +0xc9 fp=0xc00008afd0 sp=0xc00008af70 pc=0x10f8249
runtime.goexit()
	/Users/cuonglm/sources/go/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc00008afd8 sp=0xc00008afd0 pc=0x105cd11
created by testing.(*T).Run
	/Users/cuonglm/sources/go/src/testing/testing.go:960 +0x351

when testing std, even after 46aa835

Note

This error does not happen on Linux.

@mdempsky
Copy link
Member Author

mdempsky commented Oct 19, 2019

@cuonglm Thanks for the report.

It looks like maybe checkptrAlignment and checkptrArithmetic need to be labeled //go:nosplit.

@gopherbot
Copy link

Change https://golang.org/cl/202157 mentions this issue: runtime: fix unsafe.Pointer alignment on Linux

@cuonglm
Copy link
Member

cuonglm commented Oct 19, 2019

@cuonglm Thanks for the report.

It looks like maybe checkptrAlignment and checkptrArithmetic need to be labeled //go:nosplit.

I tried, but still fails with go:nosplit for checkptrAlignment and checkptrArithmetic. But it does pass if findObject is marked go:nosplit, too.

@mdempsky
Copy link
Member Author

mdempsky commented Oct 19, 2019

Ugh, right.

Okay, in that case, we should probably just make //go:nosplit imply //go:nocheckptr, like how //go:cgo_unsafe_args does. (See CL 201823.)

@gopherbot
Copy link

Change https://golang.org/cl/202158 mentions this issue: cmd/compile: disable checkptr for //go:nosplit functions

@gopherbot
Copy link

Change https://golang.org/cl/202177 mentions this issue: windows, unix: fix wrong unsafe.Pointer alignment in syscall

gopherbot pushed a commit to golang/sys that referenced this issue Oct 20, 2019
Same as CL 201877 did for package syscall.

Updates golang/go#34972

Change-Id: I3929841ab32378516edafb1f02a84b1bdcc77bbd
Reviewed-on: https://go-review.googlesource.com/c/sys/+/202177
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/202580 mentions this issue: reflect, internal/reflectlite: set capacity when slicing unsafe pointers

gopherbot pushed a commit that referenced this issue Oct 21, 2019
A common idiom for turning an unsafe.Pointer into a slice is to write:

    s := (*[Big]T)(ptr)[:n:m]

This technically violates Go's unsafe pointer rules (rule #1 says T2
can't be bigger than T1), but it's fairly common and not too difficult
to recognize, so might as well allow it for now so we can make
progress on #34972.

This should be revisited if #19367 is accepted.

Updates #22218.
Updates #34972.

Change-Id: Id824e2461904e770910b6e728b4234041d2cc8bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/201839
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
gopherbot pushed a commit that referenced this issue Oct 21, 2019
Follow the idiom for allowing -d=checkptr to recognize and verify
correctness.

Updates #22218.
Updates #34972.

Change-Id: Ib6001c6f0e6dc535a36bcfaa1ae48e29e0c737f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/202580
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@aclements
Copy link
Member

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, since it takes the byte length of the sockaddr as an argument and hence could be used to make the kernel read arbitrary memory from the Go process. We try not to expose unsafe APIs from syscall, and go to some significant lengths to make APIs safe in the UNIX syscall packages (though this is certainly not the only unsafe API in the Windows syscall package).

Can we just remove WSASendTo? syscall is not covered by Go 1 compatibility, and this would fall under the security exception anyway.

@alexbrainman
Copy link
Member

WSASendto calls WSASendTo (which seems a little nuts to me, but okay...),

WSASendto was added in

https://golang.org/cl/3136042

These were the days where syscall package was free for all to do as we pleased.

What if we were to directly invoke the syscall from WSASendto with the unsafe.Pointer that came from to.sockaddr(), instead of converting it through an intermediate *RawSockaddrAny?

I will try and report it back here.

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, ...

You have to complain to Microsoft

https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsasendto

Even better. You should complain to people who designed Berkeley sockets in the first place.

Can we just remove WSASendTo?

We can do anything you want. But we are using the function ourselves in net package. And other people might be using it too. So we would have to replace it with something that call real WSASendTo anyway.

Alex

@aclements
Copy link
Member

I will try and report it back here.

Thanks!

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, ...

You have to complain to Microsoft
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsasendto
Even better. You should complain to people who designed Berkeley sockets in the first place.

Yeah, it's not great. But WSASendto exposes a perfectly safe API by taking the Sockaddr interface. That's unsafe internally, but only the syscall package can implement its methods, so that unsafeness doesn't leak through the user API. Sendto on UNIX works the same way. I think the mistake was just exporting the unsafe raw WSASendTo API underlying it (and some related functions. The UNIX packages also have an unsafe sendto API, but it's not exported.

Can we just remove WSASendTo?

We can do anything you want. But we are using the function ourselves in net package. And other people might be using it too.

I may be totally missing something, but I don't see any calls to WSASendTo in net (or x/net). I do see WSASendto. In fact, I couldn't find any calls to WSASendTo in the GitHub corpus I have access to (though I'm really confused about how much coverage that has).

@gopherbot
Copy link

Change https://golang.org/cl/220544 mentions this issue: syscall: fix windows WSASendto -d=checkptr violation

@alexbrainman
Copy link
Member

But WSASendto exposes a perfectly safe API by taking the Sockaddr interface. That's unsafe internally, but only the syscall package can implement its methods, so that unsafeness doesn't leak through the user API.

SGTM

I may be totally missing something, but I don't see any calls to WSASendTo in net ...

I did not explain myself properly. net package calls syscall.WSASendto and syscall.WSASendto calls syscall.WSASendTo. So net package calls syscall.WSASendTo indirectly. WSASendto Windows API is needed to implement net package. Whichever way we interface to it, but we must use WSASendto Windows API.

Anyway I fixed net.TestPacketConn in

https://go-review.googlesource.com/c/go/+/220544/

Unfortunately I discovered new broken test in net (I have no idea how I missed it before). This is the output after CL 220544 is applied

c:\Users\Alex\dev\go\src>go test -a -short -gcflags=all=-d=checkptr net
fatal error: checkptr: unsafe pointer conversion

goroutine 125 [running]:
runtime.throw(0x6a5112, 0x23)
        c:/users/alex/dev/go/src/runtime/panic.go:1112 +0x79 fp=0xc00017dc40 sp=0xc00017dc10 pc=0x4379c9
runtime.checkptrAlignment(0xc00020ced0, 0x668d00, 0x1)
        c:/users/alex/dev/go/src/runtime/checkptr.go:18 +0xbe fp=0xc00017dc70 sp=0xc00017dc40 pc=0x40689e
internal/poll.(*FD).WriteMsg(0xc000237400, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0x6eeec0, 0xc00020f020, 0x0, ...)
        c:/users/alex/dev/go/src/internal/poll/fd_windows.go:1130 +0x1f3 fp=0xc00017dce0 sp=0xc00017dc70 pc=0x4d3fa3
net.(*netFD).writeMsg(0xc000237400, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0x6eeec0, 0xc00020f020, 0xc00020f000, ...)
        c:/users/alex/dev/go/src/net/fd_windows.go:234 +0xb1 fp=0xc00017dd78 sp=0xc00017dce0 pc=0x5644d1
net.(*UDPConn).writeMsg(0xc0002100d8, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0xc000235200, 0x80, 0x80, ...)
        c:/users/alex/dev/go/src/net/udpsock_posix.go:94 +0x121 fp=0xc00017ddf0 sp=0xc00017dd78 pc=0x586621
net.(*UDPConn).WriteMsgUDP(0xc0002100d8, 0xc00020cec0, 0xc, 0xc, 0x0, 0x0, 0x0, 0xc000235200, 0xf, 0x85eea9, ...)
        c:/users/alex/dev/go/src/net/udpsock.go:186 +0xac fp=0xc00017de90 sp=0xc00017ddf0 pc=0x58477c
net.TestUDPConnSpecificMethods(0xc000228a20)
        c:/users/alex/dev/go/src/net/protoconn_test.go:143 +0x4d5 fp=0xc00017df80 sp=0xc00017de90 pc=0x5d98b5
testing.tRunner(0xc000228a20, 0x6abdd0)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3 fp=0xc00017dfd0 sp=0xc00017df80 pc=0x4ff473
runtime.goexit()
        c:/users/alex/dev/go/src/runtime/asm_amd64.s:1373 +0x1 fp=0xc00017dfd8 sp=0xc00017dfd0 pc=0x467b11
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 1 [chan receive]:
testing.(*T).Run(0xc000228a20, 0x6a155a, 0x1a, 0x6abdd0, 0x4c8701)
        c:/users/alex/dev/go/src/testing/testing.go:1044 +0x385
testing.runTests.func1(0xc00012c000)
        c:/users/alex/dev/go/src/testing/testing.go:1300 +0x7f
testing.tRunner(0xc00012c000, 0xc000071dd0)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
testing.runTests(0xc000004680, 0x887980, 0xbc, 0xbc, 0xbf8d542e05eb6248, 0x8bb3711725, 0x88b2a0, 0x10)
        c:/users/alex/dev/go/src/testing/testing.go:1298 +0x2df
testing.(*M).Run(0xc000112000, 0x0)
        c:/users/alex/dev/go/src/testing/testing.go:1210 +0x1ae
net.TestMain(0xc000112000)
        c:/users/alex/dev/go/src/net/main_test.go:52 +0x40
main.main()
        _testmain.go:478 +0x13c

goroutine 199 [chan receive]:
testing.(*T).Parallel(0xc0001a8fc0)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGoogleSRV(0xc0001a8fc0)
        c:/users/alex/dev/go/src/net/lookup_test.go:74 +0x4a
testing.tRunner(0xc0001a8fc0, 0x6aba50)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 200 [chan receive]:
testing.(*T).Parallel(0xc0001a90e0)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailMX(0xc0001a90e0)
        c:/users/alex/dev/go/src/net/lookup_test.go:123 +0x47
testing.tRunner(0xc0001a90e0, 0x6aba20)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 274 [chan receive]:
testing.(*T).Parallel(0xc00040c240)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestCloseUnblocksRead(0xc00040c240)
        c:/users/alex/dev/go/src/net/net_test.go:505 +0x32
testing.tRunner(0xc00040c240, 0x6ab628)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 202 [chan receive]:
testing.(*T).Parallel(0xc0001a9320)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailTXT(0xc0001a9320)
        c:/users/alex/dev/go/src/net/lookup_test.go:218 +0x4a
testing.tRunner(0xc0001a9320, 0x6aba30)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 201 [chan receive]:
testing.(*T).Parallel(0xc0001a9200)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestLookupGmailNS(0xc0001a9200)
        c:/users/alex/dev/go/src/net/lookup_test.go:169 +0x47
testing.tRunner(0xc0001a9200, 0x6aba28)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e

goroutine 275 [chan receive]:
testing.(*T).Parallel(0xc00040c360)
        c:/users/alex/dev/go/src/testing/testing.go:867 +0x1dc
net.TestNotTemporaryRead(0xc00040c360)
        c:/users/alex/dev/go/src/net/net_test.go:530 +0x32
testing.tRunner(0xc00040c360, 0x6abb48)
        c:/users/alex/dev/go/src/testing/testing.go:992 +0xe3
created by testing.(*T).Run
        c:/users/alex/dev/go/src/testing/testing.go:1043 +0x35e
FAIL    net     4.524s
FAIL

c:\Users\Alex\dev\go\src>

I will try and fix that too.

Alex

@gopherbot
Copy link

Change https://golang.org/cl/222457 mentions this issue: internal/syscall/windows: change WSAMsg.Name type

@gopherbot
Copy link

Change https://golang.org/cl/222855 mentions this issue: sha3: mark xorInUnaligned with go:nocheckptr

gopherbot pushed a commit to golang/crypto that referenced this issue Mar 11, 2020
It is unclear whether unaligned reads should be allowed, or if they
are even actually a good idea here. However, while we figure that out,
we should un-break 'go test -race' for users of this package.

Updates golang/go#37644
Updates golang/go#37298
Updates golang/go#37715
Updates golang/go#34972
Updates golang/go#35128

Change-Id: I088f5703023e4f05ee274a6753e925973f12ac1b
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/222855
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 2, 2020
WSASendto converts unsafe.Pointer to *syscall.RawSockaddrAny. But that
violates every rule of

https://golang.org/pkg/unsafe/#Pointer

Implement WSASendto by calling Windows WSASendTo API by calling
syscall.Syscall9 directly. This allows us to comply with

(4) Conversion of a Pointer to a uintptr when calling syscall.Syscall

rule.

After this change, this commands succeeds:

go test -a -short -gcflags=all=-d=checkptr -run=TestPacketConn net

Updates #34972

Change-Id: Ib9a810bedf9e05251b7d3c7f69e15bfbd177ac62
Reviewed-on: https://go-review.googlesource.com/c/go/+/220544
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Apr 2, 2020
The problem was discovered while running

go test -a -short -gcflags=all=-d=checkptr -run=TestUDPConnSpecificMethods net

WSAMsg is type defined by Windows. And WSAMsg.Name could point to two
different structures for IPv4 and IPV6 sockets.

Currently WSAMsg.Name is declared as *syscall.RawSockaddrAny. But that
violates

(1) Conversion of a *T1 to Pointer to *T2.

rule of

https://golang.org/pkg/unsafe/#Pointer

When we convert *syscall.RawSockaddrInet4 into *syscall.RawSockaddrAny,
syscall.RawSockaddrInet4 and syscall.RawSockaddrAny do not share an
equivalent memory layout.

Same for *syscall.SockaddrInet6 into *syscall.RawSockaddrAny.

This CL changes WSAMsg.Name type to *syscall.Pointer. syscall.Pointer
length is 0, and that at least makes type checker happy.

After this change I was able to run

go test -a -short -gcflags=all=-d=checkptr std cmd

without type checker complaining.

Updates #34972

Change-Id: Ic5c2321c20abd805c687ee16ef6f643a2f8cd93f
Reviewed-on: https://go-review.googlesource.com/c/go/+/222457
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@alexbrainman
Copy link
Member

I think I finished all -d=checkptr related changes. Running (on top of 801cd7c )

c:\Users\Alex\dev\go\src>go test -a -short -gcflags=all=-d=checkptr std cmd
ok      archive/tar     0.648s
ok      archive/zip     0.170s
ok      bufio   0.089s
ok      bytes   0.325s
ok      compress/bzip2  0.287s
ok      compress/flate  1.147s
ok      compress/gzip   0.250s
ok      compress/lzw    0.371s
ok      compress/zlib   0.106s
ok      container/heap  0.118s
ok      container/list  0.244s
ok      container/ring  0.082s
ok      context 0.151s
ok      crypto  0.526s
ok      crypto/aes      0.220s
ok      crypto/cipher   0.554s
ok      crypto/des      0.259s
ok      crypto/dsa      0.108s
ok      crypto/ecdsa    0.235s
ok      crypto/ed25519  0.359s
?       crypto/ed25519/internal/edwards25519    [no test files]
ok      crypto/elliptic 0.208s
ok      crypto/hmac     0.833s
?       crypto/internal/randutil        [no test files]
ok      crypto/internal/subtle  0.724s
ok      crypto/md5      0.830s
ok      crypto/rand     0.146s
ok      crypto/rc4      0.127s
ok      crypto/rsa      0.257s
ok      crypto/sha1     0.116s
ok      crypto/sha256   0.132s
ok      crypto/sha512   0.079s
ok      crypto/subtle   0.074s
ok      crypto/tls      4.314s
ok      crypto/x509     1.605s
?       crypto/x509/pkix        [no test files]
ok      database/sql    0.762s
ok      database/sql/driver     0.053s
ok      debug/dwarf     1.120s
ok      debug/elf       1.395s
ok      debug/gosym     0.157s
ok      debug/macho     0.434s
ok      debug/pe        12.299s
ok      debug/plan9obj  0.182s
?       encoding        [no test files]
ok      encoding/ascii85        0.093s
ok      encoding/asn1   0.063s
ok      encoding/base32 0.120s
ok      encoding/base64 0.098s
ok      encoding/binary 0.056s
ok      encoding/csv    0.084s
ok      encoding/gob    0.505s
ok      encoding/hex    0.495s
ok      encoding/json   0.447s
ok      encoding/pem    0.140s
ok      encoding/xml    0.096s
ok      errors  0.110s
ok      expvar  0.206s
ok      flag    0.159s
ok      fmt     0.157s
ok      go/ast  0.059s
ok      go/build        19.119s
ok      go/constant     0.113s
ok      go/doc  1.058s
ok      go/format       0.075s
ok      go/importer     0.401s
ok      go/internal/gccgoimporter       2.147s
ok      go/internal/gcimporter  2.170s
ok      go/internal/srcimporter 3.413s
ok      go/parser       0.403s
ok      go/printer      0.569s
ok      go/scanner      0.094s
ok      go/token        0.158s
ok      go/types        4.196s
ok      hash    0.051s
ok      hash/adler32    0.102s
ok      hash/crc32      0.217s
ok      hash/crc64      0.159s
ok      hash/fnv        0.160s
ok      hash/maphash    0.178s
ok      html    0.048s
ok      html/template   0.139s
ok      image   0.312s
ok      image/color     0.139s
?       image/color/palette     [no test files]
ok      image/draw      0.250s
ok      image/gif       0.747s
?       image/internal/imageutil        [no test files]
ok      image/jpeg      0.573s
ok      image/png       0.268s
ok      index/suffixarray       0.274s
?       internal/bytealg        [no test files]
?       internal/cfg    [no test files]
ok      internal/cpu    0.111s
ok      internal/fmtsort        0.096s
?       internal/goroot [no test files]
?       internal/goversion      [no test files]
?       internal/lazyregexp     [no test files]
?       internal/lazytemplate   [no test files]
?       internal/nettrace       [no test files]
?       internal/obscuretestdata        [no test files]
?       internal/oserror        [no test files]
ok      internal/poll   0.165s
?       internal/race   [no test files]
ok      internal/reflectlite    0.130s
ok      internal/singleflight   0.107s
?       internal/syscall/execenv        [no test files]
ok      internal/syscall/windows        0.085s
ok      internal/syscall/windows/registry       0.207s
?       internal/syscall/windows/sysdll [no test files]
?       internal/testenv        [no test files]
?       internal/testlog        [no test files]
ok      internal/trace  0.737s
ok      internal/xcoff  1.169s
ok      io      0.239s
ok      io/ioutil       0.302s
ok      log     0.078s
?       log/syslog      [no test files]
ok      math    0.138s
ok      math/big        1.337s
ok      math/bits       0.116s
ok      math/cmplx      0.072s
ok      math/rand       0.416s
ok      mime    0.526s
ok      mime/multipart  0.931s
ok      mime/quotedprintable    0.067s
ok      net     32.163s
ok      net/http        14.201s
ok      net/http/cgi    0.608s
ok      net/http/cookiejar      0.125s
ok      net/http/fcgi   0.891s
ok      net/http/httptest       2.162s
ok      net/http/httptrace      0.289s
ok      net/http/httputil       0.451s
ok      net/http/internal       0.299s
ok      net/http/pprof  2.254s
ok      net/internal/socktest   0.061s
ok      net/mail        0.079s
ok      net/rpc 0.357s
ok      net/rpc/jsonrpc 0.649s
ok      net/smtp        0.323s
ok      net/textproto   0.116s
ok      net/url 0.113s
ok      os      7.549s
ok      os/exec 7.068s
ok      os/signal       2.068s
ok      os/user 0.121s
ok      path    0.071s
ok      path/filepath   1.260s
ok      plugin  0.114s
ok      reflect 1.269s
ok      regexp  0.754s
ok      regexp/syntax   0.519s
ok      runtime 99.905s
?       runtime/cgo     [no test files]
ok      runtime/debug   0.096s
ok      runtime/internal/atomic 0.116s
ok      runtime/internal/math   0.060s
ok      runtime/internal/sys    0.190s
ok      runtime/pprof   13.023s
ok      runtime/pprof/internal/profile  0.085s
?       runtime/race    [no test files]
ok      runtime/trace   0.765s
ok      sort    0.101s
ok      strconv 0.690s
ok      strings 0.268s
ok      sync    0.583s
ok      sync/atomic     0.117s
ok      syscall 0.072s
ok      testing 0.737s
?       testing/internal/testdeps       [no test files]
ok      testing/iotest  0.135s
ok      testing/quick   0.270s
ok      text/scanner    0.106s
ok      text/tabwriter  0.071s
ok      text/template   0.125s
ok      text/template/parse     0.055s
ok      time    2.523s
ok      unicode 0.173s
ok      unicode/utf16   0.193s
ok      unicode/utf8    0.062s
?       unsafe  [no test files]
?       vendor/golang.org/x/crypto/chacha20     [no test files]
?       vendor/golang.org/x/crypto/chacha20poly1305     [no test files]
?       vendor/golang.org/x/crypto/cryptobyte   [no test files]
?       vendor/golang.org/x/crypto/cryptobyte/asn1      [no test files]
?       vendor/golang.org/x/crypto/curve25519   [no test files]
?       vendor/golang.org/x/crypto/hkdf [no test files]
?       vendor/golang.org/x/crypto/internal/subtle      [no test files]
?       vendor/golang.org/x/crypto/poly1305     [no test files]
?       vendor/golang.org/x/net/dns/dnsmessage  [no test files]
?       vendor/golang.org/x/net/http/httpguts   [no test files]
?       vendor/golang.org/x/net/http/httpproxy  [no test files]
?       vendor/golang.org/x/net/http2/hpack     [no test files]
?       vendor/golang.org/x/net/idna    [no test files]
?       vendor/golang.org/x/net/nettest [no test files]
?       vendor/golang.org/x/sys/cpu     [no test files]
?       vendor/golang.org/x/text/secure/bidirule        [no test files]
?       vendor/golang.org/x/text/transform      [no test files]
?       vendor/golang.org/x/text/unicode/bidi   [no test files]
?       vendor/golang.org/x/text/unicode/norm   [no test files]
ok      cmd/addr2line   20.369s
ok      cmd/api 24.461s
?       cmd/asm [no test files]
?       cmd/asm/internal/arch   [no test files]
ok      cmd/asm/internal/asm    4.127s
?       cmd/asm/internal/flags  [no test files]
ok      cmd/asm/internal/lex    0.800s
?       cmd/buildid     [no test files]
?       cmd/cgo [no test files]
ok      cmd/compile     0.624s
?       cmd/compile/internal/amd64      [no test files]
?       cmd/compile/internal/arm        [no test files]
?       cmd/compile/internal/arm64      [no test files]
ok      cmd/compile/internal/gc 18.001s
ok      cmd/compile/internal/logopt     3.718s
?       cmd/compile/internal/mips       [no test files]
?       cmd/compile/internal/mips64     [no test files]
?       cmd/compile/internal/ppc64      [no test files]
?       cmd/compile/internal/riscv64    [no test files]
?       cmd/compile/internal/s390x      [no test files]
ok      cmd/compile/internal/ssa        0.668s
ok      cmd/compile/internal/syntax     0.188s
ok      cmd/compile/internal/test       0.140s [no tests to run]
ok      cmd/compile/internal/types      0.047s
?       cmd/compile/internal/wasm       [no test files]
?       cmd/compile/internal/x86        [no test files]
ok      cmd/cover       11.431s
?       cmd/dist        [no test files]
ok      cmd/doc 0.266s
ok      cmd/fix 6.998s
go test proxy running at GOPROXY=http://127.0.0.1:62204/mod
go proxy: no archive rsc.io v1.5.2: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.0.0: file does not exist
go proxy: no archive rsc.io v1.1.0: file does not exist
go proxy: no archive rsc.io v1.5.1: file does not exist
go proxy: no archive example.com/newcycle v1.0.0: file does not exist
go proxy: no archive rsc.io v1.5.2: file does not exist
--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/test_race_install_cgo (1.32s)
        script_test.go:205:
            # Tests Issue #10500 (1.244s)
            > [!race] skip
            > [!darwin] ! stale cmd/cgo  # The darwin builders are spuriously stale; see #33598.
            FAIL: testdata\script\test_race_install_cgo.txt:5: cmd/cgo is unexpectedly stale


FAIL
FAIL    cmd/go  201.196s
ok      cmd/go/internal/auth    0.349s
?       cmd/go/internal/base    [no test files]
?       cmd/go/internal/bug     [no test files]
ok      cmd/go/internal/cache   5.082s
?       cmd/go/internal/cfg     [no test files]
?       cmd/go/internal/clean   [no test files]
?       cmd/go/internal/cmdflag [no test files]
?       cmd/go/internal/doc     [no test files]
?       cmd/go/internal/envcmd  [no test files]
?       cmd/go/internal/fix     [no test files]
?       cmd/go/internal/fmtcmd  [no test files]
ok      cmd/go/internal/generate        0.479s
ok      cmd/go/internal/get     0.385s
?       cmd/go/internal/help    [no test files]
ok      cmd/go/internal/imports 0.065s
?       cmd/go/internal/list    [no test files]
ok      cmd/go/internal/load    0.224s
ok      cmd/go/internal/lockedfile      0.381s
ok      cmd/go/internal/lockedfile/internal/filelock    0.321s
?       cmd/go/internal/modcmd  [no test files]
ok      cmd/go/internal/modconv 0.412s
ok      cmd/go/internal/modfetch        1.165s
ok      cmd/go/internal/modfetch/codehost       0.241s
ok      cmd/go/internal/modfetch/zip_sum_test   0.772s
?       cmd/go/internal/modget  [no test files]
?       cmd/go/internal/modinfo [no test files]
ok      cmd/go/internal/modload 0.237s
ok      cmd/go/internal/mvs     0.109s
ok      cmd/go/internal/par     0.078s
ok      cmd/go/internal/renameio        15.631s
?       cmd/go/internal/robustio        [no test files]
?       cmd/go/internal/run     [no test files]
ok      cmd/go/internal/search  0.304s
?       cmd/go/internal/str     [no test files]
ok      cmd/go/internal/test    1.983s
?       cmd/go/internal/tool    [no test files]
ok      cmd/go/internal/txtar   0.180s
?       cmd/go/internal/version [no test files]
?       cmd/go/internal/vet     [no test files]
ok      cmd/go/internal/web     0.654s
ok      cmd/go/internal/work    0.682s
ok      cmd/gofmt       1.146s
?       cmd/internal/bio        [no test files]
?       cmd/internal/browser    [no test files]
ok      cmd/internal/buildid    1.306s
?       cmd/internal/diff       [no test files]
ok      cmd/internal/dwarf      0.085s
ok      cmd/internal/edit       0.134s
?       cmd/internal/gcprog     [no test files]
ok      cmd/internal/goobj      2.878s
?       cmd/internal/goobj2     [no test files]
ok      cmd/internal/moddeps    8.717s
ok      cmd/internal/obj        0.080s
?       cmd/internal/obj/arm    [no test files]
ok      cmd/internal/obj/arm64  0.592s
?       cmd/internal/obj/mips   [no test files]
ok      cmd/internal/obj/ppc64  0.404s
ok      cmd/internal/obj/riscv  0.258s
?       cmd/internal/obj/s390x  [no test files]
?       cmd/internal/obj/wasm   [no test files]
ok      cmd/internal/obj/x86    6.724s
ok      cmd/internal/objabi     0.079s
?       cmd/internal/objfile    [no test files]
ok      cmd/internal/src        0.081s
?       cmd/internal/sys        [no test files]
ok      cmd/internal/test2json  0.789s
--- FAIL: TestDWARF (0.40s)
    dwarf_test.go:39: cmd/link is stale - run go install cmd/link
FAIL
FAIL    cmd/link        19.864s
?       cmd/link/internal/amd64 [no test files]
?       cmd/link/internal/arm   [no test files]
?       cmd/link/internal/arm64 [no test files]
ok      cmd/link/internal/ld    20.019s
?       cmd/link/internal/loadelf       [no test files]
?       cmd/link/internal/loader        [no test files]
?       cmd/link/internal/loadmacho     [no test files]
?       cmd/link/internal/loadpe        [no test files]
?       cmd/link/internal/loadxcoff     [no test files]
?       cmd/link/internal/mips  [no test files]
?       cmd/link/internal/mips64        [no test files]
?       cmd/link/internal/objfile       [no test files]
?       cmd/link/internal/ppc64 [no test files]
?       cmd/link/internal/riscv64       [no test files]
?       cmd/link/internal/s390x [no test files]
ok      cmd/link/internal/sym   0.407s
?       cmd/link/internal/wasm  [no test files]
?       cmd/link/internal/x86   [no test files]
ok      cmd/nm  15.778s
ok      cmd/objdump     5.883s
ok      cmd/pack        7.192s
?       cmd/pprof       [no test files]
?       cmd/test2json   [no test files]
ok      cmd/trace       1.206s
?       cmd/vendor/github.com/google/pprof/driver       [no test files]
?       cmd/vendor/github.com/google/pprof/internal/binutils    [no test files]
?       cmd/vendor/github.com/google/pprof/internal/driver      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/elfexec     [no test files]
?       cmd/vendor/github.com/google/pprof/internal/graph       [no test files]
?       cmd/vendor/github.com/google/pprof/internal/measurement [no test files]
?       cmd/vendor/github.com/google/pprof/internal/plugin      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/report      [no test files]
?       cmd/vendor/github.com/google/pprof/internal/symbolizer  [no test files]
?       cmd/vendor/github.com/google/pprof/internal/symbolz     [no test files]
?       cmd/vendor/github.com/google/pprof/internal/transport   [no test files]
?       cmd/vendor/github.com/google/pprof/profile      [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/d3       [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/d3flamegraph     [no test files]
?       cmd/vendor/github.com/google/pprof/third_party/svgpan   [no test files]
?       cmd/vendor/github.com/ianlancetaylor/demangle   [no test files]
?       cmd/vendor/golang.org/x/arch/arm/armasm [no test files]
?       cmd/vendor/golang.org/x/arch/arm64/arm64asm     [no test files]
?       cmd/vendor/golang.org/x/arch/ppc64/ppc64asm     [no test files]
?       cmd/vendor/golang.org/x/arch/x86/x86asm [no test files]
?       cmd/vendor/golang.org/x/crypto/ed25519  [no test files]
?       cmd/vendor/golang.org/x/crypto/ed25519/internal/edwards25519    [no test files]
?       cmd/vendor/golang.org/x/crypto/ssh/terminal     [no test files]
?       cmd/vendor/golang.org/x/mod/internal/lazyregexp [no test files]
?       cmd/vendor/golang.org/x/mod/modfile     [no test files]
?       cmd/vendor/golang.org/x/mod/module      [no test files]
?       cmd/vendor/golang.org/x/mod/semver      [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb       [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/dirhash       [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/note  [no test files]
?       cmd/vendor/golang.org/x/mod/sumdb/tlog  [no test files]
?       cmd/vendor/golang.org/x/mod/zip [no test files]
?       cmd/vendor/golang.org/x/sys/unix        [no test files]
?       cmd/vendor/golang.org/x/sys/windows     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/internal/analysisflags   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/internal/facts        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/asmdecl        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/assign [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/atomic [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/bools  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/buildtag       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/cgocall        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/composite      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/copylock       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/ctrlflow       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/errorsas       [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/httpresponse   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/ifaceassert    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/inspect        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/internal/analysisutil      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/loopclosure    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/lostcancel     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/nilfunc        [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/printf [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/shift  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/stdmethods     [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/stringintconv  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/structtag      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/tests  [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unmarshal      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unreachable    [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unsafeptr      [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/passes/unusedresult   [no test files]
?       cmd/vendor/golang.org/x/tools/go/analysis/unitchecker   [no test files]
?       cmd/vendor/golang.org/x/tools/go/ast/astutil    [no test files]
?       cmd/vendor/golang.org/x/tools/go/ast/inspector  [no test files]
?       cmd/vendor/golang.org/x/tools/go/cfg    [no test files]
?       cmd/vendor/golang.org/x/tools/go/types/objectpath       [no test files]
?       cmd/vendor/golang.org/x/tools/go/types/typeutil [no test files]
?       cmd/vendor/golang.org/x/xerrors [no test files]
?       cmd/vendor/golang.org/x/xerrors/internal        [no test files]
ok      cmd/vet 29.924s
FAIL

c:\Users\Alex\dev\go\src>

There are still 2 failures. But I don't think they are related to -d=checkptr.

I am not sure what to do next. Should I send windows version of

https://go-review.googlesource.com/c/go/+/201783/

?

Thank you.

Alex

@aclements
Copy link
Member

I think I finished all -d=checkptr related changes.

Yay!

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do they also happen without checkptr? I agree they certainly don't look related to checkptr, but the dashboard is clean for Windows right now.

I am not sure what to do next. Should I send windows version of https://go-review.googlesource.com/c/go/+/201783/ ?

That would be awesome.

@bcmills
Copy link
Contributor

bcmills commented Apr 2, 2020

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do those resolve if you run go install cmd without -d=checkptr before running go test?

@gopherbot
Copy link

Change https://golang.org/cl/227003 mentions this issue: cmd/compile: enable -d=checkptr even on windows

@alexbrainman
Copy link
Member

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do they also happen without checkptr?

Do they also happen without checkptr?

Yes. Running

go test -a -short std cmd

fails in exactly the same way as

go test -a -short -gcflags=all=-d=checkptr std cmd

command.

... Should I send windows version of https://go-review.googlesource.com/c/go/+/201783/ ?

That would be awesome.

I sent https://golang.org/cl/227003 But it fails here when I run all.bat with this error:

--- FAIL: TestLogOpt (0.63s)
    --- FAIL: TestLogOpt/Copy (0.27s)
        --- FAIL: TestLogOpt/Copy/arm (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build210684795: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/arm64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build080355363: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/386 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build929204411: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/amd64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build049316543: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/mips (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build251041127: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/mips64 (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build593677423: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/ppc64le (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build772328963: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/s390x (0.02s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build555385451: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
        --- FAIL: TestLogOpt/Copy/wasm (0.08s)
            logopt_test.go:199: [c:\Users\Alex\dev\go\bin\go.exe tool compile -p x -json=0,file://log/opt -o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.o C:\Users\Alex\AppData\Local\Temp\TestLogOpt659564243\copy.go]
            logopt_test.go:204: go: creating work dir: mkdir C:\WINDOWS\go-build914568571: Access is denied.
            logopt_test.go:131: -json=0,file://log/opt should have succeeded
            logopt_test.go:135: -json=0,file://log/opt missing expected log file
            logopt_test.go:138:
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:37: did not see phrase {"range":{"start":{"line":9,"character":2},"end":{"line":9,"character":2}},"severity":3,"code":"copy","source":"go compiler","message":"128 bytes"} in
            logopt_test.go:43: expected exactly 2 occurences of "code":"copy" in
FAIL
FAIL    cmd/compile/internal/logopt     0.741s

Mind you, I created CL on current tip. And current tip has new linker code. Maybe failure is related. I did not investigate it properly.

There are still 2 failures. But I don't think they are related to -d=checkptr.

Do those resolve if you run go install cmd without -d=checkptr before running go test?

Running

go install cmd

before

go test -a -short -gcflags=all=-d=checkptr std cmd

makes things worse (I am showing just errors)

--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/test_relative_import_dash_i (0.48s)
        script_test.go:205:
            # Relative imports in go test -i
            # Run tests outside GOPATH. (0.000s)
            # Check that it's safe to pass -i (which installs dependencies in $GOPATH/pkg) to go test. (0.463s)
            > ! stale runtime # don't let test -i overwrite runtime
            FAIL: testdata\script\test_relative_import_dash_i.txt:7: runtime is unexpectedly stale


    --- FAIL: TestScript/test_race_install_cgo (2.53s)
        script_test.go:205:
            # Tests Issue #10500 (2.500s)
            > [!race] skip
            > [!darwin] ! stale cmd/cgo  # The darwin builders are spuriously stale; see #33598.
            FAIL: testdata\script\test_race_install_cgo.txt:5: cmd/cgo is unexpectedly stale


    --- FAIL: TestScript/build_package_not_stale_trailing_slash (0.16s)
        script_test.go:205:
            # Tests Issue #12690 (0.160s)
            > [gccgo] skip 'gccgo does not have GOROOT'
            > ! stale runtime
            FAIL: testdata\script\build_package_not_stale_trailing_slash.txt:5: runtime is unexpectedly stale


FAIL
FAIL    cmd/go  195.196s

and

--- FAIL: TestDWARF (1.32s)
    dwarf_test.go:39: cmd/link is stale - run go install cmd/link
FAIL
FAIL    cmd/link        16.210s

All tested on top of 801cd7c.

Alex

gopherbot pushed a commit that referenced this issue Apr 5, 2020
CL 201783 enable -d=checkptr when -race or -msan is specified
everywhere but windows.

But, now that all unsafe pointer conversions in the standard
library are fixed, enable -d=checkptr even on windows.

Updates #34964
Updates #34972

Change-Id: Id912fa83b0d5b46c6f1c134c742fd94d2d185835
Reviewed-on: https://go-review.googlesource.com/c/go/+/227003
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@alexbrainman
Copy link
Member

I sent https://golang.org/cl/227003 But it fails here when I run all.bat with this error:

--- FAIL: TestLogOpt (0.63s)
    --- FAIL: TestLogOpt/Copy (0.27s)
        --- FAIL: TestLogOpt/Copy/arm (0.02s)

TestLogOpt failure is unrelated to CL 227003. I filled #38251 for that.

Alex

@alexbrainman
Copy link
Member

And I submitted CL 227003. So I don't see what else to do for this current issue. Leaving it for others to close.

Alex

@mdempsky
Copy link
Member Author

Thanks @alexbrainman!

@gopherbot
Copy link

Change https://golang.org/cl/228858 mentions this issue: syscall, internal/syscall/windows: remove utf16PtrToString parameter

@gopherbot
Copy link

Change https://golang.org/cl/225418 mentions this issue: windows: fix -d=checkptr slice failures

gopherbot pushed a commit that referenced this issue May 3, 2020
CL 208617 introduced syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString functions.

Original version of CL 208617 did not include syscall.utf16PtrToString
and internal/syscall/windows.UTF16PtrToString max parameter. The
parameter was added by Brad at the request of Ian. Ian said:

"In some cases it seems at least possible that the null terminator is
not present. I think it would be safer if we passed a maximum length
here."

The syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString function are designed to work
with only null terminated strings. So max parameter is superfluous.

This change removes max parameter.

Updates #34972

Change-Id: Ifea65dbd86bca8a08353579c6b9636c6f963d165
Reviewed-on: https://go-review.googlesource.com/c/go/+/228858
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit to golang/sys that referenced this issue May 19, 2020
This CL fixes unsafe casts to slices that are missing length or capacity.

Running tests with -d=checkptr enabled may panic on casting unsafe.Pointer to a static array of large predefined length, that is most likely much bigger than the size of the actual array in memory. Checkptr check is not satisfied if slicing operator misses length and capacity arguments `(*[(1 << 30) - 1]uint16)(unsafe.Pointer(p))[:]`, or when there is no slicing at all `(*[(1 << 30) - 1]uint16)(unsafe.Pointer(p))`.

To find all potential cases I used `grep -nr ")(unsafe.Pointer(" ./windows`, then filtered out safe casts when object size is always static and known at compile time.

To reproduce the issue run tests with checkptr enabled `go test -a -gcflags=all=-d=checkptr ./windows/...`.

Updates golang/go#34972
Fixes golang/go#38355

Change-Id: I9dd2084b4f9fb7618cdb140fb2f38b56b6d6cc04
GitHub-Last-Rev: 73288ad
GitHub-Pull-Request: #65
Reviewed-on: https://go-review.googlesource.com/c/sys/+/225418
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
xujianhai666 pushed a commit to xujianhai666/go-1 that referenced this issue May 21, 2020
CL 208617 introduced syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString functions.

Original version of CL 208617 did not include syscall.utf16PtrToString
and internal/syscall/windows.UTF16PtrToString max parameter. The
parameter was added by Brad at the request of Ian. Ian said:

"In some cases it seems at least possible that the null terminator is
not present. I think it would be safer if we passed a maximum length
here."

The syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString function are designed to work
with only null terminated strings. So max parameter is superfluous.

This change removes max parameter.

Updates golang#34972

Change-Id: Ifea65dbd86bca8a08353579c6b9636c6f963d165
Reviewed-on: https://go-review.googlesource.com/c/go/+/228858
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@golang golang locked and limited conversation to collaborators Apr 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants