Thank you for raising this issue. Can you please provide more information, a sample program which demonstrates this, or if that is not possible, the pprof trace you used to identify the lock contention.

Thank you

Theoretically this could be made more efficient with sync/atomic, although I think a sync.RWMutex would still be more than sufficient.

If you have 64 hardware threads, sync.RWMutex is unlikely to do any better than sync.Mutex: you'll just transform the lock contention to cache contention. (See #17973.)

As a workaround, you can guard slow Err() call with a non-blocking Done() read:

done := ctx.Done()
for something {
  select {
  case <-done:
    return ctx.Err()
  default:
  }
}

FWIW, this just showed up in some of our profiles recently. It was only 1% of our overall CPU time, but I expected 0.

Here's a benchmark:

func BenchmarkContextCancelDone(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-ctx.Done():
                        default:
                        }
                }
        })
}

Profile:

dev:issue42564_cancelctx $ go tool pprof prof.cpu
File: issue42564_cancelctx.test
Type: cpu
Time: Jan 15, 2021 at 8:41pm (PST)
Duration: 1.51s, Total samples = 3.40s (225.88%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2630ms, 77.35% of 3400ms total
Dropped 12 nodes (cum <= 17ms)
Showing top 10 nodes out of 67
      flat  flat%   sum%        cum   cum%
     890ms 26.18% 26.18%     1280ms 37.65%  sync.(*Mutex).lockSlow
     590ms 17.35% 43.53%     2500ms 73.53%  context.(*cancelCtx).Done
     250ms  7.35% 50.88%      620ms 18.24%  sync.(*Mutex).Unlock (inline)
     220ms  6.47% 57.35%      220ms  6.47%  runtime.usleep
     150ms  4.41% 61.76%      150ms  4.41%  runtime.cansemacquire (inline)
     140ms  4.12% 65.88%      280ms  8.24%  runtime.lock2
     120ms  3.53% 69.41%      120ms  3.53%  runtime.futex
      90ms  2.65% 72.06%       90ms  2.65%  runtime.osyield
      90ms  2.65% 74.71%       90ms  2.65%  runtime.procyield
      90ms  2.65% 77.35%      120ms  3.53%  runtime.unlock2

If you cache the ctx.Done() value, though:

func BenchmarkContextCancelDone_Workaround(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        done := ctx.Done()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-done:
                        default:
                        }
                }
        })
}

Then it's much faster:

BenchmarkContextCancelDone
BenchmarkContextCancelDone-8                     8253436               164.1 ns/op
BenchmarkContextCancelDone_Workaround
BenchmarkContextCancelDone_Workaround-8         504514413                2.365 ns/op

Looks like this is due to @josharian's 986768d (context: lazily initialize cancelCtx done channel) 😄 which had good numbers, but didn't consider mutex contention.

@josharian elsewhere suggested we could switch cancelCtx to use an atomic.Value.

/cc @crawshaw

And fortunately #39351 (sync/atomic: add (*Value).Swap and (*Value).CompareAndSwap) was finally accepted after 5 years of requests (and has open CL https://go-review.googlesource.com/c/go/+/241678/), so this could use that.

I'll mark this as Go 1.17, with the assumption that that'll work out.

(It'd probably need to CompareAndSwap with an old value of an package-unexported sentinel channel that means "uninitialized", as CompareAndSwap per that proposal/CL doesn't accept a nil value for old)

Change https://golang.org/cl/288193 mentions this issue: context: reduce contention in cancelCtx.Done

This doesn't need compare and swap; we already have a mutex available for other reasons to guard writes. I took a first stab in CL 288193, but apparently I screwed something up. Will aim to fix next week...