The Go Blogtag:blog.golang.org,2013:blog.golang.org2024-03-14T00:00:00+00:00More powerful Go execution tracestag:blog.golang.org,2013:blog.golang.org/execution-traces-20242024-03-14T00:00:00+00:002024-03-14T00:00:00+00:00New features and improvements to execution traces from the last year. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/execution-traces-2024"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>More powerful Go execution traces</h1> <p class="author"> Michael Knyszek<br> 14 March 2024 </p> <p>The <a href="/pkg/runtime/trace">runtime/trace</a> package contains a powerful tool for understanding and troubleshooting Go programs. The functionality within allows one to produce a trace of each goroutine&rsquo;s execution over some time period. With the <a href="/pkg/cmd/trace"><code>go tool trace</code> command</a> (or the excellent open source <a href="https://gotraceui.dev/" rel="noreferrer" target="_blank">gotraceui tool</a>), one may then visualize and explore the data within these traces.</p> <p>The magic of a trace is that it can easily reveal things about a program that are hard to see in other ways. For example, a concurrency bottleneck where lots of goroutines block on the same channel might be quite difficult to see in a CPU profile, because there&rsquo;s no execution to sample. But in an execution trace, the <em>lack</em> of execution will show up with amazing clarity, and the stack traces of blocked goroutines will quickly point at the culprit.</p> <div class="image"> <img src="execution-traces-2024/gotooltrace.png" alt=""> </div> <p>Go developers are even able to instrument their own programs with <a href="/pkg/runtime/trace#Task">tasks</a>, <a href="/pkg/runtime/trace#WithRegion">regions</a>, and <a href="/pkg/runtime/trace#Log">logs</a> that they can use to correlate their higher-level concerns with lower-level execution details.</p> <h2 id="issues">Issues</h2> <p>Unfortunately, the wealth of information in execution traces can often be out of reach. Four big issues with traces have historically gotten in the way.</p> <ul> <li>Traces had high overheads.</li> <li>Traces didn&rsquo;t scale well, and could become too big to analyze.</li> <li>It was often unclear when to start tracing to capture a specific bad behavior.</li> <li>Only the most adventurous gophers could programmatically analyze traces, given the lack of a public package for parsing and interpreting execution traces.</li> </ul> <p>If you&rsquo;ve used traces in the last few years, you&rsquo;ve likely been frustrated by one or more of these problems. But we&rsquo;re excited to share that over the last two Go releases we&rsquo;ve made big progress in all four of these areas.</p> <h2 id="low-overhead-tracing">Low-overhead tracing</h2> <p>Prior to Go 1.21, the run-time overhead of tracing was somewhere between 10–20% CPU for many applications, which limits tracing to situational usage, rather than continuous usage like CPU profiling. It turned out that much of the cost of tracing came down to tracebacks. Many events produced by the runtime have stack traces attached, which are invaluable to actually identifying what goroutines were doing at key moments in their execution.</p> <p>Thanks to work by Felix Geisendörfer and Nick Ripley on optimizing the efficiency of tracebacks, the run-time CPU overhead of execution traces has been cut dramatically, down to 1–2% for many applications. You can read more about the work done here in <a href="https://blog.felixge.de/reducing-gos-execution-tracer-overhead-with-frame-pointer-unwinding/" rel="noreferrer" target="_blank">Felix&rsquo;s great blog post</a> on the topic.</p> <h2 id="scalable-traces">Scalable traces</h2> <p>The trace format and its events were designed around relatively efficient emission, but required tooling to parse and keep around the state of the entirety of a trace. A few hundred MiB trace could require several GiB of RAM to analyze!</p> <p>This issue is unfortunately fundamental to how traces are generated. To keep run-time overheads low, all events are written to the equivalent of thread-local buffers. But this means events appear out of their true order, and the burden is placed on the trace tooling to figure out what really happened.</p> <p>The key insight to making traces scale while keeping overheads low was to occasionally split the trace being generated. Each split point would behave a bit like simultaneously disabling and reenabling tracing in one go. All the trace data so far would represent a complete and self-contained trace, while the new trace data would seamlessly pick up from where it left off.</p> <p>As you might imagine, fixing this required <a href="/issue/60773">rethinking and rewriting a lot of the foundation of the trace implementation</a> in the runtime. We&rsquo;re happy to say that the work landed in Go 1.22 and is now generally available. <a href="/doc/go1.22#runtime/trace">A lot of nice improvements</a> came with the rewrite, including some improvements to the <a href="/doc/go1.22#trace"><code>go tool trace</code> command</a> as well. The gritty details are all in the <a href="https://github.com/golang/proposal/blob/master/design/60773-execution-tracer-overhaul.md" rel="noreferrer" target="_blank">design document</a>, if you&rsquo;re curious.</p> <p>(Note: <code>go tool trace</code> still loads the full trace into memory, but <a href="/issue/65315">removing this limitation</a> for traces produced by Go 1.22+ programs is now feasible.)</p> <h2 id="flight-recording">Flight recording</h2> <p>Suppose you work on a web service and an RPC took a very long time. You couldn&rsquo;t start tracing at the point you knew the RPC was already taking a while, because the root cause of the slow request already happened and wasn&rsquo;t recorded.</p> <p>There&rsquo;s a technique that can help with this called flight recording, which you may already be familiar with from other programming environments. The insight with flight recording is to have tracing on continuously and always keep the most recent trace data around, just in case. Then, once something interesting happens, the program can just write out whatever it has!</p> <p>Before traces could be split, this was pretty much a non-starter. But because continuous tracing is now viable thanks to low overheads, and the fact that the runtime can now split traces any time it needs, it turns out it was straightforward to implement flight recording.</p> <p>As a result, we&rsquo;re happy to announce a flight recorder experiment, available in the <a href="/pkg/golang.org/x/exp/trace#FlightRecorder">golang.org/x/exp/trace package</a>.</p> <p>Please try it out! Below is an example that sets up flight recording to capture a long HTTP request to get you started.</p> <div class="code"> <pre> <span class="comment">// Set up the flight recorder.</span> fr := trace.NewFlightRecorder() fr.Start() <span class="comment">// Set up and run an HTTP server.</span> var once sync.Once http.HandleFunc(&#34;/my-endpoint&#34;, func(w http.ResponseWriter, r *http.Request) { start := time.Now() <span class="comment">// Do the work...</span> doWork(w, r) <span class="comment">// We saw a long request. Take a snapshot!</span> if time.Since(start) &gt; 300*time.Millisecond { <span class="comment">// Do it only once for simplicity, but you can take more than one.</span> once.Do(func() { <span class="comment">// Grab the snapshot.</span> var b bytes.Buffer _, err = fr.WriteTo(&amp;b) if err != nil { log.Print(err) return } <span class="comment">// Write it to a file.</span> if err := os.WriteFile(&#34;trace.out&#34;, b.Bytes(), 0o755); err != nil { log.Print(err) return } }) } }) log.Fatal(http.ListenAndServe(&#34;:8080&#34;, nil)) </pre> </div> <p>If you have any feedback, positive or negative, please share it to the <a href="/issue/63185">proposal issue</a>!</p> <h2 id="trace-reader-api">Trace reader API</h2> <p>Along with the trace implementation rewrite came an effort to clean up the other trace internals, like <code>go tool trace</code>. This spawned an attempt to create a trace reader API that was good enough to share and that could make traces more accessible.</p> <p>Just like the flight recorder, we&rsquo;re happy to announce that we also have an experimental trace reader API that we&rsquo;d like to share. It&rsquo;s available in the <a href="/pkg/golang.org/x/exp/trace#Reader">same package as the flight recorder, golang.org/x/exp/trace</a>.</p> <p>We think it&rsquo;s good enough to start building things on top of, so please try it out! Below is an example that measures the proportion of goroutine block events that blocked to wait on the network.</p> <div class="code"> <pre> <span class="comment">// Start reading from STDIN.</span> r, err := trace.NewReader(os.Stdin) if err != nil { log.Fatal(err) } var blocked int var blockedOnNetwork int for { <span class="comment">// Read the event.</span> ev, err := r.ReadEvent() if err == io.EOF { break } else if err != nil { log.Fatal(err) } <span class="comment">// Process it.</span> if ev.Kind() == trace.EventStateTransition { st := ev.StateTransition() if st.Resource.Kind == trace.ResourceGoroutine { id := st.Resource.Goroutine() from, to := st.GoroutineTransition() <span class="comment">// Look for goroutines blocking, and count them.</span> if from.Executing() &amp;&amp; to == trace.GoWaiting { blocked++ if strings.Contains(st.Reason, &#34;network&#34;) { blockedOnNetwork++ } } } } } <span class="comment">// Print what we found.</span> p := 100 * float64(blockedOnNetwork) / float64(blocked) fmt.Printf(&#34;%2.3f%% instances of goroutines blocking were to block on the network\n&#34;, p) </pre> </div> <p>And just like the flight recorder, there&rsquo;s a <a href="/issue/62627">proposal issue</a> that would be a great place to leave feedback!</p> <p>We&rsquo;d like to quickly call out Dominik Honnef as someone who tried it out early, provided great feedback, and has contributed support for older trace versions to the API.</p> <h2 id="thank-you">Thank you!</h2> <p>This work was completed, in no small part, thanks to the help of the those in the <a href="/issue/57175">diagnostics working group</a>, started over a year ago as a collaboration between stakeholders from across the Go community, and open to the public.</p> <p>We&rsquo;d like to take a moment to thank those community members who have attended the diagnostic meetings regularly over the last year: Felix Geisendörfer, Nick Ripley, Rhys Hiltner, Dominik Honnef, Bryan Boreham, thepudds.</p> <p>The discussions, feedback, and work you all put in have been instrumental to getting us to where we are today. Thank you!</p> </div> <div class="Article prevnext"> <p> <b>Previous article: </b><a href="/blog/generic-slice-functions">Robust generic functions on slices</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Robust generic functions on slicestag:blog.golang.org,2013:blog.golang.org/generic-slice-functions2024-02-22T00:00:00+00:002024-02-22T00:00:00+00:00Avoiding memory leaks in the slices package. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/generic-slice-functions"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Robust generic functions on slices</h1> <p class="author"> Valentin Deleplace<br> 22 February 2024 </p> <p>The <a href="/pkg/slices">slices</a> package provides functions that work for slices of any type. In this blog post we&rsquo;ll discuss how you can use these functions more effectively by understanding how slices are represented in memory and how that affects the garbage collector, and we&rsquo;ll cover how we recently adjusted these functions to make them less surprising.</p> <p>With <a href="/blog/deconstructing-type-parameters">Type parameters</a> we can write functions like <a href="/pkg/slices#Index">slices.Index</a> once for all types of slices of comparable elements:</p> <pre><code>// Index returns the index of the first occurrence of v in s, // or -1 if not present. func Index[S ~[]E, E comparable](s S, v E) int { for i := range s { if v == s[i] { return i } } return -1 } </code></pre> <p>It is no longer necessary to implement <code>Index</code> again for each different type of element.</p> <p>The <a href="/pkg/slices">slices</a> package contains many such helpers to perform common operations on slices:</p> <pre><code> s := []string{&quot;Bat&quot;, &quot;Fox&quot;, &quot;Owl&quot;, &quot;Fox&quot;} s2 := slices.Clone(s) slices.Sort(s2) fmt.Println(s2) // [Bat Fox Fox Owl] s2 = slices.Compact(s2) fmt.Println(s2) // [Bat Fox Owl] fmt.Println(slices.Equal(s, s2)) // false </code></pre> <p>Several new functions (<code>Insert</code>, <code>Replace</code>, <code>Delete</code>, etc.) modify the slice. To understand how they work, and how to properly use them, we need to examine the underlying structure of slices.</p> <p>A slice is a view of a portion of an array. <a href="/blog/slices-intro">Internally</a>, a slice contains a pointer, a length, and a capacity. Two slices can have the same underlying array, and can view overlapping portions.</p> <p>For example, this slice <code>s</code> is a view on 4 elements of an array of size 6:</p> <div class="image"> <img src="generic-slice-functions/1_sample_slice_4_6.svg" width="450" alt=""> </div> <p>If a function changes the length of a slice passed as a parameter, then it needs to return a new slice to the caller. The underlying array may remain the same if it doesn&rsquo;t have to grow. This explains why <a href="/blog/slices">append</a> and <code>slices.Compact</code> return a value, but <code>slices.Sort</code>, which merely reorders the elements, does not.</p> <p>Consider the task of deleting a portion of a slice. Prior to generics, the standard way to delete the portion <code>s[2:5]</code> from the slice <code>s</code> was to call the <a href="/ref/spec#Appending_and_copying_slices">append</a> function to copy the end portion over the middle portion:</p> <pre><code>s = append(s[:2], s[5:]...) </code></pre> <p>The syntax was complex and error-prone, involving subslices and a variadic parameter. We added <a href="/pkg/slices#Delete">slice.Delete</a> to make it easier to delete elements:</p> <pre><code>func Delete[S ~[]E, E any](s S, i, j int) S { return append(s[:i], s[j:]...) } </code></pre> <p>The one-line function <code>Delete</code> more clearly expresses the programmer&rsquo;s intent. Let’s consider a slice <code>s</code> of length 6 and capacity 8, containing pointers:</p> <div class="image"> <img src="generic-slice-functions/2_sample_slice_6_8.svg" width="600" alt=""> </div> <p>This call deletes the elements at <code>s[2]</code>, <code>s[3]</code>, <code>s[4]</code> from the slice <code>s</code>:</p> <pre><code>s = slices.Delete(s, 2, 5) </code></pre> <div class="image"> <img src="generic-slice-functions/3_delete_s_2_5.svg" width="600" alt=""> </div> <p>The gap at the indices 2, 3, 4 is filled by shifting the element <code>s[5]</code> to the left, and setting the new length to <code>3</code>.</p> <p><code>Delete</code> need not allocate a new array, as it shifts the elements in place. Like <code>append</code>, it returns a new slice. Many other functions in the <code>slices</code> package follow this pattern, including <code>Compact</code>, <code>CompactFunc</code>, <code>DeleteFunc</code>, <code>Grow</code>, <code>Insert</code>, and <code>Replace</code>.</p> <p>When calling these functions we must consider the original slice invalid, because the underlying array has been modified. It would be a mistake to call the function but ignore the return value:</p> <pre><code> slices.Delete(s, 2, 5) // incorrect! // s still has the same length, but modified contents </code></pre> <h2 id="a-problem-of-unwanted-liveness">A problem of unwanted liveness</h2> <p>Before Go 1.22, <code>slices.Delete</code> didn&rsquo;t modify the elements between the new and original lengths of the slice. While the returned slice wouldn&rsquo;t include these elements, the &ldquo;gap&rdquo; created at the end of the original, now-invalidated slice continued to hold onto them. These elements could contain pointers to large objects (a 20MB image), and the garbage collector would not release the memory associated with these objects. This resulted in a memory leak that could lead to significant performance issues.</p> <p>In this above example, we’re successfully deleting the pointers <code>p2</code>, <code>p3</code>, <code>p4</code> from <code>s[2:5]</code>, by shifting one element to the left. But <code>p3</code> and <code>p4</code> are still present in the underlying array, beyond the new length of <code>s</code>. The garbage collector won’t reclaim them. Less obviously, <code>p5</code> is not one of the deleted elements, but its memory may still leak because of the <code>p5</code> pointer kept in the gray part of the array.</p> <p>This could be confusing for developers, if they were not aware that &ldquo;invisible&rdquo; elements were still using memory.</p> <p>So we had two options:</p> <ul> <li>Either keep the efficient implementation of <code>Delete</code>. Let users set obsolete pointers to <code>nil</code> themselves, if they want to make sure the values pointed to can be freed.</li> <li>Or change <code>Delete</code> to always set the obsolete elements to zero. This is extra work, making <code>Delete</code> slightly less efficient. Zeroing pointers (setting them to <code>nil</code>) enables the garbage collection of the objects, when they become otherwise unreachable.</li> </ul> <p>It was not obvious which option was best. The first one provided performance by default, and the second one provided memory frugality by default.</p> <h2 id="the-fix">The fix</h2> <p>A key observation is that &ldquo;setting the obsolete pointers to <code>nil</code>&rdquo; is not as easy as it seems. In fact, this task is so error-prone that we should not put the burden on the user to write it. Out of pragmatism, we chose to modify the implementation of the five functions <code>Compact</code>, <code>CompactFunc</code>, <code>Delete</code>, <code>DeleteFunc</code>, <code>Replace</code> to &ldquo;clear the tail&rdquo;. As a nice side effect, the cognitive load is reduced and users now don’t need to worry about these memory leaks.</p> <p>In Go 1.22, this is what the memory looks like after calling Delete:</p> <div class="image"> <img src="generic-slice-functions/4_delete_s_2_5_nil.svg" width="600" alt=""> </div> <p>The code changed in the five functions uses the new built-in function <a href="/pkg/builtin#clear">clear</a> (Go 1.21) to set the obsolete elements to the zero value of the element type of <code>s</code>:</p> <div class="image"> <img src="generic-slice-functions/5_Delete_diff.png" width="800" alt=""> </div> <p>The zero value of <code>E</code> is <code>nil</code> when <code>E</code> is a type of pointer, slice, map, chan, or interface.</p> <h2 id="tests-failing">Tests failing</h2> <p>This change has led to some tests that passed in Go 1.21 now failing in Go 1.22, when the slices functions are used incorrectly. This is good news. When you have a bug, tests should let you know.</p> <p>If you ignore the return value of <code>Delete</code>:</p> <pre><code>slices.Delete(s, 2, 3) // !! INCORRECT !! </code></pre> <p>then you may incorrectly assume that <code>s</code> does not contain any nil pointer. <a href="/play/p/NDHuO8vINHv">Example in the Go Playground</a>.</p> <p>If you ignore the return value of <code>Compact</code>:</p> <pre><code>slices.Sort(s) // correct slices.Compact(s) // !! INCORRECT !! </code></pre> <p>then you may incorrectly assume that <code>s</code> is properly sorted and compacted. <a href="/play/p/eFQIekiwlnu">Example</a>.</p> <p>If you assign the return value of <code>Delete</code> to another variable, and keep using the original slice:</p> <pre><code>u := slices.Delete(s, 2, 3) // !! INCORRECT, if you keep using s !! </code></pre> <p>then you may incorrectly assume that <code>s</code> does not contain any nil pointer. <a href="/play/p/rDxWmJpLOVO">Example</a>.</p> <p>If you accidentally shadow the slice variable, and keep using the original slice:</p> <pre><code>s := slices.Delete(s, 2, 3) // !! INCORRECT, using := instead of = !! </code></pre> <p>then you may incorrectly assume that <code>s</code> does not contain any nil pointer. <a href="/play/p/KSpVpkX8sOi">Example</a>.</p> <h2 id="conclusion">Conclusion</h2> <p>The API of the <code>slices</code> package is a net improvement over the traditional pre-generics syntax to delete or insert elements.</p> <p>We encourage developers to use the new functions, while avoiding the &ldquo;gotchas&rdquo; listed above.</p> <p>Thanks to the recent changes in the implementation, a class of memory leaks is automatically avoided, without any change to the API, and with no extra work for the developers.</p> <h2 id="further-reading">Further reading</h2> <p>The signature of the functions in the <code>slices</code> package is heavily influenced by the specifics of the representation of slices in memory. We recommend reading</p> <ul> <li><a href="/blog/slices-intro">Go Slices: usage and internals</a></li> <li><a href="/blog/slices">Arrays, slices: The mechanics of &lsquo;append&rsquo;</a></li> <li>The <a href="https://en.wikipedia.org/wiki/Dynamic_array" rel="noreferrer" target="_blank">dynamic array</a> data structure</li> <li>The <a href="/pkg/slices">documentation</a> of the package slices</li> </ul> <p>The <a href="/issue/63393">original proposal</a> about zeroing obsolete elements contains many details and comments.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/execution-traces-2024">More powerful Go execution traces</a><br> <b>Previous article: </b><a href="/blog/routing-enhancements">Routing Enhancements for Go 1.22</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Routing Enhancements for Go 1.22tag:blog.golang.org,2013:blog.golang.org/routing-enhancements2024-02-13T00:00:00+00:002024-02-13T00:00:00+00:00Go 1.22&#39;s additions to patterns for HTTP routes. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/routing-enhancements"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Routing Enhancements for Go 1.22</h1> <p class="author"> Jonathan Amsterdam, on behalf of the Go team<br> 13 February 2024 </p> <p>Go 1.22 brings two enhancements to the <code>net/http</code> package&rsquo;s router: method matching and wildcards. These features let you express common routes as patterns instead of Go code. Although they are simple to explain and use, it was a challenge to come up with the right rules for selecting the winning pattern when several match a request.</p> <p>We made these changes as part of our continuing effort to make Go a great language for building production systems. We studied many third-party web frameworks, extracted what we felt were the most used features, and integrated them into <code>net/http</code>. Then we validated our choices and improved our design by collaborating with the community in a <a href="https://github.com/golang/go/discussions/60227" rel="noreferrer" target="_blank">GitHub discussion</a> and a <a href="/issue/61410">proposal issue</a>. Adding these features to the standard library means one fewer dependency for many projects. But third-party web frameworks remain a fine choice for current users or programs with advanced routing needs.</p> <h2 id="enhancements">Enhancements</h2> <p>The new routing features almost exclusively affect the pattern string passed to the two <code>net/http.ServeMux</code> methods <code>Handle</code> and <code>HandleFunc</code>, and the corresponding top-level functions <code>http.Handle</code> and <code>http.HandleFunc</code>. The only API changes are two new methods on <code>net/http.Request</code> for working with wildcard matches.</p> <p>We&rsquo;ll illustrate the changes with a hypothetical blog server in which every post has an integer identifier. A request like <code>GET /posts/234</code> retrieves the post with ID 234. Before Go 1.22, the code for handling those requests would start with a line like this:</p> <pre><code>http.HandleFunc(&quot;/posts/&quot;, handlePost) </code></pre> <p>The trailing slash routes all requests beginning <code>/posts/</code> to the <code>handlePost</code> function, which would have to check that the HTTP method was GET, extract the identifier, and retrieve the post. Since the method check isn&rsquo;t strictly necessary to satisfy the request, it would be a natural mistake to omit it. That would mean that a request like <code>DELETE /posts/234</code> would fetch the post, which is surprising at the least.</p> <p>In Go 1.22, the existing code will continue to work, or you could instead write this:</p> <pre><code>http.HandleFunc(&quot;GET /posts/{id}&quot;, handlePost2) </code></pre> <p>This pattern matches a GET request whose path begins &ldquo;/posts/&rdquo; and has two segments. (As a special case, GET also matches HEAD; all the other methods match exactly.) The <code>handlePost2</code> function no longer needs to check the method, and extracting the identifier string can be written using the new <code>PathValue</code> method on <code>Request</code>:</p> <pre><code>idString := req.PathValue(&quot;id&quot;) </code></pre> <p>The rest of <code>handlePost2</code> would behave like <code>handlePost</code>, converting the string identifier to an integer and fetching the post.</p> <p>Requests like <code>DELETE /posts/234</code> will fail if no other matching pattern is registered. In accordance with <a href="https://httpwg.org/specs/rfc9110.html#status.405" rel="noreferrer" target="_blank">HTTP semantics</a>, a <code>net/http</code> server will reply to such a request with a <code>405 Method Not Allowed</code> error that lists the available methods in an <code>Allow</code> header.</p> <p>A wildcard can match an entire segment, like <code>{id}</code> in the example above, or if it ends in <code>...</code> it can match all the remaining segments of the path, as in the pattern <code>/files/{pathname...}</code>.</p> <p>There is one last bit of syntax. As we showed above, patterns ending in a slash, like <code>/posts/</code>, match all paths beginning with that string. To match only the path with the trailing slash, you can write <code>/posts/{$}</code>. That will match <code>/posts/</code> but not <code>/posts</code> or <code>/posts/234</code>.</p> <p>And there is one last bit of API: <code>net/http.Request</code> has a <code>SetPathValue</code> method so that routers outside the standard library can make the results of their own path parsing available via <code>Request.PathValue</code>.</p> <h2 id="precedence">Precedence</h2> <p>Every HTTP router must deal with overlapping patterns, like <code>/posts/{id}</code> and <code>/posts/latest</code>. Both of these patterns match the path &ldquo;posts/latest&rdquo;, but at most one can serve the request. Which pattern takes precedence?</p> <p>Some routers disallow overlaps; others use the pattern that was registered last. Go has always allowed overlaps, and has chosen the longer pattern regardless of registration order. Preserving order-independence was important to us (and necessary for backwards compatibility), but we needed a better rule than &ldquo;longest wins.&rdquo; That rule would select <code>/posts/latest</code> over <code>/posts/{id}</code>, but would choose <code>/posts/{identifier}</code> over both. That seems wrong: the wildcard name shouldn&rsquo;t matter. It feels like <code>/posts/latest</code> should always win this competition, because it matches a single path instead of many.</p> <p>Our quest for a good precedence rule led us to consider many properties of patterns. For example, we considered preferring the pattern with the longest literal (non-wildcard) prefix. That would choose <code>/posts/latest</code> over <code>/posts/ {id}</code>. But it wouldn&rsquo;t distinguish between <code>/users/{u}/posts/latest</code> and <code>/users/{u}/posts/{id}</code>, and it seems like the former should take precedence.</p> <p>We eventually chose a rule based on what the patterns mean instead of how they look. Every valid pattern matches a set of requests. For example, <code>/posts/latest</code> matches requests with the path <code>/posts/latest</code>, while <code>/posts/{id}</code> matches requests with any two-segment path whose first segment is &ldquo;posts&rdquo;. We say that one pattern is <em>more specific</em> than another if it matches a strict subset of requests. The pattern <code>/posts/latest</code> is more specific than <code>/posts/{id}</code> because the latter matches every request that the former does, and more.</p> <p>The precedence rule is simple: the most specific pattern wins. This rule matches our intuition that <code>posts/latests</code> should be preferred to <code>posts/{id}</code>, and <code>/users/{u}/posts/latest</code> should be preferred to <code>/users/{u}/posts/{id}</code>. It also makes sense for methods. For example, <code>GET /posts/{id}</code> takes precedence over <code>/posts/{id}</code> because the first only matches GET and HEAD requests, while the second matches requests with any method.</p> <p>The &ldquo;most specific wins&rdquo; rule generalizes the original &ldquo;longest wins&rdquo; rule for the path parts of original patterns, those without wildcards or <code>{$}</code>. Such patterns only overlap when one is a prefix of the other, and the longer is the more specific.</p> <p>What if two patterns overlap but neither is more specific? For example, <code>/posts/{id}</code> and <code>/{resource}/latest</code> both match <code>/posts/latest</code>. There is no obvious answer to which takes precedence, so we consider these patterns to conflict with each other. Registering both of them (in either order!) will panic.</p> <p>The precedence rule works exactly as above for methods and paths, but we had to make one exception for hosts to preserve compatibility: if two patterns would otherwise conflict and one has a host while the other does not, then the pattern with the host takes precedence.</p> <p>Students of computer science may recall the beautiful theory of regular expressions and regular languages. Each regular expression picks out a regular language, the set of strings matched by the expression. Some questions are easier to pose and answer by talking about languages rather than expressions. Our precedence rule was inspired by this theory. Indeed, each routing pattern corresponds to a regular expression, and sets of matching requests play the role of regular languages.</p> <p>Defining precedence by languages instead of expressions makes it easy to state and understand. But there is a downside to having a rule based on potentially infinite sets: it isn&rsquo;t clear how to implement it efficiently. It turns out we can determine whether two patterns conflict by walking them segment by segment. Roughly speaking, if one pattern has a literal segment wherever the other has a wildcard, it is more specific; but if literals align with wildcards in both directions, the patterns conflict.</p> <p>As new patterns are registered on a <code>ServeMux</code>, it checks for conflicts with previously registered patterns. But checking every pair of patterns would take quadratic time. We use an index to skip patterns that cannot conflict with a new pattern; in practice, it works quite well. In any case, this check happens when patterns are registered, usually at server startup. The time to match incoming requests in Go 1.22 hasn&rsquo;t changed much from previous versions.</p> <h2 id="compatibility">Compatibility</h2> <p>We made every effort to keep the new functionality compatible with older versions of Go. The new pattern syntax is a superset of the old, and the new precedence rule generalizes the old one. But there are a few edge cases. For example, previous versions of Go accepted patterns with braces and treated them literally, but Go 1.22 uses braces for wildcards. The GODEBUG setting <code>httpmuxgo121</code> restores the old behavior.</p> <p>For more details about these routing enhancements, see the <a href="/pkg/net/http#ServeMux"><code>net/http.ServeMux</code> documentation</a>.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/generic-slice-functions">Robust generic functions on slices</a><br> <b>Previous article: </b><a href="/blog/go1.22">Go 1.22 is released!</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Go 1.22 is released!tag:blog.golang.org,2013:blog.golang.org/go1.222024-02-06T00:00:00+00:002024-02-06T00:00:00+00:00Go 1.22 enhances for loops, brings new standard library functionality and improves performance. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/go1.22"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Go 1.22 is released!</h1> <p class="author"> Eli Bendersky, on behalf of the Go team<br> 6 February 2024 </p> <p>Today the Go team is thrilled to release Go 1.22, which you can get by visiting the <a href="/dl/">download page</a>.</p> <p>Go 1.22 comes with several important new features and improvements. Here are some of the notable changes; for the full list, refer to the <a href="/doc/go1.22">release notes</a>.</p> <h2 id="language-changes">Language changes</h2> <p>The long-standing &ldquo;for&rdquo; loop gotcha with accidental sharing of loop variables between iterations is now resolved. Starting with Go 1.22, the following code will print &ldquo;a&rdquo;, &ldquo;b&rdquo;, and &ldquo;c&rdquo; in some order:</p> <pre><code>func main() { done := make(chan bool) values := []string{&quot;a&quot;, &quot;b&quot;, &quot;c&quot;} for _, v := range values { go func() { fmt.Println(v) done &lt;- true }() } // wait for all goroutines to complete before exiting for _ = range values { &lt;-done } } </code></pre> <p>For more information about this change and the tooling that helps keep code from breaking accidentally, see the earlier <a href="/blog/loopvar-preview">loop variable blog post</a>.</p> <p>The second language change is support for ranging over integers:</p> <pre><code>package main import &quot;fmt&quot; func main() { for i := range 10 { fmt.Println(10 - i) } fmt.Println(&quot;go1.22 has lift-off!&quot;) } </code></pre> <p>The values of <code>i</code> in this countdown program go from 0 to 9, inclusive. For more details, please refer to <a href="/ref/spec#For_range">the spec</a>.</p> <h2 id="improved-performance">Improved performance</h2> <p>Memory optimization in the Go runtime improves CPU performance by 1-3%, while also reducing the memory overhead of most Go programs by around 1%.</p> <p>In Go 1.21, <a href="/blog/pgo">we shipped</a> profile-guided optimization (PGO) for the Go compiler and this functionality continues to improve. One of the optimizations added in 1.22 is improved devirtualization, allowing static dispatch of more interface method calls. Most programs will see improvements between 2-14% with PGO enabled.</p> <h2 id="standard-library-additions">Standard library additions</h2> <ul> <li> <p>A new <a href="/pkg/math/rand/v2">math/rand/v2</a> package provides a cleaner, more consistent API and uses higher-quality, faster pseudo-random generation algorithms. See <a href="/issue/61716">the proposal</a> for additional details.</p> </li> <li> <p>The patterns used by <a href="/pkg/net/http#ServeMux">net/http.ServeMux</a> now accept methods and wildcards.</p> <p>For example, the router accepts a pattern like <code>GET /task/{id}/</code>, which matches only <code>GET</code> requests and captures the value of the <code>{id}</code> segment in a map that can be accessed through <a href="/pkg/net/http#Request">Request</a> values.</p> </li> <li> <p>A new <code>Null[T]</code> type in <a href="/pkg/database/sql">database/sql</a> provides a way to scan nullable columns.</p> </li> <li> <p>A <code>Concat</code> function was added in package <a href="/pkg/slices">slices</a>, to concatenate multiple slices of any type.</p> </li> </ul> <hr> <p>Thanks to everyone who contributed to this release by writing code and documentation, filing bugs, sharing feedback, and testing the release candidates. Your efforts helped to ensure that Go 1.22 is as stable as possible. As always, if you notice any problems, please <a href="/issue/new">file an issue</a>.</p> <p>Enjoy Go 1.22!</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/routing-enhancements">Routing Enhancements for Go 1.22</a><br> <b>Previous article: </b><a href="/blog/survey2024-h1">Share your feedback about developing with Go</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Share your feedback about developing with Gotag:blog.golang.org,2013:blog.golang.org/survey2024-h12024-01-23T00:00:00+00:002024-01-23T00:00:00+00:00Help shape the future of Go by sharing your thoughts via the Go Developer Survey <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/survey2024-h1"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Share your feedback about developing with Go</h1> <p class="author"> Alice Merrick, for the Go team<br> 23 January 2024 </p> <p>Happy New Year! <a href="https://google.qualtrics.com/jfe/form/SV_083SVAUCji98YeO?s=b" rel="noreferrer" target="_blank">The Go Developer Survey is now open</a>, and we want to hear from you!</p> <p>Since 2016, the insights from our Go Developer Surveys have helped us identify key usage patterns, understand developer challenges, discover tooling preferences, and track emerging trends within the community. Your feedback is essential in shaping the future of Go, so please take 10–15 minutes to complete the survey by February 11, 2024. <a href="https://google.qualtrics.com/jfe/form/SV_083SVAUCji98YeO?s=b" rel="noreferrer" target="_blank">Take the survey now!</a></p> <p>The more developers who participate, the better we&rsquo;ll be able to understand the needs of the Go community. You can help spread the word by sharing this survey on your social media channels, with your co-workers, and in any other relevant communities.</p> <p>Thank you for your time and feedback!</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/go1.22">Go 1.22 is released!</a><br> <b>Previous article: </b><a href="/blog/deadcode">Finding unreachable functions with deadcode</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Finding unreachable functions with deadcodetag:blog.golang.org,2013:blog.golang.org/deadcode2023-12-12T00:00:00+00:002023-12-12T00:00:00+00:00deadcode is a new command to help identify functions that cannot be called. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/deadcode"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Finding unreachable functions with deadcode</h1> <p class="author"> Alan Donovan<br> 12 December 2023 </p> <p>Functions that are part of your project&rsquo;s source code but can never be reached in any execution are called &ldquo;dead code&rdquo;, and they exert a drag on codebase maintenance efforts. Today we&rsquo;re pleased to share a tool named <code>deadcode</code> to help you identify them.</p> <pre><code>$ go install golang.org/x/tools/cmd/deadcode@latest $ deadcode -help The deadcode command reports unreachable functions in Go programs. Usage: deadcode [flags] package... </code></pre> <h2 id="example">Example</h2> <p>Over the last year or so, we&rsquo;ve been making a lot of changes to the structure of <a href="/blog/gopls-scalability">gopls</a>, the language server for Go that powers VS Code and other editors. A typical change might rewrite some existing function, taking care to ensure that its new behavior satisfies the needs of all existing callers. Sometimes, after putting in all that effort, we would discover to our frustration that one of the callers was never actually reached in any execution, so it could safely have been been deleted. If we had known this beforehand our refactoring task would have been easier.</p> <p>The simple Go program below illustrates the problem:</p> <pre><code>module example.com/greet go 1.21 </code></pre> <pre><code>package main import &quot;fmt&quot; func main() { var g Greeter g = Helloer{} g.Greet() } type Greeter interface{ Greet() } type Helloer struct{} type Goodbyer struct{} var _ Greeter = Helloer{} // Helloer implements Greeter var _ Greeter = Goodbyer{} // Goodbyer implements Greeter func (Helloer) Greet() { hello() } func (Goodbyer) Greet() { goodbye() } func hello() { fmt.Println(&quot;hello&quot;) } func goodbye() { fmt.Println(&quot;goodbye&quot;) } </code></pre> <p>When we execute it, it says hello:</p> <pre><code>$ go run . hello </code></pre> <p>It&rsquo;s clear from its output that this program executes the <code>hello</code> function but not the <code>goodbye</code> function. What&rsquo;s less clear at a glance is that the <code>goodbye</code> function can never be called. However, we can&rsquo;t simply delete <code>goodbye</code>, because it&rsquo;s required by the <code>Goodbyer.Greet</code> method, which in turn is required to implement the <code>Greeter</code> interface whose <code>Greet</code> method we can see is called from <code>main</code>. But if we work forwards from main, we can see that no <code>Goodbyer</code> values are ever created, so the <code>Greet</code> call in <code>main</code> can only reach <code>Helloer.Greet</code>. That&rsquo;s the idea behind the algorithm used by the <code>deadcode</code> tool.</p> <p>When we run deadcode on this program, the tool tells us that the <code>goodbye</code> function and the <code>Goodbyer.Greet</code> method are both unreachable:</p> <pre><code>$ deadcode . greet.go:23: unreachable func: goodbye greet.go:20: unreachable func: Goodbyer.Greet </code></pre> <p>With this knowledge, we can safely remove both functions, along with the <code>Goodbyer</code> type itself.</p> <p>The tool can also explain why the <code>hello</code> function is live. It responds with a chain of function calls that reaches <code>hello</code>, starting from main:</p> <pre><code>$ deadcode -whylive=example.com/greet.hello . example.com/greet.main dynamic@L0008 --&gt; example.com/greet.Helloer.Greet static@L0019 --&gt; example.com/greet.hello </code></pre> <p>The output is designed to be easy to read on a terminal, but you can use the <code>-json</code> or <code>-f=template</code> flags to specify richer output formats for consumption by other tools.</p> <h2 id="how-it-works">How it works</h2> <p>The <code>deadcode</code> command <a href="https://pkg.go.dev/golang.org/x/tools/go/packages" rel="noreferrer" target="_blank">loads</a>, <a href="https://pkg.go.dev/go/parser" rel="noreferrer" target="_blank">parses</a>, and <a href="https://pkg.go.dev/go/types" rel="noreferrer" target="_blank">type-checks</a> the specified packages, then converts them into an <a href="https://pkg.go.dev/golang.org/x/tools/go/ssa" rel="noreferrer" target="_blank">intermediate representation</a> similar to a typical compiler.</p> <p>It then uses an algorithm called <a href="https://pkg.go.dev/golang.org/x/tools/go/callgraph/rta" rel="noreferrer" target="_blank">Rapid Type Analysis</a> (RTA) to build up the set of functions that are reachable, which is initially just the entry points of each <code>main</code> package: the <code>main</code> function, and the package initializer function, which assigns global variables and calls functions named <code>init</code>.</p> <p>RTA looks at the statements in the body of each reachable function to gather three kinds of information: the set of functions it calls directly; the set of dynamic calls it makes through interface methods; and the set of types it converts to an interface.</p> <p>Direct function calls are easy: we just add the callee to the set of reachable functions, and if it&rsquo;s the first time we&rsquo;ve encountered the callee, we inspect its function body the same way we did for main.</p> <p>Dynamic calls through interface methods are trickier, because we don&rsquo;t know the set of types that implement the interface. We don&rsquo;t want to assume that every possible method in the program whose type matches is a possible target for the call, because some of those types may be instantiated only from dead code! That&rsquo;s why we gather the set of types converted to interfaces: the conversion makes each of these types reachable from <code>main</code>, so that its methods are now possible targets of dynamic calls.</p> <p>This leads to a chicken-and-egg situation. As we encounter each new reachable function, we discover more interface method calls and more conversions of concrete types to interface types. But as the cross product of these two sets (interface method calls × concrete types) grows ever larger, we discover new reachable functions. This class of problems, called &ldquo;dynamic programming&rdquo;, can be solved by (conceptually) making checkmarks in a large two-dimensional table, adding rows and columns as we go, until there are no more checks to add. The checkmarks in the final table tells us what is reachable; the blank cells are the dead code.</p> <div class="image"> <center> <img src="deadcode-rta.svg" alt="illustration of Rapid Type Analysis"/><br/> <i> The <code>main</code> function causes <code>Helloer</code> to be instantiated, and the <code>g.Greet</code> call<br/> dispatches to the <code>Greet</code> method of each type instantiated so far. </i> </center> </div> <p>Dynamic calls to (non-method) functions are treated similar to interfaces of a single method. And calls made <a href="https://pkg.go.dev/reflect#Value.Call" rel="noreferrer" target="_blank">using reflection</a> are considered to reach any method of any type used in an interface conversion, or any type derivable from one using the <code>reflect</code> package. But the principle is the same in all cases.</p> <h2 id="tests">Tests</h2> <p>RTA is a whole-program analysis. That means it always starts from a main function and works forward: you can&rsquo;t start from a library package such as <code>encoding/json</code>.</p> <p>However, most library packages have tests, and tests have main functions. We don&rsquo;t see them because they are generated behind the scenes of <code>go test</code>, but we can include them in the analysis using the <code>-test</code> flag.</p> <p>If this reports that a function in a library package is dead, that&rsquo;s a sign that your test coverage could be improved. For example, this command lists all the functions in <code>encoding/json</code> that are not reached by any of its tests:</p> <pre><code>$ deadcode -test -filter=encoding/json encoding/json encoding/json/decode.go:150:31: unreachable func: UnmarshalFieldError.Error encoding/json/encode.go:225:28: unreachable func: InvalidUTF8Error.Error </code></pre> <p>(The <code>-filter</code> flag restricts the output to packages matching the regular expression. By default, the tool reports all packages in the initial module.)</p> <h2 id="soundness">Soundness</h2> <p>All static analysis tools <a href="https://en.wikipedia.org/wiki/Rice%27s_theorem" rel="noreferrer" target="_blank">necessarily</a> produce imperfect approximations of the possible dynamic behaviors of the target program. A tool&rsquo;s assumptions and inferences may be &ldquo;sound&rdquo;, meaning conservative but perhaps overly cautious, or &ldquo;unsound&rdquo;, meaning optimistic but not always correct.</p> <p>The deadcode tool is no exception: it must approximate the set of targets of dynamic calls through function and interface values or using reflection. In this respect, the tool is sound. In other words, if it reports a function as dead code, it means the function cannot be called even through these dynamic mechanisms. However the tool may fail to report some functions that in fact can never be executed.</p> <p>The deadcode tool must also approximate the set of calls made from functions not written in Go, which it cannot see. In this respect, the tool is not sound. Its analysis is not aware of functions called exclusively from assembly code, or of the aliasing of functions that arises from the <a href="https://pkg.go.dev/cmd/compile#hdr-Compiler_Directives" rel="noreferrer" target="_blank"><code>go:linkname</code> directive</a>. Fortunately both of these features are rarely used outside the Go runtime.</p> <h2 id="try-it-out">Try it out</h2> <p>We run <code>deadcode</code> periodically on our projects, especially after refactoring work, to help identify parts of the program that are no longer needed.</p> <p>With the dead code laid to rest, you can focus on eliminating code whose time has come to an end but that stubbornly remains alive, continuing to drain your life force. We call such undead functions &ldquo;vampire code&rdquo;!</p> <p>Please try it out:</p> <pre><code>$ go install golang.org/x/tools/cmd/deadcode@latest </code></pre> <p>We&rsquo;ve found it useful, and we hope you do too.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/survey2024-h1">Share your feedback about developing with Go</a><br> <b>Previous article: </b><a href="/blog/survey2023-h2-results">Go Developer Survey 2023 H2 Results</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Go Developer Survey 2023 H2 Resultstag:blog.golang.org,2013:blog.golang.org/survey2023-h2-results2023-12-05T00:00:00+00:002023-12-05T00:00:00+00:00What we learned from our 2023 H2 developer survey <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/survey2023-h2-results"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Go Developer Survey 2023 H2 Results</h1> <p class="author"> Todd Kulesza<br> 5 December 2023 </p> <style type="text/css" scoped> .chart { margin-left: 1.5rem; margin-right: 1.5rem; width: 800px; } blockquote p { color: var(--color-text-subtle) !important; } .quote_source { font-style: italic; } @media (prefers-color-scheme: dark) { .chart { border-radius: 8px; } } </style> <h2 id="background">Background</h2> <p>In August 2023, the Go team at Google conducted our bi-annual survey of Go developers. We recruited participants via a public post on the Go blog and a randomized prompt in VS Code, resulting in 4,005 responses. We primarily focused survey questions around a few topics: general sentiment and feedback about developing with Go, technology stacks used alongside Go, how developers start new Go projects, recent experiences with toolchain error messages, and understanding developer interest around ML/AI.</p> <p>Thank you to everyone who participated in this survey! This report shares what we learned from your feedback.</p> <h2 id="tldr">tl;dr</h2> <ol> <li>Go developers said they are <strong>more interested in AI/ML tooling that improves the quality, reliability, and performance of code they write</strong>, rather than writing code for them. An always-awake, never-busy expert &ldquo;reviewer&rdquo; might be one of the more helpful forms of AI developer assistance.</li> <li>The top requests for improving toolchain warnings and errors were to <strong>make the messages more comprehensible and actionable</strong>; this sentiment was shared by developers of all experience levels, but was particularly strong among newer Go developers.</li> <li>Our experiment with project templates (<code>gonew</code>) appears to solve critical problems for Go developers (especially developers new to Go) and does so in a way that matches their existing workflows for starting a new project. Based on these findings, we believe <strong><code>gonew</code> can substantially reduce onboarding barriers for new Go developers and ease adoption of Go in organizations</strong>.</li> <li>Three out of every four respondents work on Go software that also uses cloud services; this is evidence that <strong>developers see Go as a language for modern, cloud-based development</strong>.</li> <li><strong>Developer sentiment towards Go remains extremely positive</strong>, with 90% of survey respondents saying they felt satisfied while working with Go during the prior year.</li> </ol> <h2 id="contents">Contents</h2> <ul> <li><a href="#sentiment">Developer sentiment</a></li> <li><a href="#devenv">Developer environments</a></li> <li><a href="#stacks">Tech stacks</a></li> <li><a href="#gonew">How developers start new Go projects</a></li> <li><a href="#err_handling">Developer goals for error handling</a></li> <li><a href="#mlai">Understanding ML/AI use cases</a></li> <li><a href="#err_msgs">Toolchain error messages</a></li> <li><a href="#microservices">Microservices</a></li> <li><a href="#modules">Module authorship and maintenance</a></li> <li><a href="#demographics">Demographics</a></li> <li><a href="#firmographics">Firmographics</a></li> <li><a href="#methodology">Methodology</a></li> <li><a href="#closing">Closing</a></li> </ul> <h2 id="sentiment">Developer sentiment</h2> <p>Go developers continue to report high levels of satisfaction with the Go ecosystem. A large majority of respondents said they felt satisfied while working with Go over the past year (90% satisfied, 6% dissatisfied), and a majority (52%) went further and said they were &ldquo;very satisfied&rdquo;, the highest rating. Longtime readers have likely noticed that this number doesn&rsquo;t change much from year to year. This is expected for a large, stable project like Go; we view this metric as a <a href="https://en.wikipedia.org/wiki/Economic_indicator#Lagging_indicators" rel="noreferrer" target="_blank">lagging indicator</a> that can help confirm widespread issues in the Go ecosystem, but isn&rsquo;t where we expect to first learn about potential problems.</p> <p>We typically find that the longer someone has worked with Go, the more likely they are to report being satisfied with it. This trend continued in 2023; among respondents with less than one year of Go experience, 82% reported satisfaction with the Go development experience, compared to the 94% of Go developers with five or more years of experience. There are likely a mix of factors contributing to this, such as some respondents developing an appreciation for Go&rsquo;s design choices over time, or deciding Go isn&rsquo;t a good fit for their work and so not returning to this survey in following years (i.e., <a href="https://en.wikipedia.org/wiki/Survivorship_bias" rel="noreferrer" target="_blank">survivorship bias</a>). Still, this data helps us quantify the current getting started experience for Go developers, and it seems clear we could do more to help emerging Gophers find their footing and enjoy early successes developing with Go.</p> <p>The key takeaway is that a large majority of people who chose to work with Go during the past year were happy with their experience. Further, the number of people working with Go continues to increase; we see evidence of this from external research like <a href="https://survey.stackoverflow.co/2023/#most-popular-technologies-language-prof" rel="noreferrer" target="_blank">Stack Overflow&rsquo;s Developer Survey</a> (which found 14% of professional developers worked with Go during the past year, a roughly 15% year-over-year increase), as well as analytics for <a href="/">go.dev</a> (which show an 8% rise in visitors year-over-year). Combining this growth with a high satisfaction score is evidence that Go continues to appeal to developers, and suggests that many developers who choose to learn the language feel good about their decision long afterwards. In their own words:</p> <blockquote> <p>&ldquo;After 30+ years of development in C, C++, Java, and now seven years of programming in Go, it is still the most productive language by far. It&rsquo;s not perfect (no language is), but it has the best balance of productivity, complexity, and performance.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 5 &ndash; 9 years of experience</span></p> </blockquote> <blockquote> <p>&ldquo;This is currently the best language I know, and I&rsquo;ve tried many. The tooling is awesome, compile times are great, and I can be really productive. I&rsquo;m glad I have Go as a tool, and I don&rsquo;t need to use TypeScript server-side. Thanks.&rdquo; <span class="quote_source">&mdash; Open source Go developer w/ 3 &ndash; 4 years of experience</span></p> </blockquote> <p><img src="survey2023h2/csat.svg" alt="Chart of developer satisfaction with Go" class="chart" /></p> <h2 id="devenv">Developer environments</h2> <p>As in prior years, the majority of survey respondents told us they work with Go on Linux (63%) and macOS (58%) systems. Small variations in these numbers from year to year are most likely dependent upon who finds and responds to this survey (particularly on the Go blog), as we don&rsquo;t see consistent year-over-year trends in the random sample coming from VS Code.</p> <p>We do continue to see that newer members of the Go community are more likely to be working with Windows than more experienced Go developers. We interpret this as a signal that Windows-based development is important for onboarding new developers to the Go ecosystem, and is a topic our team hopes to focus on more in 2024.</p> <p><img src="survey2023h2/os_dev.svg" alt="Chart of operating systems respondents use when developing Go software" class="chart" /> <img src="survey2023h2/os_dev_exp.svg" alt="Chart of operating systems respondents use when developing Go software, split by duration of experience" class="chart" /></p> <p>Respondents continue to be heavily focused on Linux deployments. Given the prevalence of Go for cloud development and containerized workloads, this is not surprising but is still an important confirmation. We found few meaningful differences based on factors such as organization size or experience level; indeed, while novice Go developers appear more likely to <em>develop</em> on Windows, 92% still <em>deploy</em> to Linux systems. Perhaps the most interesting finding from this breakdown is that more experienced Go developers said they deploy to a wider variety of systems (most notably WebAssembly and IoT), though it&rsquo;s unclear if this is because such deployments are challenging for newer Go developers or the result of experienced Go developers using Go in a broader range of contexts. We also observed that both IoT and WebAssembly have steadily increased in recent years, with each rising from 3% in 2021 to 6% and 5% in 2023, respectively.</p> <p><img src="survey2023h2/os_deploy.svg" alt="Chart of platforms respondents deploy Go software to" class="chart" /></p> <p>The computing architecture landscape has changed over the past few years, and we see that reflected in the current architectures Go developers say they work with. While x86-compatible systems still account for the majority of development (89%), ARM64 is also now used by a majority of respondents (56%). This adoption appears to be partly driven by Apple Silicon; macOS developers are now more likely to say they develop for ARM64 than for x86-based architectures (76% vs. 71%). However, Apple hardware isn&rsquo;t the only factor driving ARM64 adoption: among respondents who don&rsquo;t develop on macOS at all, 29% still say they develop for ARM64.</p> <p><img src="survey2023h2/arch.svg" alt="Chart of architectures respondents use with Go" class="chart" /></p> <p>The most common code editors among Go Developer Survey respondents continue to be <a href="https://code.visualstudio.com/" rel="noreferrer" target="_blank">VS Code</a> (44%) and <a href="https://www.jetbrains.com/go/" rel="noreferrer" target="_blank">GoLand</a> (31%). Both of these proportions ticked down slightly from 2023 H1 (46% and 33%, respectively), but remain within this survey&rsquo;s margin of error. Among the &ldquo;Other&rdquo; category, <a href="https://helix-editor.com/" rel="noreferrer" target="_blank">Helix</a> accounted for the majority of responses. Similar to the results for operating systems above, we don&rsquo;t believe this represents a meaningful shift in code editor usage, but rather shows some of the variability we expect to see in a community survey such as this. In particular, we exclude the randomly sampled respondents from VS Code for this question, as we know that group is heavily biased towards VS Code. However, that has the side effect of making these results more susceptible to variation each year.</p> <p>We also looked at respondents&rsquo; level of satisfaction with Go based on the editor they prefer using. After controlling for length of experience, we found no differences: we don&rsquo;t believe people enjoy working with Go more or less based on which code editor they use. That doesn&rsquo;t necessarily mean all Go editors are equal, but may reflect that people find the editor that is best for their own needs. This would suggest the Go ecosystem has a healthy diversity of different editors geared towards different use cases and developer preferences.</p> <p><img src="survey2023h2/editor_self_select.svg" alt="Chart of code editors respondents prefer to use with Go" class="chart" /></p> <h2 id="stacks">Tech stacks</h2> <p>To better understand the web of software and services that Go developers interact with, we asked several questions about tech stacks. We&rsquo;re sharing these results with the community to show which tools and platforms are in common use today, but we believe everyone should consider their own needs and use cases when selecting a tech stack. More plainly: we neither intend for readers to use this data to select components of their tech stack because they are popular, nor to avoid components because they are not commonly used.</p> <p>First, we can say with confidence that Go is a language for modern cloud-based development. Indeed, 75% of respondents work on Go software that integrates with cloud services. For nearly half of respondents, this involved AWS (48%), and almost one-third used GCP (29%) for their Go development and deployments. For both AWS and GCP, usage is equally balanced among large enterprises and smaller organizations. Microsoft Azure is the only cloud provider that is significantly more likely to be used in large organizations (companies with &gt; 1,000 employees) than smaller shops; other providers show no meaningful differences in usage based on the size of the organization.</p> <p><img src="survey2023h2/cloud.svg" alt="Chart of cloud platforms respondents use with Go" class="chart" /></p> <p>Databases are extremely common components of software systems, and we found that 91% of respondents said the Go services they work on use at least one. Most frequently this was PostgreSQL (59%), but with double digits of respondents reporting use of six additional databases, it&rsquo;s safe to say there are not just a couple of standard DBs for Go developers to consider. We again see differences based on organization size, with respondents from smaller organizations more likely to report using PostgreSQL and Redis, while developers from large organizations are somewhat more likely to use a database specific to their cloud provider.</p> <p><img src="survey2023h2/db.svg" alt="Chart of databases respondents use with Go" class="chart" /></p> <p>Another common component respondents reported using were caches or key-value stores; 68% of respondents said they work on Go software incorporating at least one of these. Redis was clearly the most common (57%), followed at a distance by etcd (10%) and memcached (7%).</p> <p><img src="survey2023h2/cache.svg" alt="Chart of caches respondents use with Go" class="chart" /></p> <p>Similar to databases, survey respondents told us they use a range of different observability systems. Prometheus and Grafana were the most commonly cited (both at 43%), but Open Telemetry, Datadog, and Sentry were all in double digits.</p> <p><img src="survey2023h2/metrics.svg" alt="Chart of metric systems respondents use with Go" class="chart" /></p> <p>Lest anyone wonder &ldquo;Have we JSON&rsquo;d all the things?&rdquo;&hellip; yes, yes we have. Nearly every respondent (96%!) said their Go software uses the JSON data format; that&rsquo;s about as close to universal as you&rsquo;ll see with self-reported data. YAML, CSV, and protocol buffers are also all used by roughly half of respondents, and double-digit proportions work with TOML and XML as well.</p> <p><img src="survey2023h2/data.svg" alt="Chart of data formats respondents use with Go" class="chart" /></p> <p>For authentication and authorization services, we found most respondents are building upon the foundations provided by standards such as <a href="https://jwt.io/introduction" rel="noreferrer" target="_blank">JWT</a> and <a href="https://oauth.net/2/" rel="noreferrer" target="_blank">OAuth2</a>. This also appears to be an area where an organization&rsquo;s cloud provider&rsquo;s solution is about as likely to be used as most turn-key alternatives.</p> <p><img src="survey2023h2/auth.svg" alt="Chart of authentication systems respondents use with Go" class="chart" /></p> <p>Finally, we have a bit of a grab bag of other services that don&rsquo;t neatly fit into the above categories. We found that nearly half of respondents work with gRPC in their Go software (47%). For infrastructure-as-code needs, Terraform was the tool of choice for about ¼ of respondents. Other fairly common technologies used alongside Go included Apache Kafka, ElasticSearch, GraphQL, and RabbitMQ.</p> <p><img src="survey2023h2/other_tech.svg" alt="Chart of authentication systems respondents use with Go" class="chart" /></p> <p>We also looked at which technologies tended to be used together. While nothing clearly analogous to the classic <a href="https://en.wikipedia.org/wiki/LAMP_(software_bundle)" rel="noreferrer" target="_blank">LAMP stack</a> emerged from this analysis, we did identify some interesting patterns:</p> <ul> <li>All or nothing: Every category (except data formats) showed a strong correlation where if a respondent answered “None” to one category, they likely answered “None” for all of the others. We interpret this as evidence that a minority of use cases require none of these tech stack components, but once the use case requires any one of them, it likely requires (or is at least simplified by) more than just one.</li> <li>A bias towards cross-platform technologies: Provider-specific solutions (i.e., services that are unique to a single cloud platform) were not commonly adopted. However, if respondents used one provider-specific solution (e.g., for metrics), they were substantially more likely to also say they used cloud-specific solutions in order areas (e.g., databases, authentication, caching, etc.).</li> <li>Multicloud: The three biggest cloud platforms were most likely to be involved in multicloud setups. For example, if an organization is using any non-AWS cloud provider, they’re probably also using AWS. This pattern was clearest for Amazon Web Services, but was also apparent (to a lesser extent) for Google Cloud Platform and Microsoft Azure.</li> </ul> <h2 id="gonew">How developers start new Go projects</h2> <p>As part of our <a href="/blog/gonew">experimentation with project templates</a>, we wanted to understand how Go developers get started with new projects today. Respondents told us their biggest challenges were choosing an appropriate way to structure their project (54%) and learning how to write idiomatic Go (47%). As two respondents phrased it:</p> <blockquote> <p>&ldquo;Finding an appropriate structure and the right abstraction levels for a new project can be quite tedious; looking at high-profile community and enterprise projects for inspiration can be quite confusing as everyone structures their project differently&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 5 &ndash; 9 years of Go experience</span></p> </blockquote> <blockquote> <p>&ldquo;It would be great if [Go had a] toolchain to create [a project&rsquo;s] basic structure for web or CLI like `go init &lt;project name&gt;`&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 3 &ndash; 4 years of experience</span></p> </blockquote> <p>Newer Go developers were even more likely to encounter these challenges: the proportions increased to 59% and 53% for respondents with less than two years of experience with Go, respectively. These are both areas we hope to improve via our <code>gonew</code> prototype: templates can provide new Go developers with well-tested project structures and design patterns, with initial implementations written in idiomatic Go. These survey results have helped our team to keep the purpose of <code>gonew</code> focused on tasks the Go community most struggle with.</p> <p><img src="survey2023h2/new_challenge.svg" alt="Chart of challenges respondents faced when starting new Go projects" class="chart" /></p> <p>A majority of respondents told us they either use templates or copy+paste code from existing projects when starting a new Go project (58%). Among respondents with less than five years of Go experience, this proportion increased to nearly ⅔ (63%). This was an important confirmation that the template-based approach in <code>gonew</code> seems to meet developers where they already are, aligning a common, informal approach with <code>go</code> command-style tooling. This is further supported by the common feature requests for project templates: a majority of respondents requested 1) a pre-configured directory structure to organize their project and 2) sample code for common tasks in the project domain. These results are well-aligned with the challenges developers said they faced in the previous section. The responses to this question also help tease apart the difference between project structure and design patterns, with nearly twice as many respondents saying they want Go project templates to provide the former than the latter.</p> <p><img src="survey2023h2/new_approach.svg" alt="Chart of approaches respondents used when starting new Go projects" class="chart" /></p> <p><img src="survey2023h2/templates.svg" alt="Chart of functionality respondents requested when starting new Go projects" class="chart" /></p> <p>A majority of respondents told us the ability to make changes to a template <em>and</em> have those changes propagate to projects based on that template was of at least moderate importance. Anecdotally, we haven&rsquo;t spoken with any developers who <em>currently</em> have this functionality with home-grown template approaches, but it suggests this is an interesting avenue for future development.</p> <p><img src="survey2023h2/template_updates.svg" alt="Chart of respondent interest in updatable templates" class="chart" /></p> <h2 id="err_handling">Developer goals for error handling</h2> <p>A perennial topic of discussion among Go developers is potential improvements to error handling. As one respondent summarized:</p> <blockquote> <p>&ldquo;Error handling adds too much boilerplate (I know, you probably heard this before)&rdquo; <span class="quote_source">&mdash; Open source Go developer w/ 1 &ndash; 2 years of experience</span></p> </blockquote> <p>But, we also hear from numerous developers that they appreciate Go&rsquo;s approach to error handling:</p> <blockquote> <p>&ldquo;Go error handling is simple and effective. As I have backends in Java and C# and exploring Rust and Zig now, I am always pleased to go back to write Go code. And one of the reasons is, believe it or not, error handling. It is really simple, plain and effective. Please leave it that way.&rdquo; <span class="quote_source">&mdash; Open source Go developer w/ 5 &ndash; 9 years of experience</span></p> </blockquote> <p>Rather than ask about specific modifications to error handling in Go, we wanted to better understand developers&rsquo; higher-level goals and whether Go&rsquo;s current approach has proven useful and usable. We found that a majority of respondents appreciate Go&rsquo;s approach to error handling (55%) and say it helps them know when to check for errors (50%). Both of these outcomes were stronger for respondents with more Go experience, suggesting that either developers grow to appreciate Go&rsquo;s approach to error handling over time, or that this is one factor leading developers to eventually leave the Go ecosystem (or at least stop responding to Go-related surveys). Many survey respondents also felt that Go requires a lot of tedious, boilerplate code to check for errors (43%); this remained true regardless of how much prior Go experience respondents had. Interestingly, when respondents said they appreciate Go&rsquo;s error handling, they were unlikely to say it also results in lots of boilerplate code&mdash;our team had a hypothesis that Go developers can both appreciate the language&rsquo;s approach to error handling and feel it&rsquo;s too verbose, but only 14% of respondents agreed with both statements.</p> <p>Specific issues that respondents cited include challenges knowing which error types to check for (28%), wanting to easily show a stack trace along with the error message (28%), and the ease with which errors can be entirely ignored (19%). About ⅓ of respondents were also interested in adopting concepts from other languages, such as Rust&rsquo;s <code>?</code> operator (31%).</p> <p>The Go team has no plans to add exceptions to the language, but since this is anecdotally a common request, we included it as a response choice. Only 1 in 10 respondents said they wished they could use exceptions in Go, and this was inversely related to experience&mdash;more veteran Go developers were less likely to be interested in exceptions than respondents newer to the Go community.</p> <p><img src="survey2023h2/error_handling.svg" alt="Chart of respondents' thoughts about Go's error handling approach" class="chart" /></p> <h2 id="mlai">Understanding ML/AI use cases</h2> <p>The Go team is considering how the unfolding landscape of new ML/AI technologies may impact software development in two distinct veins: 1) how might ML/AI tooling help engineers write better software, and 2) how might Go help engineers bring ML/AI support to their applications and services? Below, we delve into each of these areas.</p> <h3 id="helping-engineers-write-better-software">Helping engineers write better software</h3> <p>There&rsquo;s little denying we&rsquo;re in <a href="https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2023-gartner-hype-cycle" rel="noreferrer" target="_blank">a hype cycle around the possibilities for AI/ML</a>. We wanted to take a step back to focus on the broader challenges developers face and where they think AI might prove useful in their regular work. The answers were a bit surprising, especially given the industry&rsquo;s current focus on coding assistants.</p> <p>First, we see a few AI use cases that about half of respondents thought could be helpful: generating tests (49%), suggesting best practices in-situ (47%), and catching likely mistakes early in the development process (46%). A unifying theme of these top use cases is that each could help improve the quality and reliability of code an engineer is writing. A fourth use case (help writing documentation) garnered interest from about ⅓ of respondents. The remaining cases comprise a long tail of potentially fruitful ideas, but these are of significantly less general interest than the top four.</p> <p>When we look at developers&rsquo; duration of experience with Go, we find that novice respondents are interested in help resolving compiler errors and explaining what a piece of Go code does more than veteran Go developers. These might be areas where AI could help improve the getting started experience for new Gophers; for example, an AI assistant could help explain in natural language what an undocumented block of code does, or suggest common solutions to specific error messages. Conversely, we see no differences between experience levels for topics like &ldquo;catch common mistakes&rdquo;&mdash;both novice and veteran Go developers say they would appreciate tooling to help with this.</p> <p>One can squint at this data and see three broad trends:</p> <ol> <li>Respondents voiced interest in getting feedback from &ldquo;expert reviewers&rdquo; in real-time, not just during review time.</li> <li>Generally, respondents appeared most interested in tooling that saves them from potentially less-enjoyable tasks (e.g., writing tests or documenting code).</li> <li>Wholesale writing or translating of code was of fairly low interest, especially to developers with more than a year or two of experience.</li> </ol> <p>Taken together, it appears that today, developers are less excited by the prospect of machines doing the fun (e.g., creative, enjoyable, appropriately challenging) parts of software development, but do see value in another set of &ldquo;eyes&rdquo; reviewing their code and potentially handling dull or repetitive tasks for them. As one respondent phrased it:</p> <blockquote> <p>&ldquo;I&rsquo;m specifically interested in using AI/ML to improve my productivity with Go. Having a system that is trained in Go best practices, can catch anti-patterns, bugs, generate tests, with a low rate of hallucination, would be killer.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 5 &ndash; 9 years of experience</span></p> </blockquote> <p>This survey, however, is just one data point in a quickly-evolving research field, so it&rsquo;s best to keep these results in context.</p> <p><img src="survey2023h2/ml_use_cases.svg" alt="Chart of respondents' interest in AI/ML support for development tasks" class="chart" /></p> <h3 id="bringing-ai-features-to-applications-and-services">Bringing AI features to applications and services</h3> <p>In addition to looking at how Go developers might benefit from AI/ML-powered tooling, we explored their plans for building AI-powered applications and services (or supporting infrastructure) with Go. We found that we&rsquo;re still early in <a href="https://en.wikipedia.org/wiki/Technology_adoption_life_cycle" rel="noreferrer" target="_blank">the adoption curve</a>: most respondents have not yet tried to use Go in these areas, though every topic saw some level of interest from roughly half of respondents. For example, a majority of respondents reported interest in integrating the Go services they work on with LLMs (49%), but only 13% have already done so or are currently evaluating this use case. At the time of this survey, responses gently suggest that developers may be most interested in using Go to call LLMs directly, build the data pipelines needed to power ML/AI systems, and for creating API endpoints other services can call to interact with ML/AI models. As one example, this respondent described the benefits they hoped to gain by using Go in their data pipelines:</p> <blockquote> <p>&ldquo;I want to integrate the ETL [extract, transform, and load] part using Go, to keep a consistent, robust, reliable codebase.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 3 &ndash; 4 years of experience</span></p> </blockquote> <p><img src="survey2023h2/ml_adoption.svg" alt="Chart of respondents' current use of (and interest in) Go for AI/ML systems" class="chart" /></p> <h2 id="err_msgs">Toolchain error messages</h2> <p>Many developers can relate to the frustrating experience of seeing an error message, thinking they know what it means and how to resolve it, but after hours of fruitless debugging realize it meant something else entirely. One respondent explained their frustration as follows:</p> <blockquote> <p>&ldquo;So often the printed complaints wind up having nothing to do with the problem, but it can take an hour before I discover that that&rsquo;s the case. The error messages are unnervingly terse, and don&rsquo;t seem to go out of their way to guess as to what the user might be trying to do or [explain what they&rsquo;re] doing wrong.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 10+ years of experience</span></p> </blockquote> <p>We believe the warnings and errors emitted by developer tooling should be brief, understandable, and actionable: the human reading them should be able to accurately understand what went wrong and what they can do to resolve the issue. This is an admittedly high bar to strive for, and with this survey we took some measurements to understand how developers perceive Go&rsquo;s current warning and error messages.</p> <p>When thinking about the most recent Go error message they worked through, respondents told us there was much room for improvement. Only a small majority understood what the problem was from the error message alone (54%), and even fewer knew what to do next to resolve the issue (41%). It appears a relatively small amount of additional information could meaningfully increase these proportions, as ¼ of respondents said they mostly knew how to fix the problem, but needed to see an example first. Further, with 11% of respondents saying they couldn&rsquo;t make sense of the error message, we now have a baseline for current understandability of the Go toolchain&rsquo;s error messages.</p> <p>Improvements to Go&rsquo;s toolchain error messages would especially benefit less-experienced Gophers. Respondents with up to two years of experience were less likely than veteran Gophers to say they understood the problem (47% vs. 61%) or knew how to fix it (29% vs. 52%), and were twice as likely to need to search online to fix the issue (21% vs. 9%) or even make sense of what the error meant (15% vs. 7%).</p> <p>We hope to focus on improving toolchain error messages during 2024. These survey results suggest this is an area of frustration for developers of all experience levels, and will particularly help newer developers get started with Go.</p> <p><img src="survey2023h2/err_exp.svg" alt="Chart of error handling experiences" class="chart" /></p> <p><img src="survey2023h2/err_exp_exp.svg" alt="Chart of error handling experiences, split by duration of Go experience" class="chart" /></p> <p>To understand <em>how</em> these messages might be improved, we asked survey respondents an open-ended question: &ldquo;If you could make a wish and improve one thing about error messages in the Go toolchain, what would you change?&rdquo;. The responses largely align with our hypothesis that good error messages are both understandable and actionable. The most common response was some form of &ldquo;Help me understand what led to this error&rdquo; (36%), 21% of respondents explicitly asked for guidance to fix the problem, and 14% of respondents called out languages such as Rust or Elm as exemplars which strive to do both of these things. In the words of one respondent:</p> <blockquote> <p>&ldquo;For compilation errors, Elm or Rust-style output pinpointing exact issue in the source code. Errors should include suggestions to fix them where possible&hellip; I think a general policy of &lsquo;optimize error output to be read by humans&rsquo; with &lsquo;provide suggestions where possible&rsquo; would be very welcome here.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 5 &ndash; 9 years of experience</span></p> </blockquote> <p>Understandably, there is a fuzzy conceptual boundary between toolchain error messages and runtime error messages. For example, one of the top requests involved improved stack traces or other approaches to assist debugging runtime crashes (22%). Similarly, a surprising theme in 4% of the feedback was about challenges with getting help from the <code>go</code> command itself. These are great examples of the Go community helping us identify related pain points that weren&rsquo;t otherwise on our radar. We started this investigation focused on improving compile-time errors, but one of the core areas Go developers would like to see improved actually relates to run-time errors, while another was about the <code>go</code> command&rsquo;s help system.</p> <blockquote> <p>&ldquo;When an error is thrown, the call stack can be huge and includes a bunch of files I don&rsquo;t care about. I just want to know where the problem is in MY code, not the library I&rsquo;m using, or how the panic was handled.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 1 &ndash; 2 years of experience</span></p> </blockquote> <blockquote> <p>&ldquo;Getting help via `go help run` dumps a wall of text, with links to further readings to find the available command-line flags. Or the fact that it understands `go run &ndash;help` but instead of showing the help, it says &lsquo;please run go help run instead&rsquo;. Just show me list of flags in `go run &ndash;help`.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 3 &ndash; 4 years of experience</span></p> </blockquote> <p><img src="survey2023h2/text_err_wish.svg" alt="Chart of potential improvements for Go's error messages" class="chart" /></p> <h2 id="microservices">Microservices</h2> <p>We commonly hear that developers find Go to be a great fit for microservices, but we have never tried to quantify how many Go developers have adopted this type of service architecture, understand how those services communicate with one another, or the challenges developers encounter when working on them. This year we added a few questions to better understand this space.</p> <p>A plurality of respondents said they work mostly on microservices (43%), with another ¼ saying they work on a mix of both microservices and monoliths. Only about ⅕ of respondents work mostly on monolithic Go applications. This is one of the few areas where we see differences based on the size of organization respondents work at&mdash;large organizations seem more likely to have adopted a microservice architecture than smaller companies. Respondents from large organizations (&gt;1,000 employees) were most likely to say they work on microservices (55%), with only 11% of these respondents working primarily on monoliths.</p> <p><img src="survey2023h2/service_arch.svg" alt="Chart of respondents' primary service architecture" class="chart" /></p> <p>We see some bifurcation in the number of microservices comprising Go platforms. One group is composed of a handful (2 to 5) of services (40%), while the other consists of larger collections, with a minimum of 10 component services (37%). The number of microservices involved does not appear to be correlated with organization size.</p> <p><img src="survey2023h2/service_num.svg" alt="Chart of the number of microservices respondents' systems involve" class="chart" /></p> <p>A large majority of respondents use some form of direct response request (e.g., RPC, HTTP, etc.) for microservice communication (72%). A smaller proportion use message queues (14%) or a pub/sub approach (9%); again, we see no differences here based on organization size.</p> <p><img src="survey2023h2/service_comm.svg" alt="Chart of how microservices communicate with one another" class="chart" /></p> <p>A majority of respondents build microservices in a polyglot of languages, with only about ¼ exclusively using Go. Python is the most common companion language (33%), alongside Node.js (28%) and Java (26%). We again see differences based on organization size, with larger organizations more likely to be integrating Python (43%) and Java (36%) microservices, while smaller organizations are a bit more likely to only use Go (30%). Other languages appeared to be used equally based on organization size.</p> <p><img src="survey2023h2/service_lang.svg" alt="Chart of other languages that Go microservices interact with" class="chart" /></p> <p>Overall, respondents told us testing and debugging were their biggest challenge when writing microservice-based applications, followed by operational complexity. Many other challenges occupy the long tail on this graph, though &ldquo;portability&rdquo; stands out as a non-issue for most respondents. We interpret this to mean that such services aren&rsquo;t intended to be portable (beyond basic containerization); for example, if an organization&rsquo;s microservices are initially powered by PostgreSQL databases, developers aren&rsquo;t concerned with potentially porting this to an Oracle database in the near future.</p> <p><img src="survey2023h2/service_challenge.svg" alt="Chart of challenges respondents face when writing microservice-based applications" class="chart" /></p> <h2 id="modules">Module authorship and maintenance</h2> <p>Go has a vibrant ecosystem of community-driven modules, and we want to understand the motivations and challenges faced by developers who maintain these modules. We found that about ⅕ of respondents maintain (or used to maintain) an open-source Go module. This was a surprisingly high proportion, and may be biased due to how we share this survey: module maintainers may be more likely to closely follow the Go blog (where this survey is announced) than other Go developers.</p> <p><img src="survey2023h2/mod_maintainer.svg" alt="Chart of how many respondents have served as a maintainer for a public Go module" class="chart" /></p> <p>Module maintainers appear to be largely self-motivated&mdash;they report working on modules that they need for personal (58%) or work (56%) projects, that they do so because they enjoy working on these modules (63%) and being part of the public Go community (44%), and that they learn useful skills from their module maintainership (44%). More external motivations, such as receiving recognition (15%), career advancement (36%), or cash money (20%) are towards the bottom of the list.</p> <p><img src="survey2023h2/mod_motivation.svg" alt="Chart of the motivations of public module maintainers" class="chart" /></p> <p>Given the forms of <a href="https://en.wikipedia.org/wiki/Motivation#Intrinsic_and_extrinsic" rel="noreferrer" target="_blank">intrinsic motivation</a> identified above, it follows that a key challenge for module maintainers is finding time to devote to their module (41%). While this might not seem like an actionable finding in itself (we can&rsquo;t give Go developers an extra hour or two each day, right?), it&rsquo;s a helpful lens through which to view module tooling and development&mdash;these tasks are most likely occurring while the developer is already pressed for time, and perhaps it&rsquo;s been weeks or months since they last had an opportunity to work on it, so things aren&rsquo;t fresh in their memory. Thus, aspects like understandable and actionable error messages can be particularly helpful: rather than require someone to once again search for specific <code>go</code> command syntax, perhaps the error output could provide the solution they need right in their terminal.</p> <p><img src="survey2023h2/mod_challenge.svg" alt="Chart of challenges respondents face when maintaining public Go modules" class="chart" /></p> <h2 id="demographics">Demographics</h2> <p>Most survey respondents reported using Go for their primary job (78%), and a majority (59%) said they use it for personal or open-source projects. In fact, it&rsquo;s common for respondents to use Go for <em>both</em> work and personal/OSS projects, with 43% of respondents saying they use Go in each of these situations.</p> <p><img src="survey2023h2/where.svg" alt="Chart of situations in which respondents recently used Go" class="chart" /></p> <p>The majority of respondents have been working with Go for under five years (68%). As we&rsquo;ve seen in <a href="/blog/survey2023-q1-results#novice-respondents-are-more-likely-to-prefer-windows-than-more-experienced-respondents">prior years</a>, people who found this survey via VS Code tended to be less experienced than people who found the survey via other channels.</p> <p>When we break down where people use Go by their experience level, two findings stand out. First, a majority of respondents from all experience levels said they&rsquo;re using Go professionally; indeed, for people with over two years of experience, the vast majority use Go at work (85% &ndash; 91%). A similar trend exists for open-source development. The second finding is that developers with less Go experience are more likely to be using Go to expand their skill set (38%) or to evaluate it for use at work (13%) than more experienced Go developers. We interpret this to mean that many Gophers initially view Go as part of &ldquo;upskilling&rdquo; or expanding their understanding of software development, but that within a year or two, they look to Go as more of a tool for doing than learning.</p> <p><img src="survey2023h2/go_exp.svg" alt="Chart of how long respondents have been working with Go" class="chart" /></p> <p><img src="survey2023h2/where_exp.svg" alt="Chart of situations in which respondents recently used Go, split by their level of Go experience" class="chart" /></p> <p>The most common use cases for Go continue to be API/RPC services (74%) and command line tools (62%). People tell us Go is a great choice for these types of software for several reasons, including its built-in HTTP server and concurrency primitives, ease of cross-compilation, and single-binary deployments.</p> <p>The intended audience for much of this tooling is in business settings (62%), with 17% of respondents reporting that they develop primarily for more consumer-oriented applications. This isn&rsquo;t surprising given the low use of Go for consumer-focused applications such as desktop, mobile, or gaming, vs. its very high use for backend services, CLI tooling, and cloud development, but it is a useful confirmation of how heavily Go is used in B2B settings.</p> <p>We also looked for differences based on respondents&rsquo; level of experience with Go and organization size. More experienced Go developers reported building a wider variety of different things in Go; this trend was consistent across every category of app or service. We did not find any notable differences in what respondents are building based on their organization size.</p> <p><img src="survey2023h2/what.svg" alt="Chart of the types of things respondents are building with Go" class="chart" /></p> <p><img src="survey2023h2/enduser.svg" alt="Chart of the audience using the software respondents build" class="chart" /></p> <p>Respondents were about equally likely to say this was the first time they&rsquo;ve responded to the Go Developer Survey vs. saying they had taken this survey before. There is a meaningful difference between people who learned about this survey via the Go blog, where 61% reported taking this survey previously, vs. people who learned about this survey via a notification in VS Code, where only 31% said they&rsquo;ve previously taken this survey. We don&rsquo;t expect people to perfectly recall every survey they&rsquo;ve responded to on the internet, but this gives us some confidence that we&rsquo;re hearing from a balanced mix of new and repeat respondents with each survey. Further, this tells us our combination of social media posts and random in-editor sampling are both necessary for hearing from a diverse set of Go developers.</p> <p><img src="survey2023h2/return_respondent.svg" alt="Chart of how many respondents said they have taken this survey before" class="chart" /></p> <h2 id="firmographics">Firmographics</h2> <p>Respondents to this survey reported working at a mix of different organizations, from thousand-person-plus enterprises (27%), to midsize businesses (25%) and smaller organizations with &lt; 100 employees (44%). About half of respondents work in the technology industry (50%), a large increase over the next most-common industry&mdash;financial services&mdash;at 13%.</p> <p>This is statistically unchanged from the past few Go Developer Surveys&mdash;we continue to hear from people in different countries and in organizations of different sizes and industries at consistent rates year after year.</p> <p><img src="survey2023h2/org_size.svg" alt="Chart of the different organization sizes where respondents use Go" class="chart" /></p> <p><img src="survey2023h2/industry.svg" alt="Chart of the different industries where respondents use Go" class="chart" /></p> <p><img src="survey2023h2/location.svg" alt="Chart of countries or regions where respondents are located" class="chart" /></p> <h2 id="methodology">Methodology</h2> <p>Most survey respondents &ldquo;self-selected&rdquo; to take this survey, meaning they found it on the Go blog or other social Go channels. A potential problem with this approach is that people who don&rsquo;t follow these channels are less likely to learn about the survey, and might respond differently than people who do closely follow them. About 40% of respondents were randomly sampled, meaning they responded to the survey after seeing a prompt for it in VS Code (everyone using the VS Code Go plugin between mid-July &ndash; mid-August 2023 had a 10% of receiving this random prompt). This randomly sampled group helps us generalize these findings to the larger community of Go developers.</p> <h3 id="how-to-read-these-results">How to read these results</h3> <p>Throughout this report we use charts of survey responses to provide supporting evidence for our findings. All of these charts use a similar format. The title is the exact question that survey respondents saw. Unless otherwise noted, questions were multiple choice and participants could only select a single response choice; each chart&rsquo;s subtitle will tell the reader if the question allowed multiple response choices or was an open-ended text box instead of a multiple choice question. For charts of open-ended text responses, a Go team member read and manually categorized the responses. Many open-ended questions elicited a wide variety of responses; to keep the chart sizes reasonable, we condensed them to a maximum of the top 10 themes, with additional themes all grouped under &ldquo;Other&rdquo;. The percentage labels shown in charts are rounded to the nearest integer (e.g., 1.4% and 0.8% will both be displayed as 1%), but the length of each bar and row ordering are based on the unrounded values.</p> <p>To help readers understand the weight of evidence underlying each finding, we included error bars showing the 95% <a href="https://en.wikipedia.org/wiki/Confidence_interval" rel="noreferrer" target="_blank">confidence interval</a> for responses; narrower bars indicate increased confidence. Sometimes two or more responses have overlapping error bars, which means the relative order of those responses is not statistically meaningful (i.e., the responses are effectively tied). The lower right of each chart shows the number of people whose responses are included in the chart, in the form &ldquo;n = [number of respondents]&rdquo;.</p> <p>We include select quotes from respondents to help clarify many of our findings. These quotes include the length of times the respondent has used Go. If the respondent said they use Go at work, we refer to them as a &ldquo;professional Go developer&rdquo;; if they don&rsquo;t use Go at work but do use Go for open-source development, we refer to them as an &ldquo;open-source Go developer&rdquo;.</p> <h2 id="closing">Closing</h2> <p>The final question on our survey always asks respondents whether there&rsquo;s anything else they&rsquo;d like to share with us about Go. The most common piece of feedback people provide is &ldquo;thanks!&rdquo;, and this year was no different (33%). In terms of requested language improvements, we see a three-way statistical tie between improved expressivity (12%), improved error handling (12%), and improved type safety or reliability (9%). Respondents had a variety of ideas for improving expressivity, with the general trend of this feedback being &ldquo;Here&rsquo;s a specific thing I write frequently, and I wish it were easier to express this in Go&rdquo;. The issues with error handling continue to be complaints about the verbosity of this code today, while feedback about type safety most commonly touched on <a href="https://en.wikipedia.org/wiki/Tagged_union" rel="noreferrer" target="_blank">sum types</a>. This type of high-level feedback is extremely useful when the Go team tries to plan focus areas for the coming year, as it tells us general directions in which the community is hoping to steer the ecosystem.</p> <blockquote> <p>&ldquo;I know about Go&rsquo;s attitude towards simplicity and I appreciate it. I just wish there [were] slightly more features. For me it would be better error handling (not exceptions though), and maybe some common creature comforts like map/reduce/filter and ternary operators. Anything not too obscure that&rsquo;ll save me some &lsquo;if&rsquo; statements.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 1 &ndash; 2 years of experience</span></p> </blockquote> <blockquote> <p>&ldquo;Please keep Go in line with the long term values Go established so long ago — language and library stability. [&hellip;] It is an environment I can rely on to not break my code after 2 or 3 years. For that, thank you very much.&rdquo; <span class="quote_source">&mdash; Professional Go developer w/ 10+ years of experience</span></p> </blockquote> <p><img src="survey2023h2/text_anything_else.svg" alt="Chart of other topics respondents shared with us" class="chart" /></p> <p>That&rsquo;s all for this bi-annual iteration of the Go Developer Survey. Thanks to everyone who shared their feedback about Go&mdash;we have immense gratitude for taking your time to help shape Go&rsquo;s future, and we hope you see some of your own feedback reflected in this report. 🩵</p> <p>&mdash; Todd (on behalf of the Go team at Google)</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/deadcode">Finding unreachable functions with deadcode</a><br> <b>Previous article: </b><a href="/blog/14years">Fourteen Years of Go</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Fourteen Years of Gotag:blog.golang.org,2013:blog.golang.org/14years2023-11-10T00:00:00+00:002023-11-10T00:00:00+00:00Happy Birthday, Go! <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/14years"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Fourteen Years of Go</h1> <p class="author"> Russ Cox, for the Go team<br> 10 November 2023 </p> <img src="/doc/gopher/gopherdrink.png" height="219" width="223" align="right" style="margin: 0 0 1em 1em;"> <p>Today we celebrate the fourteenth birthday of the Go open source release! Go has had a great year, with two feature-filled releases and other important milestones.</p> <p>We released <a href="/blog/go1.20">Go 1.20 in February</a> and <a href="/blog/go1.21">Go 1.21 in August</a>, focusing more on implementation improvements than new language changes.</p> <p>Profile-guided optimization (PGO), <a href="/blog/pgo-preview">previewed in Go 1.20</a> and <a href="/blog/pgo">released in Go 1.21</a>, allows the Go compiler to read a profile of your program and then spend more time optimizing the parts of your program that run most often. In Go 1.21, workloads typically get between 2% and 7% CPU usage improvements from enabling PGO. See “<a href="/blog/pgo">Profile-guided optimization in Go 1.21</a>” for an overview and the <a href="/doc/pgo">profile-guided optimization user guide</a> for complete documentation.</p> <p>Go has provided support for gathering coverage profiles during <code>go test</code> <a href="/blog/cover">since Go 1.2</a>. Go 1.20 added support for gathering coverage profiles in binaries built by <code>go build</code>, allowing you to gather coverage during larger integration tests as well. See “<a href="/blog/integration-test-coverage">Code coverage for Go integration tests</a>” for details.</p> <p>Compatibility has been an important part of Go since “<a href="/doc/go1compat">Go 1 and the Future of Go Programs</a>”. Go 1.21 improved compatibility further by expanding the conventions for use of GODEBUG in situations where we need to make a change, such as an important bug fix, that must be permitted but may still break existing programs. See the blog post “<a href="/blog/compat">Backward Compatibility, Go 1.21, and Go 2</a>” for an overview and the documentation “<a href="/doc/godebug">Go, Backwards Compatibility, and GODEBUG</a>” for details.</p> <p>Go 1.21 also shipped support for built-in toolchain management, allowing you to change which version of the Go toolchain you use in a specific module as easily as you change the versions of other dependencies. See the blog post “<a href="/blog/toolchain">Forward Compatibility and Toolchain Management in Go 1.21</a>” for an overview and the documentation “<a href="/doc/toolchain">Go Toolchains</a>” for details.</p> <p>Another important tooling achievement was the integration of on-disk indexes into gopls, the Go LSP server. This cut gopls&rsquo;s startup latency and memory usage by 3-5X in typical use cases. “<a href="/blog/gopls-scalability">Scaling gopls for the growing Go ecosystem</a>” explains the technical details. You can make sure you&rsquo;re running the latest gopls by running:</p> <pre><code>go install golang.org/x/tools/gopls@latest </code></pre> <p>Go 1.21 introduced new <a href="/pkg/cmp/">cmp</a>, <a href="/pkg/maps/">maps</a>, and <a href="/pkg/slices/">slices</a> packages — Go’s first generic standard libraries — as well as expanding the set of comparable types. For details about that, see the blog post “<a href="/blog/comparable">All your comparable types</a>”.</p> <p>Overall, we continue to refine generics and to write talks and blog posts explaining important details. Two notable posts this year were “<a href="/blog/deconstructing-type-parameters">Deconstructing Type Parameters</a>”, and “<a href="/blog/type-inference">Everything You Always Wanted to Know About Type Inference – And a Little Bit More</a>”.</p> <p>Another important new package in Go 1.21 is <a href="/pkg/log/slog/">log/slog</a>, which adds an official API for structured logging to the standard library. See “<a href="/blog/slog">Structured logging with slog</a>” for an overview.</p> <p>For the WebAssembly (Wasm) port, Go 1.21 shipped support for running on WebAssembly System Interface (WASI) preview 1. WASI preview 1 is a new “operating system” interface for Wasm that is supported by most server-side Wasm environments. See “<a href="/blog/wasi">WASI support in Go</a>” for a walkthrough.</p> <p>On the security side, we are continuing to make sure Go leads the way in helping developers understand their dependencies and vulnerabilities, with <a href="/blog/govulncheck">Govulncheck 1.0 launching in July</a>. If you use VS Code, you can run govulncheck directly in your editor using the Go extension: see <a href="/doc/tutorial/govulncheck-ide">this tutorial</a> to get started. And if you use GitHub, you can run govulncheck as part of your CI/CD, with the <a href="https://github.com/marketplace/actions/golang-govulncheck-action" rel="noreferrer" target="_blank">GitHub Action for govulncheck</a>. For more about checking your dependencies for vulnerability problems, see this year&rsquo;s Google I/O talk, “<a href="https://www.youtube.com/watch?v=HSt6FhsPT8c&amp;ab_channel=TheGoProgrammingLanguage" rel="noreferrer" target="_blank">Build more secure apps with Go and Google</a>”.)</p> <p>Another important security milestone was Go 1.21&rsquo;s highly reproducible toolchain builds. See “<a href="/blog/rebuild">Perfectly Reproducible, Verified Go Toolchains</a>” for details, including a demonstration of reproducing an Ubuntu Linux Go toolchain on a Mac without using any Linux tools at all.</p> <p>It has been a busy year!</p> <p>In Go&rsquo;s 15th year, we&rsquo;ll keep working to make Go the best environment for software engineering at scale. One change we&rsquo;re particularly excited about is redefining for loop <code>:=</code> semantics to remove the potential for accidental aliasing bugs. See “<a href="/blog/loopvar-preview">Fixing For Loops in Go 1.22</a>” for details, including instructions for previewing this change in Go 1.21.</p> <h2 id="thank-you">Thank You!</h2> <p>The Go project has always been far more than just us on the Go team at Google. Thank you to all our contributors and everyone in the Go community for making Go what it is today. We wish you all the best in the year ahead.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/survey2023-h2-results">Go Developer Survey 2023 H2 Results</a><br> <b>Previous article: </b><a href="/blog/type-inference">Everything You Always Wanted to Know About Type Inference - And a Little Bit More</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Everything You Always Wanted to Know About Type Inference - And a Little Bit Moretag:blog.golang.org,2013:blog.golang.org/type-inference2023-10-09T00:00:00+00:002023-10-09T00:00:00+00:00A description of how type inference for Go works. Based on the GopherCon 2023 talk with the same title. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/type-inference"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Everything You Always Wanted to Know About Type Inference - And a Little Bit More</h1> <p class="author"> Robert Griesemer<br> 9 October 2023 </p> <p>This is the blog version of my talk on type inference at GopherCon 2023 in San Diego, slightly expanded and edited for clarity.</p> <h2 id="what-is-type-inference">What is type inference?</h2> <p>Wikipedia defines type inference as follows:</p> <blockquote> <p>Type inference is the ability to automatically deduce, either partially or fully, the type of an expression at compile time. The compiler is often able to infer the type of a variable or the type signature of a function, without explicit type annotations having been given.</p> </blockquote> <p>The key phrase here is &ldquo;automatically deduce &hellip; the type of an expression&rdquo;. Go supported a basic form of type inference from the start:</p> <pre><code class="language-Go">const x = expr // the type of x is the type of expr var x = expr x := expr </code></pre> <p>No explicit types are given in these declarations, and therefore the types of the constant and variables <code>x</code> on the left of <code>=</code> and <code>:=</code> are the types of the respective initialization expressions, on the right. We say that the types are <em>inferred</em> from (the types of) their initialization expressions. With the introduction of generics in Go 1.18, Go&rsquo;s type inference abilities were significantly expanded.</p> <h3 id="why-type-inference">Why type inference?</h3> <p>In non-generic Go code, the effect of leaving away types is most pronounced in a short variable declaration. Such a declaration combines type inference and a little bit of syntactic sugar—the ability to leave away the <code>var</code> keyword—into one very compact statement. Consider the following map variable declaration:</p> <pre><code class="language-Go">var m map[string]int = map[string]int{} </code></pre> <p>vs</p> <pre><code class="language-Go">m := map[string]int{} </code></pre> <p>Omitting the type on the left of <code>:=</code> removes repetition and at the same time increases readability.</p> <p>Generic Go code has the potential to significantly increase the number of types appearing in code: without type inference, each generic function and type instantiation requires type arguments. Being able to omit them becomes even more important. Consider using the following two functions from the new <a href="https://pkg.go.dev/slices" rel="noreferrer" target="_blank">slices package</a>:</p> <pre><code class="language-Go">package slices func BinarySearch[S ~[]E, E cmp.Ordered](x S, target E) (int, bool) func Sort[S ~[]E, E cmp.Ordered](x S) </code></pre> <p>Without type inference, calling <code>BinarySearch</code> and <code>Sort</code> requires explicit type arguments:</p> <pre><code class="language-Go">type List []int var list List slices.Sort[List, int](list) index, found := slices.BinarySearch[List, int](list, 42) </code></pre> <p>We&rsquo;d rather not repeat <code>[List, int]</code> with each such generic function call. With type inference the code simplifies to:</p> <pre><code class="language-Go">type List []int var list List slices.Sort(list) index, found := slices.BinarySearch(list, 42) </code></pre> <p>This is both cleaner and more compact. In fact it looks exactly like non-generic code, and type inference makes this possible.</p> <p>Importantly, type inference is an optional mechanism: if type arguments make code clearer, by all means, write them down.</p> <h2 id="type-inference-is-a-form-of-type-pattern-matching">Type inference is a form of type pattern matching</h2> <p>Inference compares type patterns, where a type pattern is a type containing type parameters. For reasons that will become obvious in a bit, type parameters are sometimes also called <em>type variables</em>. Type pattern matching allows us to infer the types that need to go into these type variables. Let&rsquo;s consider a short example:</p> <pre><code class="language-Go">// From the slices package // func Sort[S ~[]E, E cmp.Ordered](x S) type List []int var list List slices.Sort(list) </code></pre> <p>The <code>Sort</code> function call passes the <code>list</code> variable as function argument for the parameter <code>x</code> of <a href="https://pkg.go.dev/slices#Sort" rel="noreferrer" target="_blank"><code>slices.Sort</code></a>. Therefore the type of <code>list</code>, which is <code>List</code>, must match the type of <code>x</code>, which is type parameter <code>S</code>. If <code>S</code> has the type <code>List</code>, this assignment becomes valid. In reality, the <a href="/ref/spec#Assignability">rules for assignments</a> are complicated, but for now it&rsquo;s good enough to assume that the types must be identical.</p> <p>Once we have inferred the type for <code>S</code>, we can look at the <a href="/ref/spec#Type_constraints">type constraint</a> for <code>S</code>. It says—because of the tilde <code>~</code> symbol—that the <a href="/ref/spec#Underlying_types"><em>underlying type</em></a> of <code>S</code> must be the slice <code>[]E</code>. The underlying type of <code>S</code> is <code>[]int</code>, therefore <code>[]int</code> must match <code>[]E</code>, and with that we can conclude that <code>E</code> must be <code>int</code>. We&rsquo;ve been able to find types for <code>S</code> and <code>E</code> such that corresponding types match. Inference has succeeded!</p> <p>Here&rsquo;s a more complicated scenario where we have a lot of type parameters: <code>S1</code>, <code>S2</code>, <code>E1</code>, and <code>E2</code> from <code>slices.EqualFunc</code>, and <code>E1</code> and <code>E2</code> from the generic function <code>equal</code>. The local function <code>foo</code> calls <code>slices.EqualFunc</code> with the <code>equal</code> function as an argument:</p> <pre><code class="language-Go">// From the slices package // func EqualFunc[S1 ~[]E1, S2 ~[]E2, E1, E2 any](s1 S1, s2 S2, eq func(E1, E2) bool) bool // Local code func equal[E1, E2 comparable](E1, E2) bool { … } func foo(list1 []int, list2 []float64) { … if slices.EqualFunc(list1, list2, equal) { … } … } </code></pre> <p>This is an example where type inference really shines as we can potentially leave away six type arguments, one for each of the type parameters. The type pattern matching approach still works, but we can see how it may get complicated quickly because the number of type relationships is proliferating. We need a systematic approach to determine which type parameters and which types get involved with which patterns.</p> <p>It helps to look at type inference in a slightly different way.</p> <h2 id="type-equations">Type equations</h2> <p>We can reframe type inference as a problem of solving type equations. Solving equations is something that we are all familiar with from high school algebra. Luckily, solving type equations is a simpler problem as we will see shortly.</p> <p>Let&rsquo;s look again at our earlier example:</p> <pre><code class="language-Go">// From the slices package // func Sort[S ~[]E, E cmp.Ordered](x S) type List []int var list List slices.Sort(list) </code></pre> <p>Inference succeeds if the type equations below can be solved. Here <code>≡</code> stands for <a href="/ref/spec#Type_identity"><em>is identical to</em></a>, and <code>under(S)</code> represents the <a href="/ref/spec#Underlying_types">underlying type</a> of <code>S</code>:</p> <pre><code>S ≡ List // find S such that S ≡ List is true under(S) ≡ []E // find E such that under(S) ≡ []E is true </code></pre> <p>The type parameters are the <em>variables</em> in the equations. Solving the equations means finding values (type arguments) for these variables (type parameters), such that the equations become true. This view makes the type inference problem more tractable because it gives us a formal framework that allows us to write down the information that flows into inference.</p> <h3 id="being-precise-with-type-relations">Being precise with type relations</h3> <p>Until now we have simply talked about types having to be <a href="/ref/spec#Type_identity">identical</a>. But for actual Go code that is too strong a requirement. In the previous example, <code>S</code> need not be identical to <code>List</code>, rather <code>List</code> must be <a href="/ref/spec#Assignability">assignable</a> to <code>S</code>. Similarly, <code>S</code> must <a href="/ref/spec#Satisfying_a_type_constraint">satisfy</a> its corresponding type constraint. We can formulate our type equations more precisely by using specific operators that we write as <code>:≡</code> and <code>∈</code>:</p> <pre><code>S :≡ List // List is assignable to S S ∈ ~[]E // S satisfies constraint ~[]E E ∈ cmp.Ordered // E satisfies constraint cmp.Ordered </code></pre> <p>Generally, we can say that type equations come in three forms: two types must be identical, one type must be assignable to the other type, or one type must satisfy a type constraint:</p> <pre><code>X ≡ Y // X and Y must be identical X :≡ Y // Y is assignable to X X ∈ Y // X satisfies constraint Y </code></pre> <p>(Note: In the GopherCon talk we used the symbols <code>≡</code><sub>A</sub> for <code>:≡</code> and <code>≡</code><sub>C</sub> for <code>∈</code>. We believe <code>:≡</code> more clearly evokes an assignment relation; and <code>∈</code> directly expresses that the type represented by a type parameter must be an element of its constraint&rsquo;s <a href="/ref/spec#Interface_types">type set</a>.)</p> <h3 id="sources-of-type-equations">Sources of type equations</h3> <p>In a generic function call we may have explicit type arguments, though most of the time we hope that they can be inferred. Typically we also have ordinary function arguments. Each explicit type argument contributes a (trivial) type equation: the type parameter must be identical to the type argument because the code says so. Each ordinary function argument contributes another type equation: the function argument must be assignable to its corresponding function parameter. And finally, each type constraint provides a type equation as well by constraining what types satisfy the constraint.</p> <p>Altogether, this produces <code>n</code> type parameters and <code>m</code> type equations. In contrast to basic high school algebra, <code>n</code> and <code>m</code> don&rsquo;t have to be the same for type equations to be solvable. For instance, the single equation below allows us to infer the type arguments for two type parameters:</p> <pre><code class="language-Go">map[K]V ≡ map[int]string // K ➞ int, V ➞ string (n = 2, m = 1) </code></pre> <p>Let&rsquo;s look at each of these sources of type equations in turn:</p> <h4 id="1-type-equations-from-type-arguments">1. Type equations from type arguments</h4> <p>For each type parameter declaration</p> <pre><code class="language-Go">func f[…, P constraint, …]… </code></pre> <p>and explicitly provided type argument</p> <pre><code class="language-Go">f[…, A, …]… </code></pre> <p>we get the type equation</p> <pre><code>P ≡ A </code></pre> <p>We can trivially solve this for <code>P</code>: <code>P</code> must be <code>A</code> and we write <code>P ➞ A</code>. In other words, there is nothing to do here. We could still write down the respective type equation for completeness, but in this case, the Go compiler simply substitutes the type arguments for their type parameters throughout and then those type parameters are gone and we can forget about them.</p> <h4 id="2-type-equations-from-assignments">2. Type equations from assignments</h4> <p>For each function argument <code>x</code> passed to a function parameter <code>p</code></p> <pre><code class="language-Go">f(…, x, …) </code></pre> <p>where <code>p</code> or <code>x</code> contain type parameters, the type of <code>x</code> must be assignable to the type of the parameter <code>p</code>. We can express this with the equation</p> <pre><code>𝑻(p) :≡ 𝑻(x) </code></pre> <p>where <code>𝑻(x)</code> means &ldquo;the type of <code>x</code>&rdquo;. If neither <code>p</code> nor <code>x</code> contains type parameters, there is no type variable to solve for: the equation is either true because the assignment is valid Go code, or false if the code is invalid. For this reason, type inference only considers types that contain type parameters of the involved function (or functions).</p> <p>Starting with Go 1.21, an uninstantiated or partially instantiated function (but not a function call) may also be assigned to a function-typed variable, as in:</p> <pre><code class="language-Go">// From the slices package // func Sort[S ~[]E, E cmp.Ordered](x S) var intSort func([]int) = slices.Sort </code></pre> <p>Analogous to parameter passing, such assignments lead to a corresponding type equation. For this example it would be</p> <pre><code>𝑻(intSort) :≡ 𝑻(slices.Sort) </code></pre> <p>or simplified</p> <pre><code>func([]int) :≡ func(S) </code></pre> <p>together with equations for the constraints for <code>S</code> and <code>E</code> from <code>slices.Sort</code> (see below).</p> <h4 id="3-type-equations-from-constraints">3. Type equations from constraints</h4> <p>Finally, for each type parameter <code>P</code> for which we want to infer a type argument, we can extract a type equation from its constraint because the type parameter must satisfy the constraint. Given the declaration</p> <pre><code class="language-Go">func f[…, P constraint, …]… </code></pre> <p>we can write down the equation</p> <pre><code>P ∈ constraint </code></pre> <p>Here, the <code>∈</code> means &ldquo;must satisfy constraint&rdquo; which is (almost) the same as being a type element of the constraint&rsquo;s type set. We will see later that some constraints (such as <code>any</code>) are not useful or currently cannot be used due to limitations of the implementation. Inference simply ignores the respective equations in those cases.</p> <h3 id="type-parameters-and-equations-may-be-from-multiple-functions">Type parameters and equations may be from multiple functions</h3> <p>In Go 1.18, inferred type parameters had to all be from the same function. Specifically, it was not possible to pass a generic, uninstantiated or partially instantiated function as a function argument, or assign it to a (function-typed) variable.</p> <p>As mentioned earlier, in Go 1.21 type inference also works in these cases. For instance, the generic function</p> <pre><code class="language-Go">func myEq[P comparable](x, y P) bool { return x == y } </code></pre> <p>can be assigned to a variable of function type</p> <pre><code class="language-Go">var strEq func(x, y string) bool = myEq // same as using myEq[string] </code></pre> <p>without <code>myEq</code> being fully instantiated, and type inference will infer that the type argument for <code>P</code> must be <code>string</code>.</p> <p>Furthermore, a generic function may be used uninstantiated or partially instantiated as an argument to another, possibly generic function:</p> <pre><code class="language-Go">// From the slices package // func CompactFunc[S ~[]E, E any](s S, eq func(E, E) bool) S type List []int var list List result := slices.CompactFunc(list, myEq) // same as using slices.CompactFunc[List, int](list, myEq[int]) </code></pre> <p>In this last example, type inference determines the type arguments for <code>CompactFunc</code> and <code>myEq</code>. More generally, type parameters from arbitrarily many functions may need to be inferred. With multiple functions involved, type equations may also be from or involve multiple functions. In the <code>CompactFunc</code> example we end up with three type parameters and five type equations:</p> <pre><code>Type parameters and constraints: S ~[]E E any P comparable Explicit type arguments: none Type equations: S :≡ List func(E, E) bool :≡ func(P, P) bool S ∈ ~[]E E ∈ any P ∈ comparable Solution: S ➞ List E ➞ int P ➞ int </code></pre> <h3 id="bound-vs-free-type-parameters">Bound vs free type parameters</h3> <p>At this point we have a clearer understanding of the various source of type equations, but we have not been very precise about which type parameters to solve the equations for. Let&rsquo;s consider another example. In the code below, the function body of <code>sortedPrint</code> calls <code>slices.Sort</code> for the sorting part. <code>sortedPrint</code> and <code>slices.Sort</code> are generic functions as both declare type parameters.</p> <pre><code class="language-Go">// From the slices package // func Sort[S ~[]E, E cmp.Ordered](x S) // sortedPrint prints the elements of the provided list in sorted order. func sortedPrint[F any](list []F) { slices.Sort(list) // 𝑻(list) is []F … // print list } </code></pre> <p>We want to infer the type argument for the <code>slices.Sort</code> call. Passing <code>list</code> to parameter <code>x</code> of <code>slices.Sort</code> gives rise to the equation</p> <pre><code>𝑻(x) :≡ 𝑻(list) </code></pre> <p>which is the same as</p> <pre><code>S :≡ []F </code></pre> <p>In this equation we have two type parameters, <code>S</code> and <code>F</code>. Which one do we need to solve the type equation for? Because the invoked function is <code>Sort</code>, we care about its type parameter <code>S</code>, not the type parameter <code>F</code>. We say that <code>S</code> is <em>bound</em> to <code>Sort</code> because it is declared by <code>Sort</code>. <code>S</code> is the relevant type variable in this equation. By contrast, <code>F</code> is bound to (declared by) <code>sortedPrint</code>. We say that <code>F</code> is <em>free</em> with respect to <code>Sort</code>. It has its own, already given type. That type is <code>F</code>, whatever that is (determined at instantiation time). In this equation, <code>F</code> is already given, it is a <em>type constant</em>.</p> <p>When solving type equations we always solve for the type parameters bound to the function we are calling (or assigning in case of a generic function assignment).</p> <h2 id="solving-type-equations">Solving type equations</h2> <p>The missing piece, now that we have established how to collect the relevant type parameters and type equations, is of course the algorithm that allows us to solve the equations. After the various examples, it probably has become obvious that solving <code>X ≡ Y</code> simply means comparing the types <code>X</code> and <code>Y</code> recursively against each other, and in the process determining suitable type arguments for type parameters that may occur in <code>X</code> and <code>Y</code>. The goal is to make the types <code>X</code> and <code>Y</code> <em>identical</em>. This matching process is called <a href="https://en.wikipedia.org/wiki/Unification_(computer_science)" rel="noreferrer" target="_blank"><em>unification</em></a>.</p> <p>The rules for <a href="/ref/spec#Type_identity">type identity</a> tell us how to compare types. Since <em>bound</em> type parameters play the role of type variables, we need to specify how they are matched against other types. The rules are as follows:</p> <ul> <li>If type parameter <code>P</code> has an inferred type, <code>P</code> stands for that type.</li> <li>If type parameter <code>P</code> doesn&rsquo;t have an inferred type and is matched against another type <code>T</code>, <code>P</code> is set to that type: <code>P ➞ T</code>. We say that the type <code>T</code> was inferred for <code>P</code>.</li> <li>If <code>P</code> matches against another type parameter <code>Q</code>, and neither <code>P</code> nor <code>Q</code> have an inferred type yet, <code>P</code> and <code>Q</code> are <em>unified</em>.</li> </ul> <p>Unification of two type parameters means that they are joined together such that going forward they both denote the same type parameter value: if one of <code>P</code> or <code>Q</code> is matched against a type <code>T</code>, both <code>P</code> and <code>Q</code> are set to <code>T</code> simultaneously (in general, any number of type parameters may be unified this way).</p> <p>Finally, if two types <code>X</code> and <code>Y</code> are different, the equation cannot be made true and solving it fails.</p> <h3 id="unifying-types-for-type-identity">Unifying types for type identity</h3> <p>A few concrete examples should make this algorithm clear. Consider two types <code>X</code> and <code>Y</code> containing three bound type parameters <code>A</code>, <code>B</code>, and <code>C</code>, all appearing in the type equation <code>X ≡ Y</code>. The goal is to the solve this equation for the type parameters; i.e., find suitable type arguments for them such that <code>X</code> and <code>Y</code> become identical and thus the equation becomes true.</p> <pre><code class="language-Go">X: map[A]struct{i int; s []B} Y: map[string]struct{i C; s []byte} </code></pre> <p>Unification proceeds by comparing the structure of <code>X</code> and <code>Y</code> recursively, starting at the top. Simply looking at the structure of the two types we have</p> <pre><code class="language-Go">map[…]… ≡ map[…]… </code></pre> <p>with the <code>…</code> representing the respective map key and value types that we&rsquo;re ignoring at this step. Since we have a map on both sides, the types are identical so far. Unification proceeds recursively, first with the key types which are <code>A</code> for the <code>X</code> map, and <code>string</code> for the <code>Y</code> map. Corresponding key types must be identical, and from that we can immediately infer that the type argument for <code>A</code> must be <code>string</code>:</p> <pre><code class="language-Go">A ≡ string =&gt; A ➞ string </code></pre> <p>Continuing with the map element types, we arrive at</p> <pre><code class="language-Go">struct{i int; s []B} ≡ struct{i C; s []byte} </code></pre> <p>Both sides are structs so unification proceeds with the struct fields. They are identical if they are in the same order, with the same names, and identical types. The first field pair is <code>i int</code> and <code>i C</code>. The names match and because <code>int</code> must unify with <code>C</code>, thus</p> <pre><code class="language-Go">int ≡ C =&gt; C ➞ int </code></pre> <p>This recursive type matching continues until the tree structure of the two types is fully traversed, or until a conflict appears. In this example, eventually we end up with</p> <pre><code class="language-Go">[]B ≡ []byte =&gt; B ≡ byte =&gt; B ➞ byte </code></pre> <p>Everything works out fine and unification infers the type arguments</p> <pre><code>A ➞ string B ➞ byte C ➞ int </code></pre> <h3 id="unifying-types-with-different-structures">Unifying types with different structures</h3> <p>Now, let&rsquo;s consider a slight variation of the previous example: here <code>X</code> and <code>Y</code> don&rsquo;t have the same type structure. When the type trees are compared recursively, unification still successfully infers the type argument for <code>A</code>. But the value types of the maps are different and unification fails.</p> <pre><code class="language-Go">X: map[A]struct{i int; s []B} Y: map[string]bool </code></pre> <p>Both <code>X</code> and <code>Y</code> are map types, so unification proceeds recursively as before, starting with the key types. We arrive at</p> <pre><code class="language-Go">A ≡ string =&gt; A ➞ string </code></pre> <p>also as before. But when we proceed with the map&rsquo;s value types we have</p> <pre><code class="language-Go">struct{…} ≡ bool </code></pre> <p>The <code>struct</code> type doesn&rsquo;t match <code>bool</code>; we have different types and unification (and thus type inference) fails.</p> <h3 id="unifying-types-with-conflicting-type-arguments">Unifying types with conflicting type arguments</h3> <p>Another kind of conflict appears when different types match against the same type parameter. Here we have again a version of our initial example but now the type parameter <code>A</code> appears twice in <code>X</code>, and <code>C</code> appears twice in <code>Y</code>.</p> <pre><code class="language-Go">X: map[A]struct{i int; s []A} Y: map[string]struct{i C; s []C} </code></pre> <p>The recursive type unification works out fine at first and we have the following pairings of type parameters and types:</p> <pre><code class="language-Go">A ≡ string =&gt; A ➞ string // map key type int ≡ C =&gt; C ➞ int // first struct field type </code></pre> <p>When we get to the second struct field type we have</p> <pre><code class="language-Go">[]A ≡ []C =&gt; A ≡ C </code></pre> <p>Since both <code>A</code> and <code>C</code> have a type argument inferred for them, they stand for those type arguments, which are <code>string</code> and <code>int</code> respectively. These are different types, so <code>A</code> and <code>C</code> can&rsquo;t possibly match. Unification and thus type inference fails.</p> <h3 id="other-type-relations">Other type relations</h3> <p>Unification solves type equations of the form <code>X ≡ Y</code> where the goal is <em>type identity</em>. But what about <code>X :≡ Y</code> or <code>X ∈ Y</code>?</p> <p>A couple of observations help us out here: The job of type inference is solely to find the types of omitted type arguments. Type inference is always followed by type or function <a href="/ref/spec#Instantiations">instantiation</a> which checks that each type argument actually satisfies its respective type constraint. Finally, in case of a generic function call, the compiler also checks that function arguments are assignable to their corresponding function parameters. All of these steps must succeed for the code to be valid.</p> <p>If type inference is not precise enough it may infer an (incorrect) type argument where no type may exist. If that is the case, either instantiation or argument passing will fail. Either way, the compiler will produce an error message. It&rsquo;s just that the error message may be slightly different.</p> <p>This insight allows us to play a bit loose with the type relations <code>:≡</code> and <code>∈</code>. Specifically, it allows us to simplify them such that they can be treated almost the same as <code>≡</code>. The goal of the simplifications is to extract as much type information as possible from a type equation, and thus to infer type arguments where a precise implementation may fail, because we can.</p> <h3 id="simplifying-x--y">Simplifying X :≡ Y</h3> <p>Go&rsquo;s assignability rules are pretty complicated, but most of the time we can actually get by with type identity, or a slight variation of it. As long as we find potential type arguments, we&rsquo;re happy, exactly because type inference is still followed by type instantiation and function invocation. If inference finds a type argument where it shouldn&rsquo;t, it&rsquo;ll be caught later. Thus, when matching for assignability, we make the following adjustments to the unfication algorithm:</p> <ul> <li>When a named (defined) type is matched against a type literal, their underlying types are compared instead.</li> <li>When comparing channel types, channel directions are ignored.</li> </ul> <p>Furthermore, the assignment direction is ignored: <code>X :≡ Y</code> is treated like <code>Y :≡ X</code>.</p> <p>These adjustments apply only at the top level of a type structure: for instance, per Go&rsquo;s <a href="/ref/spec#Assignability">assignability rules</a>, a named map type may be assigned to an unnamed map type, but the key and element types must still be identical. With these changes, unification for assignability becomes a (minor) variation of unification for type identity. The following example illustrates this.</p> <p>Let&rsquo;s assume we are passing a value of our earlier <code>List</code> type (defined as <code>type List []int</code>) to a function parameter of type <code>[]E</code> where <code>E</code> is a bound type parameter (i.e., <code>E</code> is declared by the generic function that is being called). This leads to the type equation <code>[]E :≡ List</code>. Attempting to unify these two types requires comparing <code>[]E</code> with <code>List</code> These two types are not identical, and without any changes to how unification works, it will fail. But because we are unifying for assignability, this initial match doesn&rsquo;t need to be exact. There&rsquo;s no harm in continuing with the underlying type of the named type <code>List</code>: in the worst case we may infer an incorrect type argument, but that will lead to an error later, when assignments are checked. In the best case, we find a useful and correct type argument. In our example, inexact unification succeeds and we correctly infer <code>int</code> for <code>E</code>.</p> <h3 id="simplifying-x--y-1">Simplifying X ∈ Y</h3> <p>Being able to simplify the constraint satisfaction relation is even more important as constraints can be very complex.</p> <p>Again, constraint satisfaction is checked at instantiation time, so the goal here is to help type inference where we can. These are typically situations where we know the structure of a type parameter; for instance we know that it must be a slice type and we care about the slice&rsquo;s element type. For example, a type parameter list of the form <code>[P ~[]E]</code> tells us that whatever <code>P</code> is, its underlying type must be of the form <code>[]E</code>. These are exactly the situations where the constraint has a <a href="/ref/spec#Core_types">core type</a>.</p> <p>Therefore, if we have an equation of the form</p> <pre><code>P ∈ constraint // or P ∈ ~constraint </code></pre> <p>and if <code>core(constraint)</code> (or <code>core(~constraint)</code>, respectively) exists, the equation can be simplified to</p> <pre><code>P ≡ core(constraint) under(P) ≡ core(~constraint) // respectively </code></pre> <p>In all other cases, type equations involving constraints are ignored.</p> <h3 id="expanding-inferred-types">Expanding inferred types</h3> <p>If unification is successful it produces a mapping from type parameters to inferred type arguments. But unification alone doesn&rsquo;t ensure that the inferred types are free of bound type parameters. To see why this is the case, consider the generic function <code>g</code> below which is invoked with a single argument <code>x</code> of type <code>int</code>:</p> <pre><code class="language-Go">func g[A any, B []C, C *A](x A) { … } var x int g(x) </code></pre> <p>The type constraint for <code>A</code> is <code>any</code> which doesn&rsquo;t have a core type, so we ignore it. The remaining type constraints have core types and they are <code>[]C</code> and <code>*A</code> respectively. Together with the argument passed to <code>g</code>, after minor simplifications, the type equations are:</p> <pre><code> A :≡ int B ≡ []C C ≡ *A </code></pre> <p>Since each equation pits a type parameter against a non-type parameter type, unification has little to do and immediately infers</p> <pre><code> A ➞ int B ➞ []C C ➞ *A </code></pre> <p>But that leaves the type parameters <code>A</code> and <code>C</code> in the inferred types, which is not helpful. Like in high school algebra, once an equation is solved for a variable <code>x</code>, we need to substitute <code>x</code> with its value throughout the remaining equations. In our example, in a first step, the <code>C</code> in <code>[]C</code> is substituted with the inferred type (the &ldquo;value&rdquo;) for <code>C</code>, which is <code>*A</code>, and we arrive at</p> <pre><code> A ➞ int B ➞ []*A // substituted *A for C C ➞ *A </code></pre> <p>In two more steps we replace the <code>A</code> in the inferred types <code>[]*A</code> and <code>*A</code> with the inferred type for <code>A</code>, which is <code>int</code>:</p> <pre><code> A ➞ int B ➞ []*int // substituted int for A C ➞ *int // substituted int for A </code></pre> <p>Only now inference is done. And like in high school algebra, sometimes this doesn&rsquo;t work. It&rsquo;s possible to arrive at a situation such as</p> <pre><code> X ➞ Y Y ➞ *X </code></pre> <p>After one round of substitutions we have</p> <pre><code> X ➞ *X </code></pre> <p>If we keep going, the inferred type for <code>X</code> keeps growing:</p> <pre><code> X ➞ **X // substituted *X for X X ➞ ***X // substituted *X for X etc. </code></pre> <p>Type inference detects such cycles during expansion and reports an error (and thus fails).</p> <h2 id="untyped-constants">Untyped constants</h2> <p>By now we have seen how type inference works by solving type equations with unification, followed by expansion of the result. But what if there are no types? What if the function arguments are untyped constants?</p> <p>Another example helps us shed light on this situation. Let&rsquo;s consider a function <code>foo</code> which takes an arbitrary number of arguments, all of which must have the same type. <code>foo</code> is called with a variety of untyped constant arguments, including a variable <code>x</code> of type <code>int</code>:</p> <pre><code class="language-Go">func foo[P any](...P) {} var x int foo(x) // P ➞ int, same as foo[int](x) foo(x, 2.0) // P ➞ int, 2.0 converts to int without loss of precision foo(x, 2.1) // P ➞ int, but parameter passing fails: 2.1 is not assignable to int </code></pre> <p>For type inference, typed arguments take precedence over untyped arguments. An untyped constant is considered for inference only if the type parameter it&rsquo;s assigned to doesn&rsquo;t have an inferred type yet. In these first three calls to <code>foo</code>, the variable <code>x</code> determines the inferred type for <code>P</code>: it&rsquo;s the type of <code>x</code> which is <code>int</code>. Untyped constants are ignored for type inference in this case and the calls behave exactly as if <code>foo</code> was explicitly instantiated with <code>int</code>.</p> <p>It gets more interesting if <code>foo</code> is called with untyped constant arguments only. In this case, type inference considers the <a href="/ref/spec#Constants">default types</a> of the untyped constants. As a quick reminder, here are the possible default types in Go:</p> <pre><code>Example Constant kind Default type Order true boolean constant bool 42 integer constant int earlier in list 'x' rune constant rune | 3.1416 floating-point constant float64 v -1i complex constant complex128 later in list &quot;gopher&quot; string constant string </code></pre> <p>With this information in hand, let&rsquo;s consider the function call</p> <pre><code class="language-Go">foo(1, 2) // P ➞ int (default type for 1 and 2) </code></pre> <p>The untyped constant arguments <code>1</code> and <code>2</code> are both integer constants, their default type is <code>int</code> and thus it&rsquo;s <code>int</code> that is inferred for the type parameter <code>P</code> of <code>foo</code>.</p> <p>If different constants—say untyped integer and floating-point constants—compete for the same type variable, we have different default types. Before Go 1.21, this was considered a conflict and led to an error:</p> <pre><code class="language-Go">foo(1, 2.0) // Go 1.20: inference error: default types int, float64 don't match </code></pre> <p>This behavior was not very ergonomic in use and also different from the behavior of untyped constants in expressions. For instance, Go permits the constant expression <code>1 + 2.0</code>; the result is the floating-point constant <code>3.0</code> with default type <code>float64</code>.</p> <p>In Go 1.21 the behavior was changed accordingly. Now, if multiple untyped numeric constants are matched against the same type parameter, the default type that appears later in the list of <code>int</code>, <code>rune</code>, <code>float64</code>, <code>complex</code> is selected, matching the rules for <a href="/ref/spec#Constant_expressions">constant expressions</a>:</p> <pre><code class="language-Go">foo(1, 2.0) // Go 1.21: P ➞ float64 (larger default type of 1 and 2.0; behavior like in 1 + 2.0) </code></pre> <h2 id="special-situations">Special situations</h2> <p>By now we&rsquo;ve got the big picture about type inference. But there are a couple of important special situations that deserve some attention.</p> <h3 id="parameter-order-dependencies">Parameter order dependencies</h3> <p>The first one has to do with parameter order dependencies. An important property we want from type inference is that the same types are inferred irrespective of the order of the function parameters (and corresponding argument order in each call of that function).</p> <p>Let&rsquo;s reconsider our variadic <code>foo</code> function: the type inferred for <code>P</code> should be the same irrespective of the order in which we pass the arguments <code>s</code> and <code>t</code> (<a href="/play/p/sOlWutKnDFc">playground</a>).</p> <pre><code class="language-Go">func foo[P any](...P) (x P) {} type T struct{} func main() { var s struct{} var t T fmt.Printf(&quot;%T\n&quot;, foo(s, t)) fmt.Printf(&quot;%T\n&quot;, foo(t, s)) // expect same result independent of parameter order } </code></pre> <p>From the calls to <code>foo</code> we can extract the relevant type equations:</p> <pre><code>𝑻(x) :≡ 𝑻(s) =&gt; P :≡ struct{} // equation 1 𝑻(x) :≡ 𝑻(t) =&gt; P :≡ T // equation 2 </code></pre> <p>Sadly, the simplified implementation for <code>:≡</code> produces an order dependency:</p> <p>If unification starts with equation 1, it matches <code>P</code> against <code>struct</code>; <code>P</code> doesn&rsquo;t have a type inferred for it yet and thus unification infers <code>P ➞ struct{}</code>. When unification sees type <code>T</code> later in equation 2, it proceeds with the underlying type of <code>T</code> which is <code>struct{}</code>, <code>P</code> and <code>under(T)</code> unify, and unification and thus inference succeeds.</p> <p>Vice versa, if unification starts with equation 2, it matches <code>P</code> against <code>T</code>; <code>P</code> doesn&rsquo;t have a type inferred for it yet and thus unification infers <code>P ➞ T</code>. When unification sees <code>struct{}</code> later in equation 1, it proceeds with the underlying type of the type <code>T</code> inferred for <code>P</code>. That underlying type is <code>struct{}</code>, which matches <code>struct</code> in equation 1, and unification and thus inference succeeds.</p> <p>As a consequence, depending on the order in which unification solves the two type equations, the inferred type is either <code>struct{}</code> or <code>T</code>. This is of course unsatisfying: a program may suddenly stop compiling simply because arguments may have been shuffled around during a code refactoring or cleanup.</p> <h3 id="restoring-order-independence">Restoring order independence</h3> <p>Luckily, the remedy is fairly simple. All we need is a small correction in some situations.</p> <p>Specifically, if unification is solving <code>P :≡ T</code> and</p> <ul> <li><code>P</code> is a type parameter which already has inferred a type <code>A</code>: <code>P ➞ A</code></li> <li><code>A :≡ T</code> is true</li> <li><code>T</code> is a named type</li> </ul> <p>then set the inferred type for <code>P</code> to <code>T</code>: <code>P ➞ T</code></p> <p>This ensures that <code>P</code> is the named type if there is choice, no matter at which point the named type appeared in a match against <code>P</code> (i.e., no matter in which order the type equations are solved). Note that if different named types match against the same type parameter, we always have a unfication failure because different named types are not identical by definition.</p> <p>Because we made similar simplifications for channels and interfaces, they also need similar special handling. For instance, we ignore channel directions when unifying for assignability and as a result may infer a directed or bidirectional channel depending on argument order. Similar problems occur with interfaces. We&rsquo;re not going to discuss these here.</p> <p>Going back to our example, if unification starts with equation 1, it infers <code>P ➞ struct{}</code> as before. When it proceeds with equation 2, as before, unification succeeds, but now we have exactly the condition that calls for a correction: <code>P</code> is a type parameter which already has a type (<code>struct{}</code>), <code>struct{}</code>, <code>struct{} :≡ T</code> is true (because <code>struct{} ≡ under(T)</code> is true), and <code>T</code> is a named type. Thus, unification makes the correction and sets <code>P ➞ T</code>. As a result, irrespective of the unification order, the result is the same (<code>T</code>) in both cases.</p> <h3 id="self-recursive-functions">Self-recursive functions</h3> <p>Another scenario that causes problems in a naive implementation of inference is self-recursive functions. Let&rsquo;s consider a generic factorial function <code>fact</code>, defined such that it also works for floating-point arguments (<a href="/play/p/s3wXpgHX6HQ">playground</a>). Note that this is not a mathematically correct implementation of the <a href="https://en.wikipedia.org/wiki/Gamma_function" rel="noreferrer" target="_blank">gamma function</a>, it is simply a convenient example.</p> <pre><code class="language-Go">func fact[P ~int | ~float64](n P) P { if n &lt;= 1 { return 1 } return fact(n-1) * n } </code></pre> <p>The point here is not the factorial function but rather that <code>fact</code> calls itself with the argument <code>n-1</code> which is of the same type <code>P</code> as the incoming parameter <code>n</code>. In this call, the type parameter <code>P</code> is simultaneously a bound and a free type parameter: it is bound because it is declared by <code>fact</code>, the function that we are calling recursively. But it is also free because it is declared by the function enclosing the call, which happens to also be <code>fact</code>.</p> <p>The equation resulting from passing the argument <code>n-1</code> to parameter <code>n</code> pits <code>P</code> against itself:</p> <pre><code>𝑻(n) :≡ 𝑻(n-1) =&gt; P :≡ P </code></pre> <p>Unification sees the same <code>P</code> on either side of the equation. Unification succeeds since both types are identical but there&rsquo;s no information gained and <code>P</code> remains without an inferred type. As a consequence, type inference fails.</p> <p>Luckily, the trick to address this is simple: Before type inference is invoked, and for (temporary) use by type inference only, the compiler renames the type parameters in the signatures (but not the bodies) of all functions involved in the respective call. This doesn&rsquo;t change the meaning of the function signatures: they denote the same generic functions irrespective of what the names of the type parameters are.</p> <p>For the purpose of this example, let&rsquo;s assume the <code>P</code> in the signature of <code>fact</code> got renamed to <code>Q</code>. The effect is as if the recursive call was done indirectly through a <code>helper</code> function (<a href="/play/p/TLpo-0auWwC">playground</a>):</p> <pre><code class="language-Go">func fact[P ~int | ~float64](n P) P { if n &lt;= 1 { return 1 } return helper(n-1) * n } func helper[Q ~int | ~float64](n Q) Q { return fact(n) } </code></pre> <p>With the renaming, or with the <code>helper</code> function, the equation resulting from passing <code>n-1</code> to the recursive call of <code>fact</code> (or the <code>helper</code> function, respectively) changes to</p> <pre><code>𝑻(n) :≡ 𝑻(n-1) =&gt; Q :≡ P </code></pre> <p>This equation has two type parameters: the bound type parameter <code>Q</code>, declared by the function that is being called, and the free type parameter <code>P</code>, declared by the enclosing function. This type equation is trivially solved for <code>Q</code> and results in the inference <code>Q ➞ P</code> which is of course what we&rsquo;d expect, and which we can verify by explicitly instantiating the recursive call (<a href="/play/p/zkUFvwJ54lC">playground</a>):</p> <pre><code class="language-Go">func fact[P ~int | ~float64](n P) P { if n &lt;= 1 { return 1 } return fact[P](n-1) * n } </code></pre> <h2 id="whats-missing">What&rsquo;s missing?</h2> <p>Conspicuously absent from our description is type inference for generic types: currently generic types must always be explicitly instantiated.</p> <p>There are a couple of reasons for this. First of all, for type instantiation, type inference only has type arguments to work with; there are no other arguments as is the case for function calls. As a consequence, at least one type argument must always be provided (except for pathological cases where type constraints prescribe exactly one possible type argument for all type parameters). Thus, type inference for types is only useful to complete a partially instantiated type where all the omitted type arguments can be inferred from the equations resulting from type constraints; i.e., where there are at least two type parameters. We believe this is not a very common scenario.</p> <p>Second, and more pertinent, type parameters allow an entirely new kind of recursive types. Consider the hypothetical type</p> <pre><code class="language-Go">type T[P T[P]] interface{ … } </code></pre> <p>where the constraint for <code>P</code> is the type being declared. Combined with the ablity to have multiple type parameters that may refer to each other in complex recursive fashion, type inference becomes much more complicated and we don&rsquo;t fully understand all the implications of that at the moment. That said, we believe it shouldn&rsquo;t be too hard to detect cycles and proceed with type inference where no such cycles exist.</p> <p>Finally, there are situations where type inference is simply not strong enough to make an inference, typically because unification works with certain simplifying assumptions such as the ones described earlier in this post. The primary example here is constraints which have no core type, but where a more sophisticated approach might be able to infer type information anyway.</p> <p>These are all areas where we may see incremental improvements in future Go releases. Importantly, we believe that cases where inference currently fails are either rare or unimportant in production code, and that our current implementation covers a large majority of all useful code scenarios.</p> <p>That said, if you run into a situation where you believe type inference should work or went astray, please <a href="/issue/new">file an issue</a>! As always, the Go team loves to hear from you, especially when it helps us making Go even better.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/14years">Fourteen Years of Go</a><br> <b>Previous article: </b><a href="/blog/deconstructing-type-parameters">Deconstructing Type Parameters</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script> Deconstructing Type Parameterstag:blog.golang.org,2013:blog.golang.org/deconstructing-type-parameters2023-09-26T00:00:00+00:002023-09-26T00:00:00+00:00Why the function signatures in the slices packages are so complicated. <div id="blog"><div id="content"> <div id="content"> <div class="Article" data-slug="/blog/deconstructing-type-parameters"> <h1 class="small"><a href="/blog/">The Go Blog</a></h1> <h1>Deconstructing Type Parameters</h1> <p class="author"> Ian Lance Taylor<br> 26 September 2023 </p> <h2 id="slices-package-function-signatures">slices package function signatures</h2> <p>The <a href="https://pkg.go.dev/slices#Clone" rel="noreferrer" target="_blank"><code>slices.Clone</code></a> function is pretty simple: it makes a copy of a slice of any type.</p> <pre><code class="language-Go">func Clone[S ~[]E, E any](s S) S { return append(s[:0:0], s...) } </code></pre> <p>This works because appending to a slice with zero capacity will allocate a new backing array. The function body winds up being shorter than the function signature, which is in part because the body is short, but also because the signature is long. In this blog post we&rsquo;ll explain why the signature is written the way that it is.</p> <h2 id="simple-clone">Simple Clone</h2> <p>We&rsquo;ll start by writing a simple generic <code>Clone</code> function. This is not the one in the <code>slices</code> package. We want to take a slice of any element type, and return a new slice.</p> <pre><code class="language-Go">func Clone1[E any](s []E) []E { // body omitted } </code></pre> <p>The generic function <code>Clone1</code> has a single type parameter <code>E</code>. It takes a single argument <code>s</code> which is a slice of type <code>E</code>, and it returns a slice of the same type. This signature is straightforward for anybody familiar with generics in Go.</p> <p>However, there is a problem. Named slice types are not common in Go, but people do use them.</p> <pre><code class="language-Go">// MySlice is a slice of strings with a special String method. type MySlice []string // String returns the printable version of a MySlice value. func (s MySlice) String() string { return strings.Join(s, &quot;+&quot;) } </code></pre> <p>Let&rsquo;s say that we want to make a copy of a <code>MySlice</code> and then get the printable version, but with the strings in sorted order.</p> <pre><code class="language-Go">func PrintSorted(ms MySlice) string { c := Clone1(ms) slices.Sort(c) return c.String() // FAILS TO COMPILE } </code></pre> <p>Unfortunately, this doesn&rsquo;t work. The compiler reports an error:</p> <pre><code>c.String undefined (type []string has no field or method String) </code></pre> <p>We can see the problem if we manually instantiate <code>Clone1</code> by replacing the type parameter with the type argument.</p> <pre><code class="language-Go">func InstantiatedClone1(s []string) []string </code></pre> <p>The <a href="/ref/spec#Assignability">Go assignment rules</a> allow us to pass a value of type <code>MySlice</code> to a parameter of type <code>[]string</code>, so calling <code>Clone1</code> is fine. But <code>Clone1</code> will return a value of type <code>[]string</code>, not a value of type <code>MySlice</code>. The type <code>[]string</code> doesn&rsquo;t have a <code>String</code> method, so the compiler reports an error.</p> <h2 id="flexible-clone">Flexible Clone</h2> <p>To fix this problem, we have to write a version of <code>Clone</code> that returns the same type as its argument. If we can do that, then when we call <code>Clone</code> with a value of type <code>MySlice</code>, it will return a result of type <code>MySlice</code>.</p> <p>We know that it has to look something like this.</p> <pre><code class="language-Go">func Clone2[S ?](s S) S // INVALID </code></pre> <p>This <code>Clone2</code> function returns a value that is the same type as its argument.</p> <p>Here I&rsquo;ve written the constraint as <code>?</code>, but that&rsquo;s just a placeholder. To make this work we need to write a constraint that will let us write the body of the function. For <code>Clone1</code> we could just use a constraint of <code>any</code> for the element type. For <code>Clone2</code> that won&rsquo;t work: we want to require that <code>s</code> be a slice type.</p> <p>Since we know we want a slice, the constraint of <code>S</code> has to be a slice. We don&rsquo;t care what the slice element type is, so let&rsquo;s just call it <code>E</code>, as we did with <code>Clone1</code>.</p> <pre><code class="language-Go">func Clone3[S []E](s S) S // INVALID </code></pre> <p>This is still invalid, because we haven&rsquo;t declared <code>E</code>. The type argument for <code>E</code> can be any type, which means it also has to be a type parameter itself. Since it can be any type, its constraint is <code>any</code>.</p> <pre><code class="language-Go">func Clone4[S []E, E any](s S) S </code></pre> <p>This is getting close, and at least it will compile, but we&rsquo;re not quite there yet. If we compile this version, we get an error when we call <code>Clone4(ms)</code>.</p> <pre><code>MySlice does not satisfy []string (possibly missing ~ for []string in []string) </code></pre> <p>The compiler is telling us that we can&rsquo;t use the type argument <code>MySlice</code> for the type parameter <code>S</code>, because <code>MySlice</code> does not satisfy the constraint <code>[]E</code>. That&rsquo;s because <code>[]E</code> as a constraint only permits a slice type literal, like <code>[]string</code>. It doesn&rsquo;t permit a named type like <code>MySlice</code>.</p> <h2 id="underlying-type-constraints">Underlying type constraints</h2> <p>As the error message hints, the answer is to add a <code>~</code>.</p> <pre><code class="language-Go">func Clone5[S ~[]E, E any](s S) S </code></pre> <p>To repeat, writing type parameters and constraints <code>[S []E, E any]</code> means that the type argument for <code>S</code> can be any unnamed slice type, but it can&rsquo;t be a named type defined as a slice literal. Writing <code>[S ~[]E, E any]</code>, with a <code>~</code>, means that the type argument for <code>S</code> can be any type whose underlying type is a slice type.</p> <p>For any named type <code>type T1 T2</code> the underlying type of <code>T1</code> is the underlying type of <code>T2</code>. The underlying type of a predeclared type like <code>int</code> or a type literal like <code>[]string</code> is just the type itself. For the exact details, <a href="/ref/spec#Underlying_types">see the language spec</a>. In our example, the underlying type of <code>MySlice</code> is <code>[]string</code>.</p> <p>Since the underlying type of <code>MySlice</code> is a slice, we can pass an argument of type <code>MySlice</code> to <code>Clone5</code>. As you may have noticed, the signature of <code>Clone5</code> is the same as the signature of <code>slices.Clone</code>. We&rsquo;ve finally gotten to where we want to be.</p> <p>Before we move on, let&rsquo;s discuss why the Go syntax requires a <code>~</code>. It might seem that we would always want to permit passing <code>MySlice</code>, so why not make that the default? Or, if we need to support exact matching, why not flip things around, so that a constraint of <code>[]E</code> permits a named type while a constraint of, say, <code>=[]E</code>, only permits slice type literals?</p> <p>To explain this, let&rsquo;s first observe that a type parameter list like <code>[T ~MySlice]</code> doesn&rsquo;t make sense. That&rsquo;s because <code>MySlice</code> is not the underlying type of any other type. For instance, if we have a definition like <code>type MySlice2 MySlice</code>, the underlying type of <code>MySlice2</code> is <code>[]string</code>, not <code>MySlice</code>. So either <code>[T ~MySlice]</code> would permit no types at all, or it would be the same as <code>[T MySlice]</code> and only match <code>MySlice</code>. Either way, <code>[T ~MySlice]</code> isn&rsquo;t useful. To avoid this confusion, the language prohibits <code>[T ~MySlice]</code>, and the compiler produces an error like</p> <pre><code>invalid use of ~ (underlying type of MySlice is []string) </code></pre> <p>If Go didn&rsquo;t require the tilde, so that <code>[S []E]</code> would match any type whose underlying type is <code>[]E</code>, then we would have to define the meaning of <code>[S MySlice]</code>.</p> <p>We could prohibit <code>[S MySlice]</code>, or we could say that <code>[S MySlice]</code> only matches <code>MySlice</code>, but either approach runs into trouble with predeclared types. A predeclared type, like <code>int</code> is its own underlying type. We want to permit people to be able to write constraints that accept any type argument whose underlying type is <code>int</code>. In the language today, they can do that by writing <code>[T ~int]</code>. If we don&rsquo;t require the tilde we would still need a way to say &ldquo;any type whose underlying type is <code>int</code>&rdquo;. The natural way to say that would be <code>[T int]</code>. That would mean that <code>[T MySlice]</code> and <code>[T int]</code> would behave differently, although they look very similar.</p> <p>We could perhaps say that <code>[S MySlice]</code> matches any type whose underlying type is the underlying type of <code>MySlice</code>, but that makes <code>[S MySlice]</code> unnecessary and confusing.</p> <p>We think it&rsquo;s better to require the <code>~</code> and be very clear about when we are matching the underlying type rather than the type itself.</p> <h2 id="type-inference">Type inference</h2> <p>Now that we&rsquo;ve explained the signature of <code>slices.Clone</code>, let&rsquo;s see how actually using <code>slices.Clone</code> is simplified by type inference. Remember, the signature of <code>Clone</code> is</p> <pre><code class="language-Go">func Clone[S ~[]E, E any](s S) S </code></pre> <p>A call of <code>slices.Clone</code> will pass a slice to the parameter <code>s</code>. Simple type inference will let the compiler infer that the type argument for the type parameter <code>S</code> is the type of the slice being passed to <code>Clone</code>. Type inference is then powerful enough to see that the type argument for <code>E</code> is the element type of the type argument passed to <code>S</code>.</p> <p>This means that we can write</p> <pre><code class="language-Go"> c := Clone(ms) </code></pre> <p>without having to write</p> <pre><code class="language-Go"> c := Clone[MySlice, string](ms) </code></pre> <p>If we refer to <code>Clone</code> without calling it, we do have to specify a type argument for <code>S</code>, as the compiler has nothing it can use to infer it. Fortunately, in that case, type inference is able to infer the type argument for <code>E</code> from the argument for <code>S</code>, and we don&rsquo;t have to specify it separately.</p> <p>That is, we can write</p> <pre><code class="language-Go"> myClone := Clone[MySlice] </code></pre> <p>without having to write</p> <pre><code class="language-Go"> myClone := Clone[MySlice, string] </code></pre> <h2 id="deconstructing-type-parameters">Deconstructing type parameters</h2> <p>The general technique we&rsquo;ve used here, in which we define one type parameter <code>S</code> using another type parameter <code>E</code>, is a way to deconstruct types in generic function signatures. By deconstructing a type, we can name, and constrain, all aspects of the type.</p> <p>For example, here is the signature for <code>maps.Clone</code>.</p> <pre><code class="language-Go">func Clone[M ~map[K]V, K comparable, V any](m M) M </code></pre> <p>Just as with <code>slices.Clone</code>, we use a type parameter for the type of the parameter <code>m</code>, and then deconstruct the type using two other type parameters <code>K</code> and <code>V</code>.</p> <p>In <code>maps.Clone</code> we constrain <code>K</code> to be comparable, as is required for a map key type. We can constrain the component types any way we like.</p> <pre><code class="language-Go">func WithStrings[S ~[]E, E interface { String() string }](s S) (S, []string) </code></pre> <p>This says that the argument of <code>WithStrings</code> must be a slice type for which the element type has a <code>String</code> method.</p> <p>Since all Go types can be built up from component types, we can always use type parameters to deconstruct those types and constrain them as we like.</p> </div> <div class="Article prevnext"> <p> <b>Next article: </b><a href="/blog/type-inference">Everything You Always Wanted to Know About Type Inference - And a Little Bit More</a><br> <b>Previous article: </b><a href="/blog/loopvar-preview">Fixing For Loops in Go 1.22</a><br> <b><a href="/blog/all">Blog Index</a></b> </div> </div> </div> <script src="/js/jquery.js"></script> <script src="/js/playground.js"></script> <script src="/js/play.js"></script> <script src="/js/godocs.js"></script>