Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: add UnwrapErr field and Unwrap method to *DNSError #63116

Closed
mateusz834 opened this issue Sep 20, 2023 · 32 comments
Closed

net: add UnwrapErr field and Unwrap method to *DNSError #63116

mateusz834 opened this issue Sep 20, 2023 · 32 comments

Comments

@mateusz834
Copy link
Member

mateusz834 commented Sep 20, 2023

Edit: current proposal: #63116 (comment)


This is a proposal to fix the #63109 issue.

I propose adding a causeErr field to DNSError and an Unwrap method that returns the causeErr.

// DNSError represents a DNS lookup error.
type DNSError struct {
    causeErr error // returned by the Unwrap method.
    // (....)     
}

func (e *DNSError) Unwrap() error { return e.causeErr }

This way we keep backwards compatibility for the Error method, we always construct the error from
the Err string field (we keep the current implementation). In the net pacakge we will then make Err field to be the result of the first call to causeErr.Error().

@bcmills
Copy link
Contributor

bcmills commented Sep 20, 2023

  • Should we do the same for AddrError? (It also has an Err string field.)
  • All fields of DNSError and AddrError are currently exported, so it is possible for packages outside of net to construct one of these errors directly. Should the new field be exported too?

@mateusz834
Copy link
Member Author

mateusz834 commented Sep 20, 2023

Should we do the same for AddrError? (It also has an Err string field.)

AFAIK the AddrError seems to be a standalone error (i.e. it doesn't seem to wrap anything).

All fields of DNSError and AddrError are currently exported, so it is possible for packages outside of net to construct one of these errors directly. Should the new field be exported too?

I don't know whether we want to allow this (we would have to enforce somehow that DNSError.causeErr.Error() == DNSError.Err). Maybe we should leave this for future consideration? We can export it afterwards.

There is still an issue (with the current proposal), that someone might change the Err field without updating the causeErr. (with a *DNSError returned from Lookup functions).

I am unsure about this (in either way).

@ianlancetaylor
Copy link
Contributor

Since all the existing fields of DNSError are exported, I think the new one should also be exported. I suggested Underlying.

@ianlancetaylor
Copy link
Contributor

Here is a test for #63109 that I wrote last night.

func TestCancelBeforeDial(t *testing.T) {
	ln := newLocalListener(t, "tcp")
	defer ln.Close()
	_, port, err := SplitHostPort(ln.Addr().String())
	if err != nil {
		t.Fatal(err)
	}

	ctx, cancel := context.WithCancel(context.Background())
	cancel()
	var d Dialer
	_, err = d.DialContext(ctx, "tcp", JoinHostPort("localhost", port))
	if err == nil {
		t.Error("DialContext with canceled context did not fail")
	} else if !errors.Is(err, context.Canceled) {
		t.Errorf("error.Is(%v, context.Canceled) failed", err)
	}
}

@mateusz834
Copy link
Member Author

Fine, let's make the error field exported.

The proposal looks like this now:

// DNSError represents a DNS lookup error.
type DNSError struct {
	// Underlying is an error that caused this DNSError.
	// It is returned by the Unwrap method.
	//
	// When updating this field make sure to update
	// the Err field accordingly, so that the Error
	// and Unwrap methods don't diverge.
	Underlying error

	// Err is a description of the error, for backwards compatibility
	// the net package sets this error to the value
	// returned by Underlying.Error(), it used by
	// the Error method to produce the error string.
	Err string

	// (...)
}

// Unwrap returns e.Underlying.
func (e *DNSError) Unwrap() error { return e.Underlying }

@mjl-
Copy link

mjl- commented Sep 24, 2023

Having an exported error may also come in handy with possible future support for "extended dns errors", see https://datatracker.ietf.org/doc/html/rfc8914. Could be good to take that into account. It is useful to learn about specifics of dns failures, especially with dnssec errors, and for getting a more descriptive message than "servfail".

@mateusz834
Copy link
Member Author

@mjl- Great idea! I made #63192 for this. This should allow for detailed error messages from the pure go resolver. Not sure how the exported error helps with this. I assume you mean an exported (new) error type just for extended dns errors, I don't think we will ever do this it would be impossible to support this on windows and on unix cgo resolver.

@mjl-
Copy link

mjl- commented Sep 24, 2023 via email

@mateusz834
Copy link
Member Author

mateusz834 commented Oct 3, 2023

I implemented this change in CL 532217, it also simplifies the code a bit with an internal constructor for the *DNSError, so that we don't have to set the Err, IsNotFound, IsTimeout, IsTemporary fields manually each time we construct the error, these fields will be now determined based on the Underlying error type.

Edit: this also implies that the net package will always construct *DNSError with Underlying != nil even for unexported internal errors from the net package, this way we don't overcomplicate this change.

@gopherbot
Copy link

Change https://go.dev/cl/532217 mentions this issue: net: add Unwrap to *DNSError

@mateusz834
Copy link
Member Author

@rsc Any chance of getting this through proposal process? This is a bug-fix that needs a proposal. It would be nice to fix this for go 1.23. Thanks

@ianlancetaylor ianlancetaylor changed the title proposal: net: add Unwrap method to *DNSError proposal: net: add Underlying field to *DNSError Feb 8, 2024
@ianlancetaylor ianlancetaylor changed the title proposal: net: add Underlying field to *DNSError proposal: net: add Underlying field and Unwrap method to *DNSError Feb 8, 2024
@rsc
Copy link
Contributor

rsc commented Feb 28, 2024

I thought when we added Unwrap we explicitly looked at all types with an Err field and decided which to do and not do. For net.AddrError, we explicitly decided against it.

For net.DNSError, there is no Err field currently, so we could add one. Except there is an 'Err string' field.

There are lots of times when errors should not be wrapped. I don't see a discussion here of which errors to expose with Unwrap and why that is correct.

@mateusz834
Copy link
Member Author

There are lots of times when errors should not be wrapped. I don't see a discussion here of which errors to expose with Unwrap and why that is correct.

What kind of errors shouldn't be wrapped? I think that we should wrap all errors and just for backwards-compatibility set the Err field to Underlying.Error(), as in https://go.dev/cl/532217. I can think of one case that we don't strictly speaking have to wrap errNoSuchHost, because we have the IsNotFound field, but considering that errNoSuchHost is not exported i don't see a reason not to wrap it.

@neild
Copy link
Contributor

neild commented Feb 28, 2024

Surveying existing *DNSError errors, the Err field is:

  • A constant string, such as "unrecognized address". In this case there is no error to wrap.
  • The string value of an unexported error value, such as errNoSuchHost. Wrapping the unexported error value seems harmless.
  • The string value of a context error (cancelation or timeout). Wrapping seems like a strict improvement.
  • The string value of an error returned by a syscall. This is the the most ambiguous case, but I think wrapping is somewhere between useful and harmless here.

So I support adding DNSError.Underlying and wrapping everything.

(It's been a while, but I believe the only reason we didn't add an Unwrap method to DNSError back when we added Unwrap is because it has an Err string field, and we were limiting ourselves to adding Unwrap methods, not adding new error fields.)

@rsc
Copy link
Contributor

rsc commented Mar 1, 2024

I disagree about at least the syscall return. I think wrapping is harmful, because it exposes the underlying implementation in ways that are easier to depend on. Most syscall errors are also highly non-specific. I can see wanting Unwrap to work for the context error. I think that if we proceed, we should default to Underlying==nil and Err != "" and only set Underlying for the specific cases that we want Unwrap to work for (like context).

I'm also not convinced about Underlying; perhaps it should be UnwrapErr.

@rsc
Copy link
Contributor

rsc commented Mar 1, 2024

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@mateusz834
Copy link
Member Author

I'm also not convinced about Underlying; perhaps it should be UnwrapErr.

Underlying sounds better to me

@rsc
Copy link
Contributor

rsc commented Mar 6, 2024

Underlying is a concept in go/types; it is never used for errors in API names.
UnwrapErr is clunky but clear: it is the error that Unwrap returns, which may or may not be nil.

@rsc
Copy link
Contributor

rsc commented Mar 8, 2024

Have all remaining concerns about this proposal been addressed?

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.

@rsc
Copy link
Contributor

rsc commented Mar 15, 2024

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.

@mateusz834 mateusz834 changed the title proposal: net: add Underlying field and Unwrap method to *DNSError proposal: net: add UnwrapErr field and Unwrap method to *DNSError Mar 15, 2024
@mateusz834
Copy link
Member Author

FYI, currently the net package does not return directly errors from the context package, but it maps them with mapErr.

Err: mapErr(ctx.Err()).Error(),

go/src/net/net.go

Lines 427 to 438 in c9ed561

// mapErr maps from the context errors to the historical internal net
// error values.
func mapErr(err error) error {
switch err {
case context.Canceled:
return errCanceled
case context.DeadlineExceeded:
return errTimeout
default:
return err
}
}

Do we want to keep it this way?

@ianlancetaylor
Copy link
Contributor

We should keep it that for now. Changing it seems likely to break existing code (which is we do the mapping in the first place). If somebody wants to investigate changing this behavior, that can be a separate proposal.

@mateusz834
Copy link
Member Author

mateusz834 commented Mar 16, 2024

How do we want to handle errors from Resolver.Dial and from the returned net.Conn?

Should we map the context errors, with mapErr? I think it is harmful, because we will not be able to map wrapped errors.
Should we only return errors that match: errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded), without mapping?

c, err := r.dial(ctx, network, server)
if err != nil {
return dnsmessage.Parser{}, dnsmessage.Header{}, err
}

We might also do something like:

if errors.Is(err, context.Canceled) {
    return &DnsError{
        Err: err.Error(),
        UnwrapErr: errCanceled, // same error as returned by mapErr
    }
}

I think that if we do not expose all errors, then the third approach is the best.

@ianlancetaylor
Copy link
Contributor

Sorry, I'm not sure exactly what you are asking.

If we returned a mapped error today, we should continue to do that.

Note that the mapped errors already return true for things like errors.Is(err, context.Canceled) or errors.Is(err, context.DeadlineExceeded.

@mateusz834
Copy link
Member Author

We currently do not map errors returned by Resolver.Dial:

c, err := r.dial(ctx, network, server)
if err != nil {
return dnsmessage.Parser{}, dnsmessage.Header{}, err
}

How we should handle errors returned from the Dial function?

Do we want to return the error as is.

c, err := r.dial(ctx, network, server) 
if err != nil { 
    return &DnsError{
        Err: err.Error(),
        UnwrapErr: err,
    } 
} 

Return the error, but only when it matches context.Canceled or context.DeadlineExceeded

c, err := r.dial(ctx, network, server) 
if err != nil { 
    if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: err,
        }  
    }
    return &DnsError{
        Err: err.Error(),
    }  
} 

Or return only errCanceled, errTimeout.

c, err := r.dial(ctx, network, server) 
if err != nil { 
    if errors.Is(err, context.Canceled)  {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: errCanceled,
        }  
    }
    if errors.Is(err, context.DeadlineExceeded)  {
        return &DnsError{
            Err: err.Error(),
            UnwrapErr: errTimeout,
        }  
    }
    return &DnsError{
        Err: err.Error(),
    }  
} 

@ianlancetaylor
Copy link
Contributor

Looking at Resolver.dial, it already calls mapErr. The code that you mention is in Resolver.exchange; the only caller of Resolver.exchange already wraps the error up in a DNSError. So I think the only question is what Resolver.tryOneName should put in the new UnwrapErr field of the returned DNSError. I think it should set UnwrapErr to the error returned by Resolver.exchange. Perhaps I am missing something.

@mateusz834
Copy link
Member Author

mateusz834 commented Mar 17, 2024

Looking at Resolver.dial, it already calls mapErr.

Oh, right I somehow missed that, but the mapping is obviously not great, it does not go through Unwrap errors, so it might lead to some confusing cases:

r := net.Resolver{
	Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
		return nil, context.Canceled
	},
}
_, err := r.LookupMX(context.Background(), "test")
fmt.Println(err) // (*DnsError).UnwrapErr = errCancaled
r := net.Resolver{
	Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
	    return nil, &net.OpError{Err: context.Canceled} // wrapped error
	},
}
_, err := r.LookupMX(context.Background(), "test")
fmt.Println(err) // (*DnsError).UnwrapErr = &net.OpError{Err: context.Canceled}

I think it should set UnwrapErr to the error returned by Resolver.exchange. Perhaps I am missing something.

The current status of this proposal is to only expose context errors, so we should not expose all errors from Resolver.Dial. I think that we should only return the errCanceled, errTimeout when: errors.Is(err, context.Canceled) or errors.Is(err, context.DeadlineExceeded).

So:

p, h, err := r.exchange(ctx, server, q, cfg.timeout, cfg.useTCP, cfg.trustAD)
if err != nil {
dnsErr := &DNSError{
Err: err.Error(),
Name: name,
Server: server,
}

becomes:

p, h, err := r.exchange(ctx, server, q, cfg.timeout, cfg.useTCP, cfg.trustAD)
if err != nil {
     dnsErr := &DNSError{
	Err:    err.Error(),
	Name:   name,
	Server: server,
    }
    if errors.Is(err, context.Canceled) {
        dnsErr.UnwrapErr = errCalceled
    }
    if errors.Is(err, context.DeadlineExceeded) {
        dnsErr.UnwrapErr = errTimeout
    }
    // (...)
}

This way we do not expose implementation errors and allow the *DnsError to match context.DeadlineExceeded and context.Canceled through errors.Is.

@ianlancetaylor
Copy link
Contributor

That is OK with me. I guess the only question is how we should handle an error returned by a system call: should we wrap it or not? Since we haven't wrapped it in the past, I'm fine with continuing to not wrap it, at least until we have a clear reason to do so. Thanks.

@rsc
Copy link
Contributor

rsc commented Mar 27, 2024

It sounds like the only use of Unwrap will be for Is, in which case we could decide that instead of adding an Unwrap method we could implement an Is method on DNSError. Is there a reason to prefer one or the other? Probably Unwrap is more general for dealing with future issues?

@mateusz834
Copy link
Member Author

Just as you mentioned, Unwrap is better for errors that we might want to unwrap in future, but at the current state of the proposal there is no difference between adding a Is or Unwrap method. But i guess that for Is we would also have to add a UnwrapErr field, so in that case Unwrap feels better to me.

@neild
Copy link
Contributor

neild commented Mar 27, 2024

Unwrap is simpler and clearer. If you've got a choice between Unwrap and Is, I'd always recommend picking Unwrap.

Is is more general, you can do more with it, but it's more mechanism. I don't see a reason to use it here.

@rsc
Copy link
Contributor

rsc commented Apr 4, 2024

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add UnwrapErr error to DNSError, and an Unwrap method that returns UnwrapErr. By default, DNSErrors will have UnwrapErr set to nil, but we can set it explicitly for specific errors. To start, the context errors seem to be the only ones. If there are others that programs need to distinguish, please let us know.

@rsc rsc changed the title proposal: net: add UnwrapErr field and Unwrap method to *DNSError net: add UnwrapErr field and Unwrap method to *DNSError Apr 4, 2024
@rsc rsc modified the milestones: Proposal, Backlog Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Accepted
Development

Successfully merging a pull request may close this issue.

7 participants