I've just noticed that I'd also greatly benefit from adding a EncodeRuneBytes. This way conversion routines between different formats would be greatly simplified.

func EncodeRuneBytes(dst []byte, r rune) int

The DecodeRuneBytes function adds a new concept to the utf16 package, namely the encoding of sequences of UTF-16 codes as byte strings, with the concomitant need to specify the byte order.

If we make the byte encoding the caller's concern, then can't the problem be solved with code something like this?

   a := next()
   if !utf16.IsSurrogate(a) {
       return a
   } 
   b := next()
   if !utf16.IsSurrogate(b) {
       return 0xFFFD
   }
   return utf16.DecodeRune(a, b)

(Replace 'next' with your favorite byte iterator.)

@adonovan Hmm, rune b should actually be checked with a different function. IsSurrogate does: surr1 <= r && r < surr3 while rune b is invalid if it does not fulfill surr2 <= r && r < surr3. But you are close. (close as in that could work if there was a IsSecondSurrogate)

@adonovan Hmm, rune b should actually be checked with a different function. IsSurrogate does: surr1 <= r && r < surr3 while rune b is invalid if it does not fulfill surr2 <= r && r < surr3. But you are close. (close as in that could work if there was a IsSecondSurrogate)

DecodeRune applies the correct range check to each surrogate, so my example will return the correct rune; however, when it returns U+FFFD it may consume too much: if 'a' was invalid it should not consume 'b'. To fix that we would need not only IsHighSurrogate (that's the usual term for "second") for 'b', but also IsLowSurrogate for 'a'.

CC @dsnet

The utf16 package is meant to be as minimal as possible. If you really need to decode bytes, it is easy to decode them outside the loop and then call DecodeRune with successive pairs of uint16 values.

Also we probably do not want utf16 to import encoding/binary and then depend recursively on many other packages, including reflect. Right now utf16 has no imports at all, and it is important to keep it that way for use on Windows in package syscall.

This proposal has been declined as infeasible.
— rsc for the proposal review group