Thanks for putting this together—this is exciting.

Generating a module that can act as a reactor and a command sounds like a great idea. I noticed this might conflict with how Node interprets a module. If a module exports both _start and _initialize, it will throw an exception: https://nodejs.org/api/wasi.html

1. One could argue this is undesired behavior, and Node could change.
2. What happens if a host detects and calls both _initialize and _start?
3. Exporting one or the other, but not both, implies some kind of configuration or detection.
   • In this TinyGo PR I experimented with detecting lack of main.main as the trigger for "reactor" mode with _initialize as the entry point: runtime, builder: WebAssembly reactor mode tinygo-org/tinygo#4082

The directive is only allowed on functions, not methods.

Using //go:wasmimport on methods has been helpful for mapping Component Model resource methods in WASI Preview 2:

From https://github.com/ydnar/wasm-tools-go/blob/9b4707e054a8b528b27240cba6c05557c4e26a53/wasi/io/error/error.wit.go:

// ToDebugString represents the method "wasi:io/error.error#to-debug-string".
//
// Returns a string that is suitable to assist humans in debugging
// this error.
//
// WARNING: The returned string should not be consumed mechanically!
// It may change across platforms, hosts, or other implementation
// details. Parsing this string is a major platform-compatibility
// hazard.
func (self Error) ToDebugString() string {
	var ret string
	self.to_debug_string(&ret)
	return ret
}

//go:wasmimport wasi:io/error@0.2.0-rc-2023-11-10 [method]error.to-debug-string
func (self Error) to_debug_string(ret *string)

Subjectively, using methods seems better aligned with the Component Model semantics than the equivalent:

//go:wasmimport wasi:io/error@0.2.0-rc-2023-11-10 [method]error.to-debug-string
func error__to_debug_string(self Error, ret *string)

Given that resources are opaque i32 handles, the same could be true for implementing exported methods via //go:wasmexport.

Thanks for putting this together—this is exciting.

Generating a module that can act as a reactor and a command sounds like a great idea. I noticed this might conflict with how Node interprets a module. If a module exports both _start and _initialize, it will throw an exception: https://nodejs.org/api/wasi.html

1. One could argue this is undesired behavior, and Node could change.

2. What happens if a host detects and calls both _initialize and _start?

3. Exporting one or the other, but not both, implies some kind of configuration or detection.
   
   * In this TinyGo PR I experimented with detecting lack of main.main as the trigger for "reactor" mode with _initialize as the entry point: [runtime, builder: WebAssembly reactor mode tinygo-org/tinygo#4082](https://github.com/tinygo-org/tinygo/pull/4082)

Thank you for the information about Node's behavior here, I wasn't aware. That is certainly troubling. I will try to see what if any other precedent there is for this behavior in the ecosystem to see whether we or Node are in the wrong.

If a host calls both _initialize and _start, it will run initialization once (initialization has to be protected with something like a sync.Once to be idempotent) and then run func main(). Just calling _start will accomplish the same thing.

Indeed, if we do need some way to allow users to choose whether to build a command (executing func main()) or library (just initializating and exporting functions), this proposal would need to add some way for users to turn that knob. I don't want to prejudice that discussion until we know if we need it.

Given that resources are opaque i32 handles, the same could be true for implementing exported methods via //go:wasmexport.

This may be true, but I still think this proposal serves as an MVP that we can enhance with method support in a subsequent proposal once the initial hurdles have been overcome.

Thank you for the information about Node's behavior here, I wasn't aware. That is certainly troubling. I will try to see what if any other precedent there is for this behavior in the ecosystem to see whether we or Node are in the wrong.

If a host calls both _initialize and _start, it will run initialization once (initialization has to be protected with something like a sync.Once to be idempotent) and then run func main(). Just calling _start will accomplish the same thing.

Maybe it’s a bigger question about what is defined behavior. Is having both _initialize and _start valid, or undefined? Having only one entry point is less ambiguous, e.g. the host can only call one, but not both (or choose, which could be contrary to the user’s expectation).

Indeed, if we do need some way to allow users to choose whether to build a command (executing func main()) or library (just initializating and exporting functions), this proposal would need to add some way for users to turn that knob. I don't want to prejudice that discussion until we know if we need it.

Have had previous discussions about -buildmode=wasm-reactor to mirror -buildmode=c-shared.

I created an issue to ask the NodeJS devs for the source of this design decision: nodejs/node#51544

Hey. Node developer that implemented that design decision here. 👋

That change was nearly four years ago, and I have since forgotten the exact motivation. However, I was able to dig this up: WebAssembly/WASI@d8b286c. At that point in time, WASI commands had a _start() function, and WASI reactors had an _initialize() function. Commands and reactors were mutually exclusive.

WASI has changed a good bit since then. I no longer work on WASI, so I don't know if that design decision is still valid or not. I would recommend checking with the folks in the WASI repos.

WebAssembly/wasi-http#95 contains discussion to use _initialize func. So if that is not possible to golang, it would be difficult

Any user created func init() would be run in _initialize, is this not sufficient?

Hey. Node developer that implemented that design decision here. 👋

That change was nearly four years ago, and I have since forgotten the exact motivation. However, I was able to dig this up: WebAssembly/WASI@d8b286c. At that point in time, WASI commands had a _start() function, and WASI reactors had an _initialize() function. Commands and reactors were mutually exclusive.

WASI has changed a good bit since then. I no longer work on WASI, so I don't know if that design decision is still valid or not. I would recommend checking with the folks in the WASI repos.

Thanks so much for providing your input and this reference. It seems this doc now lives at https://github.com/WebAssembly/WASI/blob/a7be582112b35e281058f1df7d8628bb30a69c3f/legacy/application-abi.md. I wonder, given that this is now under the legacy heading, whether this statement is still true:

These kinds are mutually exclusive; implementations should report an error if asked to instantiate a module containing exports which declare it to be of multiple kinds.

If so, this design would need to change to allow the user to choose whether to compile a Command or a Library (Reactor). @sunfishcode perhaps you could provide some guidance here?

The _start and _initialize functions and legacy/application-abi.md file are all Preview 1 things. Many Preview 1 Wasm engines recognize _start for commands, and some recognize _initialize as an entrypoint for reactors.

Preview 2 is based on the Wasm component model.

• For the command world, there is an exported run function which is the command entrypoint (analogous to what _start was in Preview 1).
• There isn't an export for reactor-style wasm programs. The component model itself has a mechanism to call functions on initialization. Unlike core-Wasm's start section, the component-model's start section doesn't need to worry about memory not being exported yet, so we can use for arbitrary initialization (analogous to what _initialize was in Preview 1).
  • If the tooling you use to go from a core-wasm module to a component supports it, the core-wasm _initialize function may be automatically wired up to the component-model start section.

Edit: I was mistaken about the component-model start function. It's not permitted to call imports, so it's not usable for arbitrary initialization code. There are ongoing discussions about this.

Thanks for the explanation. This proposal targets our existing wasm implementations, js/wasm and wasip1/wasm. We'll have a think about the best way to go about this that doesn't paint us into a corner when it comes to adding support for wasip2 down the line.

• If the tooling you use to go from a core-wasm module to a component supports it, the core-wasm _initialize function may be automatically wired up to the component-model start section.

What’s an example of tooling that converts a module to a component that supports the component model start section?

Wasmtime seems to not support the start section? https://github.com/bytecodealliance/wasmtime/blob/e9d580776ee27f4ed59ba334765aacbcc22fa6e4/crates/environ/src/component/translate.rs#L623

In light of the discussion around NodeJS's behavior and the documented separation between _initialize and _start in wasip1, we've updated the proposal to include a new -buildmode=wasip1-reactor, used to instruct the compiler to produce a Wasm binary with an _initialize function in place of the _start function. The use of go:wasmexport is limited to this new build mode, which is only available for GOOS=wasip1.

Thanks for the proposal! Looks good overall.

-buildmode=wasip1-reactor

Is there something similar for js/wasm? Or the library/export mechanism is very different? Also, will the mechanism be similar for later wasip2, or eventual wasi? If so, maybe we can choose a more general name like wasm-library, so we don't need to have a different build mode for each of them? (For start it is okay to only implement on wasip1, just like the c-shared build mode is not implemented on all platforms.)

_initialize

Is _initialize required to be called before any exported functions can be called? Or, the first time it calls into Go _initialize is called if not already? Or the Wasm execution engine always automatically calls _initialize on module load time, so it is guaranteed to be called first?

In the absence of threading or stack switching capability in Wasm, the simplest option is to document that all goroutines still running when the invocation of the go:wasmexport function returns will be paused until the control flow re-enters the Go application.

So, this sounds like that at the end of the exported function, the Go runtime will not try to schedule other goroutines to run but directly return to Wasm? I assume this might be okay. But js.FuncOf seems to choose a different approach. This is also related to the discussion in #42372. Could you explain the reason for choosing this approach?

GODEBUG=wasmgoroutinemon=1

I'm not sure we want this debug mode. As you mentioned, it is probably not uncommon to have background goroutines. If one wants to ensure there is no goroutine at the time of exported function exiting, one probably can check it with runtime.NumGoroutine.

Thanks.

-buildmode=wasip1-reactor

Is there something similar for js/wasm? Or the library/export mechanism is very different? Also, will the mechanism be similar for later wasip2, or eventual wasi? If so, maybe we can choose a more general name like wasm-library, so we don't need to have a different build mode for each of them? (For start it is okay to only implement on wasip1, just like the c-shared build mode is not implemented on all platforms.)

Any wasm module can declare exports, but we don't anticipate that exporting methods like this is generally useful to users of js/wasm - we have js.FuncOf today to make Go code callable from JS, and making it callable from Wasm doesn't seem nearly as useful for that platform.

For wasip2, as illustrated by Dan's reply above, it's not clear what the export mechanism would look like yet. The name wasip1-reactor is chosen to be deliberately specific to wasip1. The exact functionality in this proposal would be limited to wasip1 forever, and any hypothetical wasip2 proposal would likely have to explain how/if wasmexport will be available for that target initially.

_initialize

Is _initialize required to be called before any exported functions can be called? Or, the first time it calls into Go _initialize is called if not already? Or the Wasm execution engine always automatically calls _initialize on module load time, so it is guaranteed to be called first?

The expectation within the greater wasip1 ecosystem seems to be that if _initialize is exported by a module, it will be called before any exported methods are called. Our implementation wouldn't automatically call _initialize if it hasn't been called, it would likely just crash horribly.

In the absence of threading or stack switching capability in Wasm, the simplest option is to document that all goroutines still running when the invocation of the go:wasmexport function returns will be paused until the control flow re-enters the Go application.

So, this sounds like that at the end of the exported function, the Go runtime will not try to schedule other goroutines to run but directly return to Wasm? I assume this might be okay. But js.FuncOf seems to choose a different approach. This is also related to the discussion in #42372. Could you explain the reason for choosing this approach?

Yes, once the exported function returns, we would not schedule other available goroutines but return to the host. The reason for this is that we believe it's what users would expect to happen, since the runtime and various standard libraries maintain their own goroutines that would make it hard to predict the behavior and runtime of exported functions. If you believe that to be an incorrect assumption we're happy to reconsider this. Note that this also includes goroutines started by the exported function itself.

GODEBUG=wasmgoroutinemon=1

I'm not sure we want this debug mode. As you mentioned, it is probably not uncommon to have background goroutines. If one wants to ensure there is no goroutine at the time of exported function exiting, one probably can check it with runtime.NumGoroutine.

This is a fair point, and we could certainly slim down the proposal by removing this and consider it as a future addition. Thanks!

Sounds good, thanks.

I guess it might be fine to return to the host when the exported function returns. I guess one question is when the "background" goroutines run. If the exported functions get called and return, but none of them explicitly wait for the background goroutines, the background goroutines will probably never run? Would that be a problem for, say, timers?

The background goroutines could run again if the exported function gets called again. I think ideally users who want concurrent work in exported functions would utilize something like a sync.WaitGroup to ensure work is completed during the execution of the function. A future proposal might be able to tackle this by exposing something like _gosched to run all goroutines until asleep, but this proposal does not account for such a feature. Also, since Threads is stable in Wasm, we may be able to just spawn new threads in the near future, which could execute in parallel to the exported function.

The problem of having goroutines blocked after the export call returned isn't much different from what happens when invoking an import. When a WebAssembly module calls a host import, it yields control to the WebAssembly runtime; no goroutines can execute during that time.

The issue is amplified with exports because the WebAssembly runtime could keep the module paused for extended periods of time, and the expectation is that imports usually return shortly after they were invoked, but it isn't fundamentally different.

Despite the limitations, we can still deliver incremental value to Go developers by allowing them to declare exports.

@johanbrandhorst when it comes to background goroutines, do you know if the proposed solution different from tinygo which supports exported functions?

@johanbrandhorst when it comes to background goroutines, do you know if the proposed solution different from tinygo which supports exported functions?

We have a separate PR to TinyGo that prototypes the same model, suspending and resuming goroutines on an export call.

The background goroutines could run again if the exported function gets called again.

If the exported function (or another exported function) gets called again, and that function returns without explicitly synchronizing or rescheduling, the background goroutine may still not run? I think blocking for a little while is not a problem, but it might be a problem if it never get to run (while the exported function get called again and again)?

As you mentioned, once we have thread supports, it may not be a problem.

It's true that goroutines may never get to run if there's no point in the exported function to yield to the runtime. I think that's still what I would expect to happen if I wrote my exported function this way. All alternatives would be more confusing I think (waiting before returning or maybe running the scheduler before executing the exported function).

As you say, we can hopefully improve this with threads support in the future.

Okay. This is probably fine. We can change it later if there is any problem. Thanks.

I've removed the GODEBUG option, we can add that as an enhancement later and suggest users use NumGoroutines() for their debugging needs for now.

I think this behavior more or less explained by their RATIONALE.md:

Why do we only return a sys.ExitError on a non-zero exit code?

It is reasonable to think an exit error should be returned, even if the code is success (zero). Even on success, the module is no longer functional. For example, function exports would error later. However, wazero does not. The only time sys.ExitError is on error (non-zero).

This decision was to improve performance and ergonomics for guests that both use WASI (have a _start function), and also allow custom exports. Specifically, Rust, TinyGo and normal wasi-libc, don't exit the module during _start. If they did, it would invalidate their function exports. This means it is unlikely most compilers will change this behavior.

GOOS=waspi1 from Go 1.21 does exit during _start. However, it doesn't support other exports besides _start, and _start is not defined to be called multiple times anyway.

Since sys.ExitError is not always returned, we added Module.IsClosed for defensive checks. This helps integrators avoid calling functions which will always fail.

We use tinygo to compile Go to Wasm with -target=wasi, so that modules can be used as plugins. Not sure if reactor pattern supported by tinygo at all.

The plugin pattern I think you're referring to will be perfectly supported by the functionality introduced in this proposal. Note that it's also possible to do with a "Command" today (see the references in the Background section of the proposal), and will remain possible if this proposal is implemented.

Have all remaining concerns about this proposal been addressed?

The proposal is to add support for -buildmode=c-shared in wasm, and to add

//go:wasmexport [name]

as a directive analogous to what //export does when using cgo.

@rsc I assume that it would work for libraries, but will it work for programs that work in the background?

For example, a program acts as a server worker that accepts requests from a page.
Let's assume that client calls a WASM exported function to send a request to a WASM server.
The server processes a request and calls the client back to pass a result.

package main

var (
  requests = make(chan []int)
)

//go:wasmimport myNamespace submitResponse
func submitResponse(result int)

//go:wasmexport receiveRequest
func receiveRequest(a, b int) {
   requests <- []int{a, b}
}

func main() {
   ctx := context.TODO()
   requests := make(chan []int)
   go listen(requests)
   <-ctx.Done()
}

func listen() {
   for req := range requests {
       // do some logic and call client.
       submitResponse(42)
   }
}

Will that case work?

The proposal makes it clear what can be expected in this situation: When receiveRequest is called from the host (presumably to handle some request), the runtime will wake up (having either previously handled some other export and returned to the host, or been initialized using _initialize and returned to the host), a new goroutine calling receiveRequest will be created, and the scheduler will run. Any of the available goroutines to run at this point (which may include goroutines started by runtime initialization, during previous export invocations, or the goroutine just created to handle the request) may be scheduled.

In this case, because receiveRequest blocks on a channel send, the function would yield back to the runtime, the scheduler would eventually schedule listen, and the request would be handled by submitResponse, which calls back into the host to submit the response. This host call is blocking and no goroutines are scheduled while it is running. Once it returns, listen will go back to blocking on reading from requests and yield back to the runtime, which will schedule receiveRequest, which is now unblocked. Once it returns, the runtime will return back to the host, and no other goroutines will be handled.

That is my understanding of the runtime and scheduler behavior, which is admittedly hazy.

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add support for -buildmode=c-shared in wasm, and to add

//go:wasmexport [name]

as a directive analogous to what //export does when using cgo.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add support for -buildmode=c-shared in wasm, and to add

//go:wasmexport [name]

as a directive analogous to what //export does when using cgo.

@johanbrandhorst As the proposal has been accepted, do you have any update for implementation or prototype? Thanks.

Nothing to share at the moment, just experimenting locally with a prototype for now. We do intend to implement this.

@johanbrandhorst Any update on this? As mentioned in https://groups.google.com/g/golang-dev/c/Hy9iaQFhlHw/m/DCbJD7SkBQAJ , we are 4 weeks from the Go 1.23 release freeze. I'd hope we get this in for Go 1.23 release. It would be a good time now to prepare the CLs and start reviewing. Thank you!

We've still just been prototyping it, nothing to submit yet unfortunately. We're also hoping we could get something submitted soon but our primary goal is the 32bit port.

Thanks for the update. Would it be possible to get this submitted first? I would think this work is simpler whereas for the wasm32 port there are still details need to be fleshed out. Also let us know if there is anything we could help with. Thanks.

We're honestly still in the early stages of figuring out the changes necessary. If you have any time to help build a minimal functional prototype we could develop it from there. Unfortunately none of us are working on it full time.