I don't think we have a concrete plan (just prototypes), but we expect to be able to at least do the same IPC via mechanisms like WASI (what I do inhttps://jakebailey.dev/esbuild-playground, not to pump this side project). There's also the possibility to use plain messaging and only offer async in the browser, or some sort of async worker with SharedArrayBuffer based communication; the latter is already pretty well thought out for vscode.dev/github.dev, so there's theoretically some reusability there.

Actually, this has its own discussion here:
#514

People really need to provide their test code. I'm pretty sure the results are incorrect. Notably, threading needs to be disabled in WASM but we didn't do that in the repo yet.

Wouldn'thttps://extism.org be ideal here?

Why is exitism ideal? Which problem are you specifically trying to solve? Plugins?

To be clear, I do not think it's the direction we want to take to allow embedding languages into tsc. It could be technically feasible, but we'd now be a "runtime" and have to go through a lot of compliance work.....

@jakebailey it could allow us to hook into the compiler similarly to how its currently being done with the transformer API.

Yes that is probably my biggest concern - but since this all needs to be redesigned anyway, it sounds like you are also open to add proper hooks into these internals where it's needed. I think the Angular team, the Volar.js team and the Svelte team (which I'm a part of) are all doing lots of stuff with both the language service API and the plugin API, and are probably all very interested to talk details once the internals have stabilized a bit.

@andrewbranch I'm bummed that it won't be possible to usets-morph anymore to do codemods...people are mostly talking about more basic AST walking, but is type information going to be part of the API that operates over IPC?

We love ts-morph; it is explicitly an anti-goal to prevent ts-morph from working altogether. Type information is effectivelythe reason why we need to have an API at all 😄

Okay whew!

WE'RE SO BACK

FFI with Go is problematic because Go assumes it's the only instance of Go in a process, but people can and do import multiple versions of TypeScript (for example, to test DefinitelyTyped); while we did have some success loading multiple instances in the same process, it's not something the Go team supports.

Is there really no way to bootstrap go's global state once and then instantiate multiple things within that same global state from separate NAPI calls?

Loading multiple runtimes into the same process means they are all competing to register signal handlers, do preemption, manage certain process-specific limits, etc. Itmight work if all are built with the same exact version of Go, since things like signal handlers would still grab data at the right offsets into things like TLS, but it's definitely not a guaranteed.

See also:golang/go#65050 (comment)

What I mean is, I'm surprised it's not possible to only load a single runtime on the first FFI call, and have subsequent FFI calls instantiate structs into that same runtime...

Which from skimming that thread, sounds like it is possible using plugins instead of c-shared libraries...but I guess there's still a problem if different versions oftypescript-go are built for different versions of the Go runtime?

The Go runtime is entirely different between Go releases, and potentially Go patches. I don't even think it's guaranteed that it'll work if they happen to match.

Plugins are not an option; that's for loading Go code into Go code, not Go code into other things. That and Go plugins do not support Windows.

Absolutely; this is going to be mandatory going forward since it won't be apparent to API consumers what version they're talking to. A better versioning story is one of the reasons we want to make a more intentional design here. That may or may not correspond to the "language version", but we're going to have some concept of an "API version" that probably revs on a different axis.

Do note#455 (reply in thread)

This works but is likely not going to be safe for a number of important users.

I do actually have the code to do NAPI directly with Go, it'd be awesome if we could use it 😞

This works but is likely not going to be safe for a number of important users.

Perhaps my implementation can be provided to users who don't need to load multiple different ts-go instances 👀

@andrewbranch@jakebailey thoughts on how futureprooflibsyncrpc is?

I guess the fact that it's based upon NAPI is better thannode-fibers, but it still seems iffy

I don't really understand the question. This surely isn't worse than anyone else usingnapi-rs, and we're only using it to shuffle bytes around without hitting the event loop, not even constructing any objects or anything. It's possible that this could be implemented without NAPI by using plain reads/writes to file descriptors in JS, though that hasn't been tested in the newer IPC implementation.

Okay yeah I hadn't realized that if native code simply makes blocking calls that will block the source JS function call (and event loop).node-fibers had to do something complicated to enable function calls to block local function execution while allowing the event loop and other functions to continue executing, and that magic apparently depended on pretty specific V8 APIs. I mistakenly assumed that anything else that would make a JS function call block would be similar

#481

How would we use something typedoc with ts plugins? like, supporting .vue, .svelte, .gjs, or .gts files?

I think it’s very unlikely that any watch APIs will exist.

No, you cannot make use of any of this right now. The APIs are going to be different, so that API example from the old repo is unlikely to work. What is in the current API package is a prototype, a proof of concept.

We're certainly aware of the annoyances in the old API, and want to make it better, but we have a challenge ahead of us as to how plugins are supposed to work since we can't actually load random code into a Go binary like we did a JS process, nor would there be any way to access anything internal (only what we explicitly expose).

we can't actually load random code into a Go binary like we did a JS process

I can invoke Go from within JS using the bridge library you implemented (AFAIK that's what powers@typescript/api), are you saying there is no way to have a Go->JS call bridge?

I was able to create a rust client for the IPC-based API in a few hours, the hardest part was the encoding format. Excited to see where this goes!https://github.com/abhiaagarwal/tsgo-rust-ipc/

I can invoke Go from within JS using the bridge library you implemented (AFAIK that's what powers @typescript/api), are you saying there is no way to have a Go->JS call bridge?

Sure, but only if we actually launch the server via that bridge.

I don't get how the callbacks design doesn't already explicitly support this? Go is essentially invoking JS by asking it to return the results of a desired callback.

Concurrent requests in a single session will definitely be supported eventually. I’m not sure whether “one server many clients” will be a generally supported pattern or not.

Well, I'm planning to create a similar system. I need something where, for example, an OXC or SWC transformer can query the TypeScript TypeChecker for type information about a symbol. I'll probably fork tsgo and build an API layer on top of it, unfortunately, since tsgo keeps everything internal, I have to create a custom build. But I want to keep it compatible so I can still merge changes from the original tsgo repository into my project.

It would be nice if tsgo exposed a proper Go API so I could use it as a package instead of having to fork it.

The motivation is to provide extended type inference capabilities forRTTIST - runtime type information system (reflection). Rewriting everything (RTTIST's typegen) in Go might make sense if tsgo became the standard for build tools, replacing SWC, esbuild, and OXC - but I don't think that's likely to happen.

For what it's worth, I did look into this yesterday because I'm interested in doing the same thing (for oxc to solveoxc-project/oxc#3105) and I'm convinced it's possible: the API layer would have to expose theAcquireDocument and then, using the same binary format from the encoder (which the go implementation would need a decoder for), you can feed anast.SourceFile into tsgo. The cool part is for ASTs you don't provide, tsgo will first query the DocumentStore, and if it can't find a match, it'll fallback to the VFS/FS and generate an AST for that file. So you could register the ASTs directly of files you're interested in and fallback for files whose ASTs you don't care about.

This is a cursory look and is probably incorrect in places, but the API is designed well-enough that this is feasible.

Ohhh… I have an OXC-based transformer where I need to infer types, and that’s why I’m doing this. I didn’t know about that issue, but I’m basically solving it. 😆