Hi, and thanks for the work on neovim. I'm an avid user and fully customize my workflow (but still haven't jumped onto nightly)

I have some questions regarding the use of neovim as a "remote command handler". This may be a long post but please bear in mind that my aim is to share the pain point I bumped into, how I tackled it and if it's of any use for upstream neovim.

This may be related to this issue:Nvim can be a Lua host; Lua plugins can define RPC method/request handlers #31868

My use case

I'm an embedded software developer and work with vendor linux kernel trees and Android checkouts daily. These projects come with a big git history that I often peek into looking for patches or insight. I've been using (vim->neovim) for quite some time and always relied on vim-fugitive for git integration.

The problem is that the single-thread nature of vim/nvim results in some git operations to lock up the UI for ~10 secs. I bumped into this every time I attempted to look at the history of a file or a log graph using vim-flog (or GV). I took a stab at my own (lua) replica of the logic found in vim-fugitive to retrieve the commit history of a file and tried to work around the lockup problem.

My attempted solution

Of course I wanted to use the nvim API functions and so found that libuv threads were hard for me to use. In the end, and after looking at neovim open issues and how fzf-lua tackled some things, I came up with a module that does the following:

• The user "registers" a command passing in both "completion callback" and "implementation" functions.
• Spins up a headless neovim process for RPC via unix pipe.
• I then invokerequire("rcmd").send_remote_command("git_log_follow", { filename, gitdir }). The first time this is executed it lazily spins up the process, and then send the command request via RPC.
• I send a call for RPC "nvim_exec_lua" API function with a small lua script to pass the args and trigger my module with the implementation to do the work.
• Some time later I get the callback function invoked from the remote neovim instance and then just present the data in a float, effectively having skipped the lockup in my main neovim process.

Here's a link to a sequence diagram for graphical illustration (dont know how long this links remains valid for):
mermaid.ink sequence diag

Something I had to solve was how to pass lua dicts through RPC to the remote instance, in order return the commit history in a more friendly form. I took a cue from fzf-lua and just included the single-file lib "serpent.lua" for serialization and it worked very well.
See:https://github.com/pkulchenko/serpent
On top of this I have a small "xfer" module that introduces "transactions" and handles the sending of big tables. This is half-baked really, but a table can be split in packets for TX, getting RX confirmation for each one, and are completed once all packets arrive at the remote neovim.

Environment of remote nvim?

I rely on my own config being loaded in the remote neovim process for this to work. It feels like cheating. I would like to have the remote neovim spin up with--clean and somehow instruct it to load what's minimally required for the custom command implementation.

I read#18035 regarding serializing lua functions and it seems very complex for user-setup of lua modules and general environment (but I havent tried it, though).

It seems simpler to provide a special RTP path at command-registration time, or something of the like.

Questions

1. Is this approach completely off the mark?

I don't know about co-routines in lua but see there's some present work for them. Maybe with coroutes no additional nvim process is needed?

2. If not, is there any way I can contribute/help to move a similar feature forward? I attempted to clean up my stuff multiple times but dont know really what my "target" is.

If you read up to this point, thank you! I've been postponing this question almost all year long.
If not, that's fine