Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

multiprocessing in pyscript#2321

Neon22 started this conversation inProposals
Mar 31, 2025· 7 comments· 2 replies
Discussion options

As the new lib/multiprocessing will NOT be available for WASM.

Is it a good idea to implement it in pyscript and have workers do the job.
This would have the advantage of abstracting away all worker specific setup,comms,sync etc.

But worker config and async/await issues could be difficult.

As someone else suggested - maybe there's an intern somewhere who wants to contribute... :)

You must be logged in to vote

Replies: 7 comments 2 replies

Comment options

I like this idea. Quoting the Python Zen:

There should be one-- and preferably only one --obvious way to do it.Although that way may not be obvious at first unless you're Dutch.

Another complication would be to make this work seamlessly between MicroPython and PyOdide (as both orchestrator and workers).

You must be logged in to vote
0 replies
Comment options

feels like a mix ofSharedWorker andPyScript donkey capabilities ... we investigated the former approach and it's mostly a no-go because requires that file to be hosted where the user wants to have these features, we cannot inject aSharedWorker on our end due Web constraints + the donkey allows you already to spwan as many workers as you want providing a local/global state to ... but there is no orchestration to synchronize those, although a leading worker could do the trick.

That being said, this feels like a lot of work for zero provided use case + if WASM doesn't have it I believe there are tons of valid reasons or limitations on the Web, hence they don't have it?

This is also not currently part of my key goals so it would need coordination with the rest of the team as this would be huge effort, imho, with tons of unknowns in the making.

/cc@ntoll

You must be logged in to vote
0 replies
Comment options

This is an interesting case, that's also a good example of why PyScript is such a fascinating project.

Clearly, multiprocessing is the "blessed" CPython way to achieve a certain end. Sadly, there's no equivalent in MicroPython. Furthermore, the web world also has a similar thing in the form of web workers that, funnily enough, I often describe as isolated sub-processes of your web page, to Pythonistas who may not yet grok the browser. All this to say that we already have the capabilities of multiprocessing via web workers. I'm with@WebReflection in that we're balancing something Pythonic (multiprocessing) with the browser (web workers) and it's already a solved problem.

Perhaps the answer is really, "in idiomatic PyScript we use the donkey or built-in worker support"..?

You must be logged in to vote
0 replies
Comment options

Honestly, I never bothered to investigate what "donkey" means. I am justusing the xworker and sync APIs.
On Mon, Mar 31, 2025, 7:22 PM Nicholas Tollervey ***@***.***> wrote: This is an interesting case, that's also a good example of why PyScript is such a fascinating project. Clearly, multiprocessing is the "blessed" CPython way to achieve a certain end. Sadly, there's no equivalent in MicroPython. Furthermore, the web world also has a similar thing in the form of web workers that, funnily enough, I often describe as isolated sub-processes of your web page, to Pythonistas who may not yet grok the browser. All this to say that we already have the capabilities of multiprocessing via web workers. I'm with@WebReflection <https://github.com/WebReflection> in that we're balancing something Pythonic (multiprocessing) with the browser (web workers) and it's already a solved problem. Perhaps the answer is really, "in idiomatic PyScript we use the donkey or built-in worker support"..? — Reply to this email directly, view it on GitHub <#2321 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE4XAKFTLJS5NHANZCQDZD32XF2TZAVCNFSM6AAAAAB2D2ODF2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENRXHE2DINQ> . You are receiving this because you commented.Message ID: ***@***.***>
You must be logged in to vote
1 reply
@WebReflection
Comment options

we definitively need to provide more examples out there and right now it was a requirement for EduBlocks use case ... you bootstrap something, you can pass along Python code without blocking main or anything else, you can evaluate atomically, persist results and whatnot ... and it's currently exposed only as JS utility, you can't use it directly in Python but then again, you could create ajs_modules.main entry that gives you that, and you have full control of the lifecycle of that remote worker.

current docs here explain the API:https://docs.pyscript.net/2025.3.1/api/#pyscriptcoredonkey

we should probably add more examples in there but a basic playground is here:https://pyscript.com/@agiammarchi/pyscript-donkey/latest?files=index.html,index.js

Comment options

So:

  • The Ltk uses a pub/sub approach using the polyscript XWorker. I am supposing Chris crafted this early on. Certainly long before the donkey was created.
  • The donkey was (perhaps?) crafted for Josh to use to solve the specific problem (am I getting this right) of running a preloaded worker (so startup was immediate) running python code which could be inserted into the donkey, run, and report output to a ui element of some kind. (So typically showing a running code fragment to educate about python code.) It uses sync to send data back to main.
    • The donkey is setup in index.html
  • The Pyworker as defined in the docs -https://docs.pyscript.net/2025.3.1/user-guide/workers/ can be initiated from index.html, or from inside the python code.
    • it does one job at a time but can do different functions and sync back for each one.
    • it can be named to make it easier to keep track of stuff.
    • is an ideal way to have mpy on main and py for the worker. (faster UI startup)
    • starting it from inside the python code makes it more useful for single file components which, in turn, make it easier to make complex UIs from premade sophisticated UI elements.

So I think I'm correct if I say:

  • if you have one long running task to do every now and then
    • make a donkey or a Pyworker in index.html (so it can start without loading first)
    • call it and get the result back using sync, or have it modify a dom element directly on completion
  • if you want the fastest ui startup possible using pyscript then:
    • make main an mpy and do all code that needs full cython capability using pyodide workers or donkeys.
  • if you are making single file components for a complex UI then use a Pyworker created in python code, but get it started asap so it can be responsive

not sure how to best characterise the use cases for other approaches.
Maybe this would be a good thing to elucidate and put in the FAQ ?

(I continue to work on ltk 05 starter project (workers) whch is in a PR to the examples github repo.

You must be logged in to vote
0 replies
Comment options

So maybe the relevant aspect of his should be moved to docs ?as well as#2315

You must be logged in to vote
0 replies
Comment options

My 2¢ as an user: I'd never expect `multiprocessing` to work on PyScript, Iwould never think of using it because of how far removed PyScript is fromthe whole idea of spawning OS processes!But, in the same way that `threading` and `multiprocessing` share a coreAPI that is very similar and even interchangeable in simple use cases, I'dlike to see a module offering that same minimal API to run tasks on webworkers.Also, it would be cool to have a `WorkerPoolExecutor` class similar to the`ThreadPoolExecutor` and `ProcessPoolExecutor`. These (1) are easy to usein many cases and (2) integrate with `async/await` when the codebase ismostly async.Cheers,Luciano
On Wed, Apr 2, 2025 at 5:24 PM Neon22 ***@***.***> wrote: So maybe the relevant aspect of his should be moved to docs ?as well as#2315 <#2315> — Reply to this email directly, view it on GitHub <#2321 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAHJ6H52KZCP6GATI5KMD32XRIPRAVCNFSM6AAAAAB2D2ODF2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENZQGYYTKNY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
-- Luciano Ramalho| Author of Fluent Python (O'Reilly, 2015-2022)|https://www.oreilly.com/library/view/fluent-python-2nd/9781492056348/| Mastodon: @***@***.***
You must be logged in to vote
1 reply
@ntoll
Comment options

@ramalho yes - 100% agree with what you say. Multiprocessing and threading don't feel right in the web world, but perhaps we can "ape" the existing API's for worker support in a Pythonic manner (working in both Pyodide and MicroPython).

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
5 participants
@Neon22@ramalho@ntoll@WebReflection@laffra

[8]ページ先頭

©2009-2025 Movatter.jp