Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[Process] Add support for Fiber#43678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
lyrixx wants to merge1 commit intosymfony:7.4
base:7.4
Choose a base branch
Loading
fromlyrixx:process-fiber

Conversation

lyrixx
Copy link
Member

@lyrixxlyrixx commentedOct 23, 2021
edited by xabbuh
Loading

QA
Branch?7.1
Bug fix?no
New feature?yes
Deprecations?no
Tickets
LicenseMIT
Doc PR

Example:

(Note: The loop is very naive, but it's for demo purpose)

<?phpuseSymfony\Component\Process\Process;require__DIR__ .'/vendor/autoload.php';functionlogg(string$message,string ...$params){dump(sprintf($message, ...$params));}class Loop{publicfunction__construct(privatearray$fibers    ) {    }publicfunctionrun()    {foreach ($this->fibersas$k =>$fiber) {logg('Fiber %d started',$k);$fiber->start();        }while ($this->fibers) {foreach ($this->fibersas$k =>$fiber) {if ($fiber->isTerminated()) {logg('Fiber %d finished',$k);                    unset($this->fibers[$k]);continue;                }$fiber->resume();            }usleep(100_000);        }    }}// // Sync// $process = new Process(['sleep', 1]);// $process->mustRun();// Async$fiber1 =newFiber(function () {$process =newProcess(['sleep',1]);$process->mustRun();});$fiber2 =newFiber(function () {$process =newProcess(['sleep',2]);$process->mustRun();});$fiber3 =newFiber(function () {$process =newProcess(['sleep',3]);$process->mustRun();});// dump($fiber1);die();$loop =newLoop([$fiber3,$fiber2,$fiber1]);$loop->run();
process-fiber.mp4

PGBastien, dunglas, emullaraj, joelwurtz, chalasr, welcoMattic, ging-dev, fancyweb, Chris53897, ismail1432, and 3 more reacted with thumbs up emojishouze, Guikingone, PGBastien, zmitic, adrienbrault, welcoMattic, ging-dev, fancyweb, and yceruto reacted with heart emojiderrabus, yceruto, and sstok reacted with rocket emoji
@nicolas-grekas

This comment has been minimized.

@lyrixx

This comment has been minimized.

@b-viguier
Copy link

Glad to see that you consider using Fibers! 🤗
Just some thoughts about this:

  • What if the fiber is never resumed?
  • What if the fiber is resumed withthrow?
  • What if the fiber is resumed without callingusleep in the meantime? (To be honest, I'm not sureProcess actually needs this…)

My point is that it could be interesting to define acontract about"how to use safely a SF Process in a Fiber".
A part of this contract could also be transmitted via thesuspend function, a kind ofpromise to be resumed inn microseconds, or when a goal is reached… 🤷

I have no particular idea about this, I'm not sure if Symfony needs a complete async tools set, but I'm just wondering if thissuspend can be an open door to wrong usages.

Anyway, this PR has my full attention 😉👍

@lyrixx
Copy link
MemberAuthor

lyrixx commentedOct 23, 2021
edited
Loading

Glad to see that you consider using Fibers!

Thanks a lot for your feedback

  • What if the fiber is never resumed?

The process will last, but it's exactly the same as when you write the follow code

$p = new Process['sleep', '10'];$p->start();// Nothing here

So IMHO, nothing has to be done here

  • What if the fiber is resumed withthrow?

What would be the use case to resume it withthrow?

  • What if the fiber is resumed without callingusleep in the meantime? (To be honest, I'm not sureProcess actually needs this…)

Indeed, I was not sure about the use case, but actually, it eases the async handling of process.

My point is that it could be interesting to define acontract about"how to use safely a SF Process in a Fiber".

I totally agree with that 👍🏼, and more generally in Symfony

I'm just wondering if thissuspend can be an open door to wrong usages.

Yes, it can. I said that, because we keeps reading issue where people do really strange things :) I guess it's how OSS works :)

@b-viguier
Copy link

The process will last, but it's exactly the same as when you write the follow code [...]
So IMHO, nothing has to be done here

👍

What would be the use case to resume it withthrow?

Since it's possible, we can expect that someone will try 😅.
But it could a way to stop a too long process (timeout), or to cancel it for other external reasons (too much opened fibers, too much memory... 🤷‍♂️)

Yes, it can. I said that, because we keeps reading issue where people do really strange things :) I guess it's how OSS works :)

😅😉

@fabpotfabpot modified the milestones:5.4,6.1Oct 29, 2021
@fabpot
Copy link
Member

What's the status here?@lyrixx?

@lyrixx
Copy link
MemberAuthor

lyrixx commentedMar 28, 2022
edited
Loading

I think there is consensus yet on how to add fibrer support in Symfony. Let's close it

But if someone want it, we could also merge it.

@lyrixxlyrixx closed thisMar 28, 2022
@lyrixxlyrixx deleted the process-fiber branchFebruary 27, 2024 12:57
@soyuka
Copy link
Contributor

mhh wait why is this not merged?

@lyrixx
Copy link
MemberAuthor

Wep, too bad. People are not ready for that!

And we need that incastor

cc@joelwurtz

@fabpot
Copy link
Member

@lyrixx Not sure how you came to that conclusion as you were the one closing this PR.

@lyrixxlyrixx restored the process-fiber branchFebruary 28, 2024 07:32
@lyrixx
Copy link
MemberAuthor

lyrixx commentedFeb 28, 2024
edited
Loading

Let's re-open the PR 👍🏼 (rebased, and updated)

Comment on lines +434 to +437
$startedAt = microtime(true);
$fiber->suspend();
$sleepFor = (int) (1000 - (microtime(true) - $startedAt) * 1000000);
if (0 < $sleepFor) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

(same below)

Suggested change
$startedAt =microtime(true);
$fiber->suspend();
$sleepFor = (int) (1000 - (microtime(true) -$startedAt) *1000000);
if (0 <$sleepFor) {
$suspendedAt =microtime(true);
$fiber->suspend();
if (0 <$sleepFor = (int) (1000 - (microtime(true) -$suspendedAt) *1000000)) {

But I would also make readPipe return true/false whether the process did yield some activity or not. This way, we won't force a sleep while there are things to do.

@joelwurtz
Copy link
Contributor

joelwurtz commentedFeb 28, 2024
edited
Loading

After playing a bit with the process and fiber, i think there is no need to add fiber support in this class, the only things the process class need would be getter for thestdout andstderr stream.

Here is an example on how it may work (i use closure binding to get the streams) :

require'vendor/autoload.php';$pid =null;$running =true;// listen for sigchldpcntl_signal(SIGCHLD,function ($signo,$data)use (&$pid, &$running) {if ($pid ===$data['pid']) {$running =false;    }});pcntl_async_signals(true);$fiber =newFiber(function () {$process = \Symfony\Component\Process\Process::fromShellCommandline('echo -n "test"; echo -n "bar" 1>&2 ; sleep 2');// bind a function to get the output stream$getStdout = Closure::bind(function () {return$this->stdout;    },$process,$process);$getStderr = Closure::bind(function () {return$this->stderr;    },$process,$process);$process->start();$stdout =$getStdout();$stderr =$getStderr();stream_set_blocking($stdout,0);stream_set_blocking($stderr,0);while ($process->isRunning()) {$action = \Fiber::suspend([$stdout,$stderr]);if ($action ==='stdout') {$content =$process->getIncrementalOutput();if ($content !=='') {var_dump('stdout :' .$content);            }        }if ($action ==='stderr') {$content =$process->getIncrementalErrorOutput();if ($content !=='') {var_dump('stderr :' .$content);            }        }    }return$process->getExitCode();});[$stdout,$stderr] =$fiber->start();$streams = [$stdout,$stderr];while (!$fiber->isTerminated()) {$write =null;$except =null;$read =$streams;$modified =stream_select($read,$write,$except,0,100_000);if ($modified ===false) {break;    }if ($modified >0) {foreach ($readas$stream) {if ($fiber->isTerminated()) {break;            }if ($stream ===$stdout) {$fiber->resume('stdout');            }if ($stream ===$stderr) {$fiber->resume('stderr');            }        }    }}var_dump('exit code :' .$fiber->getReturn());

There may be some adjustement to do in the process class, but checking for fibers may not be needed IMHO

Also this totally remove sleep inside the process, (it may sleep but it depends only on the event loop implementation)

Exposing those streams make it be compatible withRevolt also, and i believe it would be better to provider a helper around theProcess class that make it compatible with revolt ?

@joelwurtz
Copy link
Contributor

joelwurtz commentedFeb 28, 2024
edited
Loading

Didn't see this was in thewait function, it can make sense for it for existing library / app that use this method.

Part of this example can be reuse, like instead of doing asleep it could do a stream_select with a timeout so it wakes up as soon as there is something to read / write or use theSIGCHLD signal to wake up also when the process has stopped (only with pcntl extension) ?

@@ -429,12 +429,31 @@ public function wait(?callable $callback = null): int
do {
$this->checkTimeout();
$running = $this->isRunning() && ('\\' === \DIRECTORY_SEPARATOR || $this->processPipes->areOpen());
$this->readPipes($running, '\\' !== \DIRECTORY_SEPARATOR || !$running);
if ($fiber = \Fiber::getCurrent()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I believe there should be an additional flag to enable this support otherwise it would be a bc break for people already using this method in aFiber

$fiber->suspend();
$sleepFor = (int) (1000 - (microtime(true) - $startedAt) * 1000000);
if (0 < $sleepFor) {
usleep($sleepFor);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

maybe it could use stream_select on the pipes instead of usleep ? so it would directly read / write if there is something to do instead of sleeping ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

we do use stream_select already so I guess we added this to address a situation where stream_select doesn't apply. git blame could help remember :)

@lyrixxlyrixx modified the milestones:6.1,7.1Mar 11, 2024
@xabbuhxabbuh modified the milestones:7.1,7.2May 15, 2024
@fabpotfabpot modified the milestones:7.2,7.3Nov 20, 2024
@fabpot
Copy link
Member

Is it still relevant?

@fabpotfabpot modified the milestones:7.3,7.4May 26, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@joelwurtzjoelwurtzjoelwurtz left review comments

@nicolas-grekasnicolas-grekasnicolas-grekas left review comments

@stofstofstof left review comments

@OskarStarkOskarStarkOskarStark left review comments

@derrabusderrabusderrabus left review comments

@dunglasdunglasdunglas approved these changes

Assignees
No one assigned
Projects
None yet
Milestone
7.4
Development

Successfully merging this pull request may close these issues.

12 participants
@lyrixx@nicolas-grekas@b-viguier@fabpot@soyuka@joelwurtz@dunglas@stof@OskarStark@derrabus@xabbuh@carsonbot

[8]ページ先頭

©2009-2025 Movatter.jp