Movatterモバイル変換

[Python-ideas] Learning from the shell in supporting asynciobackground calls

• Previous message (by thread):[Python-ideas] Concurrency Modules
• Next message (by thread):[Python-ideas] Learning from the shell in supporting asyncio background calls
• Messages sorted by:[ date ][ thread ][ subject ][ author ]

Hi folks,Based on the recent discussions Sven kicked off regarding thecomplexity of interacting with asyncio from otherwise synchronouscode, I came up with an API design that I like inspired by the waybackground and foreground tasks in the POSIX shell work.My blog post about this design is athttp://www.curiousefficiency.org/posts/2015/07/asyncio-background-calls.html,but the essential components are the following two APIs:    def run_in_background(target, *, loop=None):        """Schedules target as a background task        Returns the scheduled task.        If target is a future or coroutine, equivalent to asyncio.ensure_future        If target is a callable, it is scheduled in the default executor        """        ...    def run_in_foreground(task, *, loop=None):        """Runs event loop in current thread until the given task completes        Returns the result of the task.        For more complex conditions, combine with asyncio.wait()        To include a timeout, combine with asyncio.wait_for()        """        ...run_in_background is akin to invoking a shell command with a trailing"&" - it puts the operation into the background, leaving the currentthread to move on to the next operation (or wait for input at theREPL). When coroutines are scheduled, they won't start running untilyou start a foreground task, while callables delegated to the defaultexecutor will start running immediately.To actually get the *results* of that task, you have to run it in theforeground of the current thread using run_in_foreground - this isakin to bringing a background process to the foreground of a shellsession using "fg".To relate this idea back to some of the examples Sven was discussing,here's how translating some old serialised synchronous code to usethose APIs might look in practice:    # Serial synchronous data loading    def load_and_process_data():        data1 = load_remote_data_set1()        data2 = load_remote_data_set2()        return process_data(data1, data2)    # Parallel asynchronous data loading    def load_and_process_data():        future1 = asyncio.run_in_background(load_remote_data_set1_async())        future2 = asyncio.run_in_background(load_remote_data_set2_async())        data1 = asyncio.run_in_foreground(future1)        data2 = asyncio.run_in_foreground(future2)        return process_data(data1, data2)The application remains fundamentally synchronous, but the asyncioevent loop is exploited to obtain some local concurrency in waitingfor client IO operations.Regards,Nick.P.S. time.sleep() and asyncio.sleep() are rather handy as standins forblocking and non-blocking IO operations. I wish I'd remembered thatearlier :)-- Nick Coghlan   |ncoghlan at gmail.com   |   Brisbane, Australia

• Previous message (by thread):[Python-ideas] Concurrency Modules
• Next message (by thread):[Python-ideas] Learning from the shell in supporting asyncio background calls
• Messages sorted by:[ date ][ thread ][ subject ][ author ]