*pythongh-55454: Add IMAP4 IDLE support to imaplibThis extends imaplib with support for the rfc2177 IMAP IDLE command,as requested inpython#55454.  It allows events to be pushed to a client asthey occur, rather than having to continually poll for mailbox changes.The interface is a new idle() method, which returns an iterable contextmanager.  Entering the context starts IDLE mode, during which events(untagged responses) can be retrieved using the iteration protocol.Exiting the context sends DONE to the server, ending IDLE mode.An optional time limit for the IDLE session is supported, for use withservers that impose an inactivity timeout.The context manager also offers a burst() method, designed for programswishing to process events in batch rather than one at a time.Notable differences from other implementations:- It's an extension to imaplib, rather than a replacement.- It doesn't introduce additional threads.- It doesn't impose new requirements on the use of imaplib's existing methods.- It passes the unit tests in CPython's test/test_imaplib.py module  (and adds new ones).- It works on Windows, Linux, and other unix-like systems.- It makes IDLE available on all of imaplib's client variants  (including IMAP4_stream).- The interface is pythonic and easy to use.Caveats:- Due to a Windows limitation, the special case of IMAP4_stream running  on Windows lacks a duration/timeout feature. (This is the stdin/stdout  pipe connection variant; timeouts work fine for socket-based  connections, even on Windows.) I have documented it where appropriate.- The file-like imaplib instance attributes are changed from buffered to  unbuffered mode. This could potentially break any client code that  uses those objects directly without expecting partial reads/writes.  However, these attributes are undocumented. As such, I think (and  PEP 8 confirms) that they are fair game for changes.https://peps.python.org/pep-0008/#public-and-internal-interfacesUsage examples:python#55454 (comment)Original discussion:https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272Earlier requests and suggestions:python#55454https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/*pythongh-55454: Clarify imaplib idle() docs- Add example idle response tuples, to make the minor difference from other  imaplib response tuples more obvious.- Merge the idle context manager's burst() method docs with the IMAP  object's idle() method docs, for easier understanding.- Upgrade the Windows note regarding lack of pipe timeouts to a warning.- Rephrase various things for clarity.* docs: words instead of <=Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: improve style in an exampleCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: grammatical editCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs consistencyCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* comment -> docstringCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: refer to imaplib as "this module"Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: simplify & clarify idle debug messageCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: elaborate in idle context manager comment* imaplib: re-raise BaseException instead of bare exceptCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: convert private doc string to comment* docs: correct mistake in imaplib exampleThis is a correction to8077f2e, whichchanged a variable name in only one place and broke the subsequentreference to it, departed from the naming convention used in the rest ofthe module, and shadowed the type() builtin along the way.* imaplib: simplify example code in doc stringThis is for consistency with the documentation change in8077f2eand subsequent correction in013bbf1.* imaplib: rename _Idler to Idler, update its docs* imaplib: add comment in Idler._pop()Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: remove unnecessary blank lineCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: comment on use of unbuffered pipes* docs: imaplib: use the reStructuredText :class: roleCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* Revert "docs: imaplib: use the reStructuredText :class: role"This reverts commitf385e44, because ittriggers CI failures in the docs by referencing a class that is(deliberately) undocumented.* docs: imaplib: use the reST :class: role, escapedThis is a different approach tof385e44, which was reverted forcreating dangling link references.By prefixing the reStructuredText role target with a ! we disableconversion to a link, thereby passing continuous integration checkseven though the referenced class is deliberately absent from thedocumentation.* docs: refer to IMAP4 IDLE instead of just IDLEThis clarifies that we are referring to the email protocol, not the editor with the same name.Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>* imaplib: IDLE -> IMAP4 IDLE in exception messageCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: imaplib idle() phrasing and linking tweaks* docs: imaplib: avoid linking to an invalid targetThis reverts and rephrases part ofa3f21cdwhich created links to a method on a deliberately undocumented class.The links didn't work consistently, and caused sphinx warnings thatbroke cpython's continuous integration tests.* imaplib: update test after recent exception changeThis fixes a test that was broken by changing an exception inb01de95* imaplib: rename idle() dur argument to duration* imaplib: bytes.index() -> bytes.find()This makes it more obvious which statement triggers the branch.* imaplib: remove no-longer-necessary statementCo-authored-by: Martin Panter <vadmium@users.noreply.github.com>* docs: imaplib: concise & valid method linksThe burst() method is a little tricky to link in restructuredText, dueto quirks of its parent class.  This syntax allows sphinx to generateworking links without generating warnings (which break continuousintegration) and without burdening the reader with unimportant namespacequalifications.  It makes the reST source ugly, but few people readthe reST source, so it's a tolerable tradeoff.* imaplib: note data types present in IDLE responses* docs: imaplib: add comma to reST changes headerCo-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>* imaplib: sync doc strings with reST docs* docs: imaplib: minor Idler clarifications* imaplib: idle: emit (type, [data, ...]) tuplesThis allows our iterator to emit untagged responses that contain literalstrings in the same way that imaplib's existing methods do, while stillemitting exactly one whole response per iteration.* imaplib: while/yield instead of yield from iter()* imaplib: idle: use deadline idiom when iteratingThis simplifies the code, and avoids idle duration drift from time spentprocessing each iteration.* docs: imaplib: state duration/interval arg types* docs: imaplib: minor rephrasing of a sentence* docs: imaplib: reposition a paragraphThis might improve readability, especially when encountering Idler.burst()for the first time.* docs: imaplib: wrap long lines in idle() section* docs: imaplib: note: Idler objects require 'with'* docs: imaplib: say that 29 minutes is 1740 seconds* docs: imaplib: mark a paragraph as a 'tip'* docs: imaplib: rephrase reference to MS Windows* imaplib: end doc string titles with a period* imaplib: idle: socket timeouts instead of select()IDLE timeouts were originally implemented using select() afterchecking for the presence of already-buffered data.That allowed timeouts on pipe connetions like IMAP4_stream.However, it seemed possible that SSL data arriving without anyIMAP data afterward could cause select() to indicate availableapplication data when there was none, leading to a read() callthat would block with no timeout. It was unclear under whatconditions this would happen in practice. This change switchesto socket timeouts instead of select(), just to be safe.This also reverts IMAP4_stream changes that were made to support IDLEtimeouts, since our new implementation only supports socket connections.* imaplib: Idler: rename private state attributes* imaplib: rephrase a comment in example code* docs: imaplib: idle: use Sphinx code-block:: pycon* docs: whatsnew: imaplib: reformat IMAP4.idle entry* imaplib: idle: make doc strings briefSince we generally rely on the reST/html documentation for details, wecan keep these doc strings short. This matches the module's existing docstring style and avoids having to sync small changes between two files.* imaplib: Idler: split assert into two statements* imaplib: Idler: move assignment out of try: block* imaplib: Idler: move __exit__() for readability* imaplib: Idler: move __next__() for readability* imaplib: test: make IdleCmdHandler a global class* docs: imaplib: idle: collapse double-spaces* imaplib: warn on use of undocumented 'file' attr* imaplib: revert import reformattingSince we no longer import platform or selectors, the original importstatement style can be restored, reducing the footprint of PRpython#122542.* imaplib: restore original exception msg formattingThis reduces the footprint of PRpython#122542.* docs: imaplib: idle: versionadded:: next* imaplib: move import statement to where it's usedThis import is only needed if external code tries to use an attributethat it shouldn't be using. Making it a local import reduces moduleloading time in supported cases.* imaplib test: RuntimeWarning on IMAP4.file access* imaplib: use stacklevel=2 in warnings.warn()* imaplib test: simplify IMAP4.file warning test* imaplib test: pre-idle-continuation response* imaplib test: post-done untagged response* imaplib: downgrade idle-denied exception to errorThis makes it easier for client code to distinguish a temporaryrejection of the IDLE command from a server responding incorrectly toIDLE.* imaplib: simplify check for socket object* imaplib: narrow the scope of IDLE socket timeoutsIf an IDLE duration or burst() was in use, and an unsolicited responsecontained a literal string, and crossed a packet boundary, and thesubsequent packet was delayed beyond the IDLE feature's time limit, thetimeout would leave the incoming protocol stream in a bad state (withthe tail of that response appearing where the start of a response isexpected).This change moves the IDLE socket timeout to cover only the startof a response, so it can no longer cause that problem.* imaplib: preserve partial reads on exceptionThis ensures that short IDLE durations / burst() intervalswon't risk corrupting response lines that span multiple packets.* imaplib: read/readline: save multipart buffer tailFor resilience if read() or readline() ever complete with more than onebytes object remaining in the buffer. This is not expected to happen,but it seems wise to be prepared for a future change making it possible.* imaplib: use TimeoutError subclass only if needed* doc: imaplib: elaborate on IDLE response delivery* doc: imaplib: elaborate in note re: IMAP4.response* imaplib: comment on benefit of reading in chunksOur read() implementation designed to support IDLE replaces the one fromPRpython#119514, fixing the same problem it was addressing. The tests that itadded are preserved.* imaplib: readline(): treat ConnectionError as EOF---------Co-authored-by: Gregory P. Smith <greg@krypto.org>Co-authored-by: Peter Bierma <zintensitydev@gmail.com>Co-authored-by: Guido van Rossum <guido@python.org>Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

*pythongh-55454: Add IMAP4 IDLE support to imaplibThis extends imaplib with support for the rfc2177 IMAP IDLE command,as requested inpython#55454.  It allows events to be pushed to a client asthey occur, rather than having to continually poll for mailbox changes.The interface is a new idle() method, which returns an iterable contextmanager.  Entering the context starts IDLE mode, during which events(untagged responses) can be retrieved using the iteration protocol.Exiting the context sends DONE to the server, ending IDLE mode.An optional time limit for the IDLE session is supported, for use withservers that impose an inactivity timeout.The context manager also offers a burst() method, designed for programswishing to process events in batch rather than one at a time.Notable differences from other implementations:- It's an extension to imaplib, rather than a replacement.- It doesn't introduce additional threads.- It doesn't impose new requirements on the use of imaplib's existing methods.- It passes the unit tests in CPython's test/test_imaplib.py module  (and adds new ones).- It works on Windows, Linux, and other unix-like systems.- It makes IDLE available on all of imaplib's client variants  (including IMAP4_stream).- The interface is pythonic and easy to use.Caveats:- Due to a Windows limitation, the special case of IMAP4_stream running  on Windows lacks a duration/timeout feature. (This is the stdin/stdout  pipe connection variant; timeouts work fine for socket-based  connections, even on Windows.) I have documented it where appropriate.- The file-like imaplib instance attributes are changed from buffered to  unbuffered mode. This could potentially break any client code that  uses those objects directly without expecting partial reads/writes.  However, these attributes are undocumented. As such, I think (and  PEP 8 confirms) that they are fair game for changes.https://peps.python.org/pep-0008/#public-and-internal-interfacesUsage examples:python#55454 (comment)Original discussion:https://discuss.python.org/t/gauging-interest-in-my-imap4-idle-implementation-for-imaplib/59272Earlier requests and suggestions:python#55454https://mail.python.org/archives/list/python-ideas@python.org/thread/C4TVEYL5IBESQQPPS5GBR7WFBXCLQMZ2/*pythongh-55454: Clarify imaplib idle() docs- Add example idle response tuples, to make the minor difference from other  imaplib response tuples more obvious.- Merge the idle context manager's burst() method docs with the IMAP  object's idle() method docs, for easier understanding.- Upgrade the Windows note regarding lack of pipe timeouts to a warning.- Rephrase various things for clarity.* docs: words instead of <=Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: improve style in an exampleCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: grammatical editCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs consistencyCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* comment -> docstringCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: refer to imaplib as "this module"Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: simplify & clarify idle debug messageCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: elaborate in idle context manager comment* imaplib: re-raise BaseException instead of bare exceptCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: convert private doc string to comment* docs: correct mistake in imaplib exampleThis is a correction to8077f2e, whichchanged a variable name in only one place and broke the subsequentreference to it, departed from the naming convention used in the rest ofthe module, and shadowed the type() builtin along the way.* imaplib: simplify example code in doc stringThis is for consistency with the documentation change in8077f2eand subsequent correction in013bbf1.* imaplib: rename _Idler to Idler, update its docs* imaplib: add comment in Idler._pop()Co-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: remove unnecessary blank lineCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* imaplib: comment on use of unbuffered pipes* docs: imaplib: use the reStructuredText :class: roleCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* Revert "docs: imaplib: use the reStructuredText :class: role"This reverts commitf385e44, because ittriggers CI failures in the docs by referencing a class that is(deliberately) undocumented.* docs: imaplib: use the reST :class: role, escapedThis is a different approach tof385e44, which was reverted forcreating dangling link references.By prefixing the reStructuredText role target with a ! we disableconversion to a link, thereby passing continuous integration checkseven though the referenced class is deliberately absent from thedocumentation.* docs: refer to IMAP4 IDLE instead of just IDLEThis clarifies that we are referring to the email protocol, not the editor with the same name.Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>* imaplib: IDLE -> IMAP4 IDLE in exception messageCo-authored-by: Peter Bierma <zintensitydev@gmail.com>* docs: imaplib idle() phrasing and linking tweaks* docs: imaplib: avoid linking to an invalid targetThis reverts and rephrases part ofa3f21cdwhich created links to a method on a deliberately undocumented class.The links didn't work consistently, and caused sphinx warnings thatbroke cpython's continuous integration tests.* imaplib: update test after recent exception changeThis fixes a test that was broken by changing an exception inb01de95* imaplib: rename idle() dur argument to duration* imaplib: bytes.index() -> bytes.find()This makes it more obvious which statement triggers the branch.* imaplib: remove no-longer-necessary statementCo-authored-by: Martin Panter <vadmium@users.noreply.github.com>* docs: imaplib: concise & valid method linksThe burst() method is a little tricky to link in restructuredText, dueto quirks of its parent class.  This syntax allows sphinx to generateworking links without generating warnings (which break continuousintegration) and without burdening the reader with unimportant namespacequalifications.  It makes the reST source ugly, but few people readthe reST source, so it's a tolerable tradeoff.* imaplib: note data types present in IDLE responses* docs: imaplib: add comma to reST changes headerCo-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>* imaplib: sync doc strings with reST docs* docs: imaplib: minor Idler clarifications* imaplib: idle: emit (type, [data, ...]) tuplesThis allows our iterator to emit untagged responses that contain literalstrings in the same way that imaplib's existing methods do, while stillemitting exactly one whole response per iteration.* imaplib: while/yield instead of yield from iter()* imaplib: idle: use deadline idiom when iteratingThis simplifies the code, and avoids idle duration drift from time spentprocessing each iteration.* docs: imaplib: state duration/interval arg types* docs: imaplib: minor rephrasing of a sentence* docs: imaplib: reposition a paragraphThis might improve readability, especially when encountering Idler.burst()for the first time.* docs: imaplib: wrap long lines in idle() section* docs: imaplib: note: Idler objects require 'with'* docs: imaplib: say that 29 minutes is 1740 seconds* docs: imaplib: mark a paragraph as a 'tip'* docs: imaplib: rephrase reference to MS Windows* imaplib: end doc string titles with a period* imaplib: idle: socket timeouts instead of select()IDLE timeouts were originally implemented using select() afterchecking for the presence of already-buffered data.That allowed timeouts on pipe connetions like IMAP4_stream.However, it seemed possible that SSL data arriving without anyIMAP data afterward could cause select() to indicate availableapplication data when there was none, leading to a read() callthat would block with no timeout. It was unclear under whatconditions this would happen in practice. This change switchesto socket timeouts instead of select(), just to be safe.This also reverts IMAP4_stream changes that were made to support IDLEtimeouts, since our new implementation only supports socket connections.* imaplib: Idler: rename private state attributes* imaplib: rephrase a comment in example code* docs: imaplib: idle: use Sphinx code-block:: pycon* docs: whatsnew: imaplib: reformat IMAP4.idle entry* imaplib: idle: make doc strings briefSince we generally rely on the reST/html documentation for details, wecan keep these doc strings short. This matches the module's existing docstring style and avoids having to sync small changes between two files.* imaplib: Idler: split assert into two statements* imaplib: Idler: move assignment out of try: block* imaplib: Idler: move __exit__() for readability* imaplib: Idler: move __next__() for readability* imaplib: test: make IdleCmdHandler a global class* docs: imaplib: idle: collapse double-spaces* imaplib: warn on use of undocumented 'file' attr* imaplib: revert import reformattingSince we no longer import platform or selectors, the original importstatement style can be restored, reducing the footprint of PRpython#122542.* imaplib: restore original exception msg formattingThis reduces the footprint of PRpython#122542.* docs: imaplib: idle: versionadded:: next* imaplib: move import statement to where it's usedThis import is only needed if external code tries to use an attributethat it shouldn't be using. Making it a local import reduces moduleloading time in supported cases.* imaplib test: RuntimeWarning on IMAP4.file access* imaplib: use stacklevel=2 in warnings.warn()* imaplib test: simplify IMAP4.file warning test* imaplib test: pre-idle-continuation response* imaplib test: post-done untagged response* imaplib: downgrade idle-denied exception to errorThis makes it easier for client code to distinguish a temporaryrejection of the IDLE command from a server responding incorrectly toIDLE.* imaplib: simplify check for socket object* imaplib: narrow the scope of IDLE socket timeoutsIf an IDLE duration or burst() was in use, and an unsolicited responsecontained a literal string, and crossed a packet boundary, and thesubsequent packet was delayed beyond the IDLE feature's time limit, thetimeout would leave the incoming protocol stream in a bad state (withthe tail of that response appearing where the start of a response isexpected).This change moves the IDLE socket timeout to cover only the startof a response, so it can no longer cause that problem.* imaplib: preserve partial reads on exceptionThis ensures that short IDLE durations / burst() intervalswon't risk corrupting response lines that span multiple packets.* imaplib: read/readline: save multipart buffer tailFor resilience if read() or readline() ever complete with more than onebytes object remaining in the buffer. This is not expected to happen,but it seems wise to be prepared for a future change making it possible.* imaplib: use TimeoutError subclass only if needed* doc: imaplib: elaborate on IDLE response delivery* doc: imaplib: elaborate in note re: IMAP4.response* imaplib: comment on benefit of reading in chunksOur read() implementation designed to support IDLE replaces the one fromPRpython#119514, fixing the same problem it was addressing. The tests that itadded are preserved.* imaplib: readline(): treat ConnectionError as EOF---------Co-authored-by: Gregory P. Smith <greg@krypto.org>Co-authored-by: Peter Bierma <zintensitydev@gmail.com>Co-authored-by: Guido van Rossum <guido@python.org>Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>