Interrupted system calls

On POSIX systems, signals are common. Code calling system calls must beprepared to handle them. Examples of signals:

• The most common signal isSIGINT, the signal sent when CTRL+c ispressed. By default, Python raises aKeyboardInterrupt exceptionwhen this signal is received.
• When running subprocesses, theSIGCHLD signal is sent when achild process exits.
• Resizing the terminal sends theSIGWINCH signal to theapplications running in the terminal.
• Putting the application in background (ex: press CTRL-z and thentype thebg command) sends theSIGCONT signal.

Writing a C signal handler is difficult: only “async-signal-safe”functions can be called (for example,printf() andmalloc()are not async-signal safe), and there are issues with reentrancy.Therefore, when a signal is received by a process during the executionof a system call, the system call can fail with theEINTR error togive the program an opportunity to handle the signal without therestriction on signal-safe functions.

This behaviour is system-dependent: on certain systems, using theSA_RESTART flag, some system calls are retried automatically insteadof failing withEINTR. Regardless, Python’ssignal.signal()function clears theSA_RESTART flag when setting the signal handler:all system calls will probably fail withEINTR in Python.

Since receiving a signal is a non-exceptional occurrence, robust POSIX codemust be prepared to handleEINTR (which, in most cases, meansretry in a loop in the hope that the call eventually succeeds).Without special support from Python, this can make application codemuch more verbose than it needs to be.

Status in Python 3.4

In Python 3.4, handling theInterruptedError exception (EINTR’sdedicated exception class) is duplicated at every call site on a case-by-casebasis. Only a few Python modules actually handle this exception,and fixes usually took several years to cover a whole module. Example ofcode retryingfile.read() onInterruptedError:

whileTrue:try:data=file.read(size)breakexceptInterruptedError:continue

List of Python modules in the standard library which handleInterruptedError:

• asyncio
• asyncore
• io,_pyio
• multiprocessing
• selectors
• socket
• socketserver
• subprocess

Other programming languages like Perl, Java and Go retry system callsfailing withEINTR at a lower level, so that libraries and applicationsneedn’t bother.

Use Case 1: Don’t Bother With Signals

In most cases, you don’t want to be interrupted by signals and youdon’t expect to getInterruptedError exceptions. For example, doyou really want to write such complex code for a “Hello World”example?

whileTrue:try:print("Hello World")breakexceptInterruptedError:continue

InterruptedError can happen in unexpected places. For example,os.close() andFileIO.close() may raiseInterruptedError:see the articleclose() and EINTR.

ThePython issues related to EINTR section below gives examples ofbugs caused byEINTR.

The expectation in this use case is that Python hides theInterruptedError and retries system calls automatically.

Use Case 2: Be notified of signals as soon as possible

Sometimes yet, you expect some signals and you want to handle them assoon as possible. For example, you may want to immediately quit aprogram using theCTRL+c keyboard shortcut.

Besides, some signals are not interesting and should not disrupt theapplication. There are two options to interrupt an application ononlysome signals:

• Set up a custom signal handler which raises an exception, such asKeyboardInterrupt forSIGINT.
• Use a I/O multiplexing function likeselect() together with Python’ssignal wakeup file descriptor: see the functionsignal.set_wakeup_fd().

The expectation in this use case is for the Python signal handler to beexecuted timely, and the system call to fail if the handler raised anexception – otherwise restart.

Modified functions

Example of standard library functions that need to be modified to complywith this PEP:

• open(),os.open(),io.open()
• functions of thefaulthandler module
• os functions:
  • os.fchdir()
  • os.fchmod()
  • os.fchown()
  • os.fdatasync()
  • os.fstat()
  • os.fstatvfs()
  • os.fsync()
  • os.ftruncate()
  • os.mkfifo()
  • os.mknod()
  • os.posix_fadvise()
  • os.posix_fallocate()
  • os.pread()
  • os.pwrite()
  • os.read()
  • os.readv()
  • os.sendfile()
  • os.wait3()
  • os.wait4()
  • os.wait()
  • os.waitid()
  • os.waitpid()
  • os.write()
  • os.writev()
  • special cases:os.close() andos.dup2() now ignoreEINTR error,the syscall is not retried
• select.select(),select.poll.poll(),select.epoll.poll(),select.kqueue.control(),select.devpoll.poll()
• socket.socket() methods:
  • accept()
  • connect() (except for non-blocking sockets)
  • recv()
  • recvfrom()
  • recvmsg()
  • send()
  • sendall()
  • sendmsg()
  • sendto()
• signal.sigtimedwait(),signal.sigwaitinfo()
• time.sleep()

(Note: theselector module already retries onInterruptedError, but itdoesn’t recompute the timeout yet)

os.close,close() methods andos.dup2() are a special case: theywill ignoreEINTR instead of retrying. The reason is complex but involvesbehaviour under Linux and the fact that the file descriptor may really beclosed even if EINTR is returned. See articles:

• Returning EINTR from close()
• (LKML) Re: [patch 7/7] uml: retry host close() on EINTR
• close() and EINTR

Thesocket.socket.connect() method does not retryconnect() fornon-blocking sockets if it is interrupted by a signal (fails withEINTR).The connection runs asynchronously in background. The caller is responsibleto wait until the socket becomes writable (ex: usingselect.select())and then callsocket.socket.getsockopt(socket.SOL_SOCKET,socket.SO_ERROR)to check if the connection succeeded (getsockopt() returns0) or failed.

InterruptedError handling

Since interrupted system calls are automatically retried, theInterruptedError exception should not occur anymore when calling thosesystem calls. Therefore, manual handling ofInterruptedError asdescribed inStatus in Python 3.4 can be removed, which will simplifystandard library code.

Wakeup file descriptor

Since Python 3.3,signal.set_wakeup_fd() writes the signal numberinto the file descriptor, whereas it only wrote a null byte before.It becomes possible to distinguish between signals using the wakeup filedescriptor.

Linux has asignalfd() system call which provides more information oneach signal. For example, it’s possible to know the pid and uid who sentthe signal. This function is not exposed in Python yet (seeissue 12304).

On Unix, theasyncio module uses the wakeup file descriptor towake up its event loop.

Multithreading

A C signal handler can be called from any thread, but Pythonsignal handlers will always be called in the main Python thread.

Python’s C API provides thePyErr_SetInterrupt() function which callstheSIGINT signal handler in order to interrupt the main Python thread.

Signals on Windows

Links