Frequently Used Arguments¶

To support a wide variety of use cases, thePopen constructor (andthe convenience functions) accept a large number of optional arguments. Formost typical use cases, many of these arguments can be safely left at theirdefault values. The arguments that are most commonly needed are:

args is required for all calls and should be a string, or a sequence ofprogram arguments. Providing a sequence of arguments is generallypreferred, as it allows the module to take care of any required escapingand quoting of arguments (e.g. to permit spaces in file names). If passinga single string, eithershell must beTrue (see below) or elsethe string must simply name the program to be executed without specifyingany arguments.

stdin,stdout andstderr specify the executed program’s standard input,standard output and standard error file handles, respectively. Valid valuesarePIPE,DEVNULL, an existing file descriptor (a positiveinteger), an existing file object with a valid file descriptor, andNone.PIPE indicates that a new pipe to the child should be created.DEVNULL indicates that the special fileos.devnull willbe used. With the default settings ofNone, no redirection will occur;the child’s file handles will be inherited from the parent.Additionally,stderr can beSTDOUT, which indicates that thestderr data from the child process should be captured into the same filehandle as forstdout.

Ifencoding orerrors are specified, ortext (also known asuniversal_newlines) is true,the file objectsstdin,stdout andstderr will be opened in textmode using theencoding anderrors specified in the call or thedefaults forio.TextIOWrapper.

Forstdin, line ending characters'\n' in the input will be convertedto the default line separatoros.linesep. Forstdout andstderr,all line endings in the output will be converted to'\n'. For moreinformation see the documentation of theio.TextIOWrapper classwhen thenewline argument to its constructor isNone.

If text mode is not used,stdin,stdout andstderr will be opened asbinary streams. No encoding or line ending conversion is performed.

New in version 3.6:Addedencoding anderrors parameters.

New in version 3.7:Added thetext parameter as an alias foruniversal_newlines.

Note

The newlines attribute of the file objectsPopen.stdin,Popen.stdout andPopen.stderr are not updated bythePopen.communicate() method.

Ifshell isTrue, the specified command will be executed throughthe shell. This can be useful if you are using Python primarily for theenhanced control flow it offers over most system shells and still wantconvenient access to other shell features such as shell pipes, filenamewildcards, environment variable expansion, and expansion of~ to auser’s home directory. However, note that Python itself offersimplementations of many shell-like features (in particular,glob,fnmatch,os.walk(),os.path.expandvars(),os.path.expanduser(), andshutil).

Changed in version 3.3:Whenuniversal_newlines isTrue, the class uses the encodinglocale.getpreferredencoding(False)instead oflocale.getpreferredencoding(). See theio.TextIOWrapper class for more information on this change.

Note

Read theSecurity Considerations section before usingshell=True.

These options, along with all of the other options, are described in moredetail in thePopen constructor documentation.

Popen Constructor¶

The underlying process creation and management in this module is handled bythePopen class. It offers a lot of flexibility so that developersare able to handle the less common cases not covered by the conveniencefunctions.

classsubprocess.Popen(args,bufsize=-1,executable=None,stdin=None,stdout=None,stderr=None,preexec_fn=None,close_fds=True,shell=False,cwd=None,env=None,universal_newlines=None,startupinfo=None,creationflags=0,restore_signals=True,start_new_session=False,pass_fds=(),*,group=None,extra_groups=None,user=None,umask=-1,encoding=None,errors=None,text=None)¶

Execute a child program in a new process. On POSIX, the class usesos.execvp()-like behavior to execute the child program. On Windows,the class uses the WindowsCreateProcess() function. The arguments toPopen are as follows.

args should be a sequence of program arguments or else a single stringorpath-like object.By default, the program to execute is the first item inargs ifargs isa sequence. Ifargs is a string, the interpretation isplatform-dependent and described below. See theshell andexecutablearguments for additional differences from the default behavior. Unlessotherwise stated, it is recommended to passargs as a sequence.

An example of passing some arguments to an external programas a sequence is:

Popen(["/usr/bin/git","commit","-m","Fixes a bug."])

On POSIX, ifargs is a string, the string is interpreted as the name orpath of the program to execute. However, this can only be done if notpassing arguments to the program.

Note

It may not be obvious how to break a shell command into a sequence of arguments,especially in complex cases.shlex.split() can illustrate how todetermine the correct tokenization forargs:

>>>importshlex,subprocess>>>command_line=input()/bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'">>>args=shlex.split(command_line)>>>print(args)['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]>>>p=subprocess.Popen(args)# Success!

Note in particular that options (such as-input) and arguments (suchaseggs.txt) that are separated by whitespace in the shell go in separatelist elements, while arguments that need quoting or backslash escaping whenused in the shell (such as filenames containing spaces or theecho commandshown above) are single list elements.

On Windows, ifargs is a sequence, it will be converted to a string in amanner described inConverting an argument sequence to a string on Windows. This is becausethe underlyingCreateProcess() operates on strings.

Changed in version 3.6:args parameter accepts apath-like object ifshell isFalse and a sequence containing path-like objects on POSIX.

Changed in version 3.8:args parameter accepts apath-like object ifshell isFalse and a sequence containing bytes and path-like objectson Windows.

Theshell argument (which defaults toFalse) specifies whether to usethe shell as the program to execute. Ifshell isTrue, it isrecommended to passargs as a string rather than as a sequence.

On POSIX withshell=True, the shell defaults to/bin/sh. Ifargs is a string, the string specifies the commandto execute through the shell. This means that the string must beformatted exactly as it would be when typed at the shell prompt. Thisincludes, for example, quoting or backslash escaping filenames with spaces inthem. Ifargs is a sequence, the first item specifies the command string, andany additional items will be treated as additional arguments to the shellitself. That is to say,Popen does the equivalent of:

Popen(['/bin/sh','-c',args[0],args[1],...])

On Windows withshell=True, theCOMSPEC environment variablespecifies the default shell. The only time you need to specifyshell=True on Windows is when the command you wish to execute is builtinto the shell (e.g.dir orcopy). You do not needshell=True to run a batch file or console-based executable.

Note

Read theSecurity Considerations section before usingshell=True.

bufsize will be supplied as the corresponding argument to theopen() function when creating the stdin/stdout/stderr pipefile objects:

• 0 means unbuffered (read and write are onesystem call and can return short)

• 1 means line buffered(only usable ifuniversal_newlines=True i.e., in a text mode)

• any other positive value means use a buffer of approximately thatsize

• negative bufsize (the default) means the system default ofio.DEFAULT_BUFFER_SIZE will be used.

Changed in version 3.3.1:bufsize now defaults to -1 to enable buffering by default to match thebehavior that most code expects. In versions prior to Python 3.2.4 and3.3.1 it incorrectly defaulted to0 which was unbufferedand allowed short reads. This was unintentional and did not match thebehavior of Python 2 as most code expected.

Theexecutable argument specifies a replacement program to execute. Itis very seldom needed. Whenshell=False,executable replaces theprogram to execute specified byargs. However, the originalargs isstill passed to the program. Most programs treat the program specifiedbyargs as the command name, which can then be different from the programactually executed. On POSIX, theargs namebecomes the display name for the executable in utilities such asps. Ifshell=True, on POSIX theexecutable argumentspecifies a replacement shell for the default/bin/sh.

Changed in version 3.6:executable parameter accepts apath-like object on POSIX.

Changed in version 3.8:executable parameter accepts a bytes andpath-like objecton Windows.

Changed in version 3.9.17:Changed Windows shell search order forshell=True. The currentdirectory and%PATH% are replaced with%COMSPEC% and%SystemRoot%\System32\cmd.exe. As a result, dropping amalicious program namedcmd.exe into a current directory nolonger works.

stdin,stdout andstderr specify the executed program’s standard input,standard output and standard error file handles, respectively. Valid valuesarePIPE,DEVNULL, an existing file descriptor (a positiveinteger), an existingfile object with a valid file descriptor,andNone.PIPE indicates that a new pipe to the child shouldbe created.DEVNULL indicates that the special fileos.devnull will be used. With the default settings ofNone,no redirection will occur; the child’s file handles will be inherited fromthe parent. Additionally,stderr can beSTDOUT, which indicatesthat the stderr data from the applications should be captured into the samefile handle as for stdout.

Ifpreexec_fn is set to a callable object, this object will be called in thechild process just before the child is executed.(POSIX only)

Warning

Thepreexec_fn parameter is not safe to use in the presence of threadsin your application. The child process could deadlock before exec iscalled.If you must use it, keep it trivial! Minimize the number of librariesyou call into.

Note

If you need to modify the environment for the child use theenvparameter rather than doing it in apreexec_fn.Thestart_new_session parameter can take the place of a previouslycommon use ofpreexec_fn to call os.setsid() in the child.

Changed in version 3.8:Thepreexec_fn parameter is no longer supported in subinterpreters.The use of the parameter in a subinterpreter raisesRuntimeError. The new restriction may affect applications thatare deployed in mod_wsgi, uWSGI, and other embedded environments.

Ifclose_fds is true, all file descriptors except0,1 and2 will be closed before the child process is executed. Otherwisewhenclose_fds is false, file descriptors obey their inheritable flagas described inInheritance of File Descriptors.

On Windows, ifclose_fds is true then no handles will be inherited by thechild process unless explicitly passed in thehandle_list element ofSTARTUPINFO.lpAttributeList, or by standard handle redirection.

Changed in version 3.2:The default forclose_fds was changed fromFalse towhat is described above.

Changed in version 3.7:On Windows the default forclose_fds was changed fromFalse toTrue when redirecting the standard handles. It’s now possible tosetclose_fds toTrue when redirecting the standard handles.

pass_fds is an optional sequence of file descriptors to keep openbetween the parent and child. Providing anypass_fds forcesclose_fds to beTrue. (POSIX only)

Changed in version 3.2:Thepass_fds parameter was added.

Ifcwd is notNone, the function changes the working directory tocwd before executing the child.cwd can be a string, bytes orpath-like object. In particular, the functionlooks forexecutable (or for the first item inargs) relative tocwdif the executable path is a relative path.

Changed in version 3.6:cwd parameter accepts apath-like object on POSIX.

Changed in version 3.7:cwd parameter accepts apath-like object on Windows.

Changed in version 3.8:cwd parameter accepts a bytes object on Windows.

Ifrestore_signals is true (the default) all signals that Python has set toSIG_IGN are restored to SIG_DFL in the child process before the exec.Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals.(POSIX only)

Changed in version 3.2:restore_signals was added.

Ifstart_new_session is true the setsid() system call will be made in thechild process prior to the execution of the subprocess. (POSIX only)

Changed in version 3.2:start_new_session was added.

Ifgroup is notNone, the setregid() system call will be made in thechild process prior to the execution of the subprocess. If the providedvalue is a string, it will be looked up viagrp.getgrnam() andthe value ingr_gid will be used. If the value is an integer, itwill be passed verbatim. (POSIX only)

Availability: POSIX

New in version 3.9.

Ifextra_groups is notNone, the setgroups() system call will bemade in the child process prior to the execution of the subprocess.Strings provided inextra_groups will be looked up viagrp.getgrnam() and the values ingr_gid will be used.Integer values will be passed verbatim. (POSIX only)

Availability: POSIX

New in version 3.9.

Ifuser is notNone, the setreuid() system call will be made in thechild process prior to the execution of the subprocess. If the providedvalue is a string, it will be looked up viapwd.getpwnam() andthe value inpw_uid will be used. If the value is an integer, it willbe passed verbatim. (POSIX only)

Availability: POSIX

New in version 3.9.

Ifumask is not negative, the umask() system call will be made in thechild process prior to the execution of the subprocess.

Availability: POSIX

New in version 3.9.

Ifenv is notNone, it must be a mapping that defines the environmentvariables for the new process; these are used instead of the defaultbehavior of inheriting the current process’ environment.

Note

If specified,env must provide any variables required for the program toexecute. On Windows, in order to run aside-by-side assembly thespecifiedenvmust include a validSystemRoot.

Ifencoding orerrors are specified, ortext is true, the file objectsstdin,stdout andstderr are opened in text mode with the specifiedencoding anderrors, as described above inFrequently Used Arguments.Theuniversal_newlines argument is equivalent totext and is providedfor backwards compatibility. By default, file objects are opened in binary mode.

New in version 3.6:encoding anderrors were added.

New in version 3.7:text was added as a more readable alias foruniversal_newlines.

If given,startupinfo will be aSTARTUPINFO object, which ispassed to the underlyingCreateProcess function.creationflags, if given, can be one or more of the following flags:

• CREATE_NEW_CONSOLE

• CREATE_NEW_PROCESS_GROUP

• ABOVE_NORMAL_PRIORITY_CLASS

• BELOW_NORMAL_PRIORITY_CLASS

• HIGH_PRIORITY_CLASS

• IDLE_PRIORITY_CLASS

• NORMAL_PRIORITY_CLASS

• REALTIME_PRIORITY_CLASS

• CREATE_NO_WINDOW

• DETACHED_PROCESS

• CREATE_DEFAULT_ERROR_MODE

• CREATE_BREAKAWAY_FROM_JOB

Popen objects are supported as context managers via thewith statement:on exit, standard file descriptors are closed, and the process is waited for.

withPopen(["ifconfig"],stdout=PIPE)asproc:log.write(proc.stdout.read())

Popen and the other functions in this module that use it raise anauditing eventsubprocess.Popen with argumentsexecutable,args,cwd, andenv. The value forargsmay be a single string or a list of strings, depending on platform.

Changed in version 3.2:Added context manager support.

Changed in version 3.6:Popen destructor now emits aResourceWarning warning if the childprocess is still running.

Changed in version 3.8:Popen can useos.posix_spawn() in some cases for betterperformance. On Windows Subsystem for Linux and QEMU User Emulation,Popen constructor usingos.posix_spawn() no longer raise anexception on errors like missing program, but the child process failswith a non-zeroreturncode.

Exceptions¶

Exceptions raised in the child process, before the new program has started toexecute, will be re-raised in the parent.

The most common exception raised isOSError. This occurs, for example,when trying to execute a non-existent file. Applications should prepare forOSError exceptions. Note that, whenshell=True,OSErrorwill be raised by the child only if the selected shell itself was not found.To determine if the shell failed to find the requested application, it isnecessary to check the return code or output from the subprocess.

AValueError will be raised ifPopen is called with invalidarguments.

check_call() andcheck_output() will raiseCalledProcessError if the called process returns a non-zero returncode.

All of the functions and methods that accept atimeout parameter, such ascall() andPopen.communicate() will raiseTimeoutExpired ifthe timeout expires before the process exits.

Exceptions defined in this module all inherit fromSubprocessError.

New in version 3.3:TheSubprocessError base class was added.

Windows Constants¶

Thesubprocess module exposes the following constants.

subprocess.STD_INPUT_HANDLE¶

The standard input device. Initially, this is the console input buffer,CONIN$.

subprocess.STD_OUTPUT_HANDLE¶

The standard output device. Initially, this is the active console screenbuffer,CONOUT$.

subprocess.STD_ERROR_HANDLE¶

The standard error device. Initially, this is the active console screenbuffer,CONOUT$.

subprocess.SW_HIDE¶

Hides the window. Another window will be activated.

subprocess.STARTF_USESTDHANDLES¶

Specifies that theSTARTUPINFO.hStdInput,STARTUPINFO.hStdOutput, andSTARTUPINFO.hStdError attributescontain additional information.

subprocess.STARTF_USESHOWWINDOW¶

Specifies that theSTARTUPINFO.wShowWindow attribute containsadditional information.

subprocess.CREATE_NEW_CONSOLE¶

The new process has a new console, instead of inheriting its parent’sconsole (the default).

subprocess.CREATE_NEW_PROCESS_GROUP¶

APopencreationflags parameter to specify that a new processgroup will be created. This flag is necessary for usingos.kill()on the subprocess.

This flag is ignored ifCREATE_NEW_CONSOLE is specified.

subprocess.ABOVE_NORMAL_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have an above average priority.

New in version 3.7.

subprocess.BELOW_NORMAL_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have a below average priority.

New in version 3.7.

subprocess.HIGH_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have a high priority.

New in version 3.7.

subprocess.IDLE_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have an idle (lowest) priority.

New in version 3.7.

subprocess.NORMAL_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have an normal priority. (default)

New in version 3.7.

subprocess.REALTIME_PRIORITY_CLASS¶

APopencreationflags parameter to specify that a new processwill have realtime priority.You should almost never use REALTIME_PRIORITY_CLASS, because this interruptssystem threads that manage mouse input, keyboard input, and background diskflushing. This class can be appropriate for applications that “talk” directlyto hardware or that perform brief tasks that should have limited interruptions.

New in version 3.7.

subprocess.CREATE_NO_WINDOW¶

APopencreationflags parameter to specify that a new processwill not create a window.

New in version 3.7.

subprocess.DETACHED_PROCESS¶

APopencreationflags parameter to specify that a new processwill not inherit its parent’s console.This value cannot be used with CREATE_NEW_CONSOLE.

New in version 3.7.

subprocess.CREATE_DEFAULT_ERROR_MODE¶

APopencreationflags parameter to specify that a new processdoes not inherit the error mode of the calling process. Instead, the newprocess gets the default error mode.This feature is particularly useful for multithreaded shell applicationsthat run with hard errors disabled.

New in version 3.7.

subprocess.CREATE_BREAKAWAY_FROM_JOB¶

APopencreationflags parameter to specify that a new processis not associated with the job.

New in version 3.7.

Replacing/bin/sh shell command substitution¶

output=$(mycmdmyarg)

becomes:

output=check_output(["mycmd","myarg"])

Replacing shell pipeline¶

output=$(dmesg|grephda)

becomes:

p1=Popen(["dmesg"],stdout=PIPE)p2=Popen(["grep","hda"],stdin=p1.stdout,stdout=PIPE)p1.stdout.close()# Allow p1 to receive a SIGPIPE if p2 exits.output=p2.communicate()[0]

Thep1.stdout.close() call after starting the p2 is important in order forp1 to receive a SIGPIPE if p2 exits before p1.

Alternatively, for trusted input, the shell’s own pipeline support may stillbe used directly:

output=$(dmesg|grephda)

becomes:

output=check_output("dmesg | grep hda",shell=True)

Replacingos.system()¶

sts=os.system("mycmd"+" myarg")# becomesretcode=call("mycmd"+" myarg",shell=True)

Notes:

• Calling the program through the shell is usually not required.

• Thecall() return value is encoded differently to that ofos.system().

• Theos.system() function ignores SIGINT and SIGQUIT signals whilethe command is running, but the caller must do this separately whenusing thesubprocess module.

A more realistic example would look like this:

try:retcode=call("mycmd"+" myarg",shell=True)ifretcode<0:print("Child was terminated by signal",-retcode,file=sys.stderr)else:print("Child returned",retcode,file=sys.stderr)exceptOSErrorase:print("Execution failed:",e,file=sys.stderr)

Replacing theos.spawn family¶

P_NOWAIT example:

pid=os.spawnlp(os.P_NOWAIT,"/bin/mycmd","mycmd","myarg")==>pid=Popen(["/bin/mycmd","myarg"]).pid

P_WAIT example:

retcode=os.spawnlp(os.P_WAIT,"/bin/mycmd","mycmd","myarg")==>retcode=call(["/bin/mycmd","myarg"])

Vector example:

os.spawnvp(os.P_NOWAIT,path,args)==>Popen([path]+args[1:])

Environment example:

os.spawnlpe(os.P_NOWAIT,"/bin/mycmd","mycmd","myarg",env)==>Popen(["/bin/mycmd","myarg"],env={"PATH":"/usr/bin"})

Replacingos.popen(),os.popen2(),os.popen3()¶

(child_stdin,child_stdout)=os.popen2(cmd,mode,bufsize)==>p=Popen(cmd,shell=True,bufsize=bufsize,stdin=PIPE,stdout=PIPE,close_fds=True)(child_stdin,child_stdout)=(p.stdin,p.stdout)

(child_stdin,child_stdout,child_stderr)=os.popen3(cmd,mode,bufsize)==>p=Popen(cmd,shell=True,bufsize=bufsize,stdin=PIPE,stdout=PIPE,stderr=PIPE,close_fds=True)(child_stdin,child_stdout,child_stderr)=(p.stdin,p.stdout,p.stderr)

(child_stdin,child_stdout_and_stderr)=os.popen4(cmd,mode,bufsize)==>p=Popen(cmd,shell=True,bufsize=bufsize,stdin=PIPE,stdout=PIPE,stderr=STDOUT,close_fds=True)(child_stdin,child_stdout_and_stderr)=(p.stdin,p.stdout)

Return code handling translates as follows:

pipe=os.popen(cmd,'w')...rc=pipe.close()ifrcisnotNoneandrc>>8:print("There were some errors")==>process=Popen(cmd,stdin=PIPE)...process.stdin.close()ifprocess.wait()!=0:print("There were some errors")

Replacing functions from thepopen2 module¶

(child_stdout,child_stdin)=popen2.popen2("somestring",bufsize,mode)==>p=Popen("somestring",shell=True,bufsize=bufsize,stdin=PIPE,stdout=PIPE,close_fds=True)(child_stdout,child_stdin)=(p.stdout,p.stdin)

(child_stdout,child_stdin)=popen2.popen2(["mycmd","myarg"],bufsize,mode)==>p=Popen(["mycmd","myarg"],bufsize=bufsize,stdin=PIPE,stdout=PIPE,close_fds=True)(child_stdout,child_stdin)=(p.stdout,p.stdin)

popen2.Popen3 andpopen2.Popen4 basically work assubprocess.Popen, except that:

• Popen raises an exception if the execution fails.

• Thecapturestderr argument is replaced with thestderr argument.

• stdin=PIPE andstdout=PIPE must be specified.

• popen2 closes all file descriptors by default, but you have to specifyclose_fds=True withPopen to guarantee this behavior onall platforms or past Python versions.

Converting an argument sequence to a string on Windows¶

On Windows, anargs sequence is converted to a string that can be parsedusing the following rules (which correspond to the rules used by the MS Cruntime):

1. Arguments are delimited by white space, which is either aspace or a tab.

2. A string surrounded by double quotation marks isinterpreted as a single argument, regardless of white spacecontained within. A quoted string can be embedded in anargument.

3. A double quotation mark preceded by a backslash isinterpreted as a literal double quotation mark.

4. Backslashes are interpreted literally, unless theyimmediately precede a double quotation mark.

5. If backslashes immediately precede a double quotation mark,every pair of backslashes is interpreted as a literalbackslash. If the number of backslashes is odd, the lastbackslash escapes the next double quotation mark asdescribed in rule 3.