Getting to Know Python’sexec()

Python’s built-inexec() function allows you to execute any piece of Python code. With this function, you can executedynamically generated code. That’s the code that you read, auto-generate, or obtain during your program’s execution. Normally, it’s a string.

Theexec() function takes a piece of code and executes it as your Python interpreter would. Python’sexec() is likeeval() but even more powerful and prone to security issues. Whileeval() can only evaluateexpressions,exec() can execute sequences of statements, as well asimports, function calls and definitions, class definitions and instantiations, and more. Essentially,exec() can execute an entire fully featured Python program.

The signature ofexec() has the following form:

exec(code[,globals[,locals]])

The function executescode, which can be either astring containing valid Python code or acompiled code object.

Ifcode is astring, then it’sparsed as a suite of Python statements, which is then internallycompiled into bytecode, and finallyexecuted, unless a syntax error occurs during the parsing or compilation step. Ifcode holds a compiled code object, then it’s executed directly, making the process a bit more efficient.

Theglobals andlocals arguments allow you to provide dictionaries representing theglobal andlocal namespaces in whichexec() will run the target code.

Theexec() function’sreturn value isNone, probably because not every piece of code has a final, unique, and concrete result. It may just have someside effects. This behavior notably differs fromeval(), which returns the result of the evaluated expression.

To get an initial feeling of howexec() works, you can create a rudimentary Python interpreter with two lines of code:

>>>whileTrue:...exec(input("->> "))...->> print("Hello, World!")Hello, World!->> import thisThe Zen of Python, by Tim PetersBeautiful is better than ugly.Explicit is better than implicit.Simple is better than complex.    ...->> x = 10->> if 1 <= x <= 10: print(f"{x} is between 1 and 10")10 is between 1 and 10

In this example, you use an infinitewhile loop to mimic the behavior of a Pythoninterpreter orREPL. Inside the loop, you useinput() to get the user’s input at the command line. Then you useexec() to process and run the input.

This example showcases what’s arguably the main use case ofexec():executing code that comes to you as a string.

You’ll commonly useexec() when you need to dynamically run code that comes as a string. For example, you can write a program that generates strings containing valid Python code. You can build these strings from parts that you obtain at different moments in your program’s execution. You can also use the user’s input or any other input source to construct these strings.

Once you’ve built the target code as strings, then you can useexec() to execute them as you would execute any Python code.

In this situation, you can rarely be certain of what your strings will contain. That’s one reason whyexec() implies serious security risks. This is particularly true if you’re using untrusted input sources, like a user’s direct input, in building your code.

In programming, a function likeexec() is an incredibly powerful tool because it allows you to write programs that generate and execute new code dynamically. To generate this new code, your programs will use information available at runtime only. To run the code, your programs will useexec().

However, with great power comes great responsibility. Theexec() function implies serioussecurity risks, as you’ll learn soon. So, you should avoid usingexec() most of the time.

In the following sections, you’ll learn howexec() works and how to use this function for executing code that comes as strings or as compiled code objects.

Running Code From a String Input

The most common way to callexec() is with code that comes from a string-based input. To build this string-based input, you can use:

• Single lines of code orone-liner code snippets
• Multiple lines of code separated bysemicolons
• Multiple lines of code separated bynewline characters
• Multiple lines of code withintriple-quoted strings and with proper indentation

Aone-liner program consists of a single line of code that performs multiple actions at once. Say that you have a sequence of numbers, and you want to build a new sequence containing the sum of squares of all the even numbers in an input sequence.

To solve this problem, you can use the following one-liner code:

>>>numbers=[2,3,7,4,8]>>>sum(number**2fornumberinnumbersifnumber%2==0)84

In the highlighted line, you use agenerator expression to compute the square value of all the even numbers in the input sequence of values. Then you usesum() to compute the total sum of squares.

To run this code withexec(), you just need to transform your one-liner code into a single-line string:

>>>exec("result = sum(number**2 for number in numbers if number % 2 == 0)")>>>result84

In this example, you express the one-liner code as a string. Then you feed this string intoexec() for execution. The only difference between your original code and the string is that the latter stores the computation result in a variable for later access. Remember thatexec() returnsNone rather than a concrete execution result. Why? Because not every piece of code has a final unique result.

Python allows you to write multiplestatements in a single line of code, using semicolons to separate them. Even though this practice is discouraged, nothing will stop you from doing something like this:

>>>name=input("Your name: ");print(f"Hello,{name}!")Your name: LeodanisHello, Leodanis!

You can use semicolons to separate multiple statements and build a single-line string that serves as an argument toexec(). Here’s how:

>>>exec("name = input('Your name: '); print(f'Hello,{name}!')")Your name: LeodanisHello, Leodanis!

The idea of this example is that you can combine multiple Python statements into a single-line string by using semicolons to separate them. In the example, the first statement takes the user’s input, while the second statementprints a greeting message to the screen.

You can also aggregate multiple statements in a single-line string using the newline character,\n:

>>>exec("name = input('Your name: ')\nprint(f'Hello,{name}!')")Your name: LeodanisHello, Leodanis!

The newline character makesexec() understand your single-line string as a multiline set of Python statements. Thenexec() runs the aggregated statements in a row, which works like a multiline code file.

The final approach to building a string-based input for feedingexec() is to use triple-quoted strings. This approach is arguably more flexible and allows you to generate string-based input that looks and works like normal Python code.

It’s important to note that this approach requires you to use proper indentation and code formatting. Consider the following example:

>>>code="""...numbers = [2, 3, 7, 4, 8]......def is_even(number):...    return number % 2 == 0......even_numbers = [number for number in numbers if is_even(number)]......squares = [number**2 for number in even_numbers]......result = sum(squares)......print("Original data:", numbers)...print("Even numbers:", even_numbers)...print("Square values:", squares)...print("Sum of squares:", result)...""">>>exec(code)Original data: [2, 3, 7, 4, 8]Even numbers: [2, 4, 8]Square values: [4, 16, 64]Sum of squares: 84

In this example, you use a triple-quoted string to provide the input toexec(). Note that this string looks like any regular piece of Python code. It uses appropriate indentation, naming style, and formatting. Theexec() function will understand and execute this string as a regular Python code file.

You should note that when you pass a string with code toexec(), the function will parse and compile the target code into Python bytecode. In all cases, the input string should contain valid Python code.

Ifexec() finds anyinvalid syntax during the parsing and compilation steps, then the input code won’t run:

>>>exec("print('Hello, World!)")Traceback (most recent call last):  File"<stdin>", line1, in<module>File"<string>",line1print('Hello, World!)^SyntaxError:unterminated string literal (detected at line 1)

In this example, the target code contains a call toprint() that takes a string as an argument. This string isn’t properly ended with a closing single quotation mark, soexec() raises aSyntaxError pointing out the issue and doesn’t run the input code. Note that Python pinpoints the error at the start of the string rather than at the end, where the closing single quotation mark should go.

Running code that comes as a string, like you did in the example above, is arguably the natural way of usingexec(). However, if you need to run the input code many times, then using a string as an argument will make the function run the parsing and compilation steps every time. This behavior can make your code inefficient in terms of execution speed.

In this situation, the most convenient approach is to compile the target code beforehand and then run the resulting bytecode withexec() as many times as needed. In the following section, you’ll learn how to useexec() with compiled code objects.

Executing Compiled Code

In practice,exec() can be quite slow when you use it to process strings containing code. If you ever need to dynamically run a given piece of code more than once, then compiling it beforehand will be the most performant and recommended approach. Why? Because you’ll be running the parsing and compilation steps only once and then reusing the compiled code.

To compile a piece of Python code, you can usecompile(). This built-in function takes a string as an argument and runs a one-time bytecode compilation on it, generating a code object that you can then pass toexec() for execution.

The signature ofcompile() has the following form:

compile(source,filename,mode,flags=0,dont_inherit=False,optimize=-1)

In this tutorial, you’ll only use the three first arguments tocompile(). Thesource argument holds the code that you need to compile into bytecode. Thefilename argument will hold the file from which the code was read. To read from a string object, you’ll have to setfilename to the"<string>" value.

Finally,compile() can generate code objects that you can execute using eitherexec() oreval(), depending on themode argument’s value. This argument should be set to"exec" or"eval", depending on the target execution function:

>>>string_input="""...def sum_of_even_squares(numbers):...    return sum(number**2 for number in numbers if number % 2 == 0)......print(sum_of_even_squares(numbers))...""">>>compiled_code=compile(string_input,"<string>","exec")>>>exec(compiled_code)>>>numbers=[2,3,7,4,8]>>>exec(compiled_code)84>>>numbers=[5,3,9,6,1]>>>exec(compiled_code)36

Compiling often-repeated code up front withcompile() can help you slightly improve your code’s performance by skipping the parsing and bytecode compilation steps on each call toexec().

Running Code From Python Source Files

You can also useexec() to run code that you’ve read from a reliable.pyfile in your file system or somewhere else. To do this, you can use the built-inopen() function to read the file’s content as a string, which you can then pass as an argument toexec().

For example, say that you have a Python file namedhello.py containing the following code:

# hello.pyprint("Hello, Pythonista!")print("Welcome to Real Python!")defgreet(name="World"):print(f"Hello,{name}!")

This sample script prints a greeting and a welcome message to the screen. It also defines a samplegreet() function for testing purposes. The function takes a name as an argument and prints a customized greeting to the screen.

Now get back to your Python interactive session and run the following code:

>>>withopen("hello.py",mode="r",encoding="utf-8")ashello:...code=hello.read()...>>>exec(code)Hello, Pythonista!Welcome to Real Python!>>>greet()Hello, World!>>>greet("Pythonista")Hello, Pythonista!

In this example, you first open the target.py file as a regular text file using the built-inopen() function in awith statement. Then you call.read() on the file object to read the file’s content into thecodevariable. This call to.read() returns the file’s content as a string. The final step is to callexec() with this string as an argument.

This example runs the code and makes thegreet() function and objects that live inhello.py available in your current namespace. That’s why you can usegreet() directly. The secret behind this behavior has to do with theglobals andlocals arguments, which you’ll learn about in the next section.

Using the technique in the above example, you can open, read, and execute any file containing Python code. This technique may work when you don’t know up front which source files you’ll be running. So, you can’t writeimport module, because you don’t know the module’s name when you’re writing the code.

If you ever choose to use this technique, then make sure that you only execute code from trusted source files. Ideally, the most reliable source files are those thatyou have consciously created to run dynamically. You must never run code files that come from external sources, including your users, without inspecting the code first.

Using theglobals andlocals Arguments

You can pass an execution context toexec() using theglobals andlocals arguments. These arguments can accept dictionary objects that’ll work as the global and localnamespaces thatexec() will use to run the target code.

These arguments are optional. If you omit them, thenexec() will execute the input code in the currentscope, and all the names and objects in this scope will be available toexec(). Likewise, all the names and objects that you define in the input code will be available in the current scope after the call toexec().

Consider the following example:

>>>code="""...z = x + y...""">>># Global names are accessible from exec()>>>x=42>>>y=21>>>zTraceback (most recent call last):...NameError:name 'z' is not defined>>>exec(code)>>># Names in code are available in the current scope>>>z63

This example shows that if you callexec() without providing specific values to theglobals andlocals arguments, then the function runs the input code in the current scope. In this case, the current scope is the global one.

Note that after you callexec(), the names defined in the input code are also available in the current scope. That’s why you can accessz in the final line of code.

If you only provide a value toglobals, then that value must be a dictionary. Theexec() function will use this dictionary for both global and local names. This behavior will restrict access to most names in the current scope:

>>>code="""...z = x + y...""">>>x=42>>>y=21>>>exec(code,{"x":x})Traceback (most recent call last):...NameError:name 'y' is not defined>>>exec(code,{"x":x,"y":y})>>>zTraceback (most recent call last):...NameError:name 'z' is not defined

In the first call toexec(), you use a dictionary as theglobals argument. Because your dictionary doesn’t provide a key holding they name, the call toexec() doesn’t have access to this name and raises aNameError exception.

In the second call toexec(), you provide a different dictionary toglobals. In this case, the dictionary contains both variables,x andy, which allows the function to work correctly. However, this time you don’t have access toz after the call toexec(). Why? Because you’re using a custom dictionary to provide an execution scope toexec() rather than falling back to your current scope.

If you callexec() with aglobals dictionary that doesn’t contain the__builtins__ key explicitly, then Python will automatically insert a reference to thebuilt-in scope or namespace under that key. So, all the built-in objects will be accessible from your target code:

>>>code="""...print(__builtins__)...""">>>exec(code,{}){'__name__': 'builtins', '__doc__': "Built-in functions, ...}

In this example, you’ve provided an empty dictionary to theglobals argument. Note thatexec() still has access to the built-in namespace because this namespace is automatically inserted into the provided dictionary under the__builtins__ key.

If you provide a value for thelocals argument, then it can be anymapping object. This mapping object will hold the local namespace whenexec() is running your target code:

>>>code="""...z = x + y...print(f"{z=}")...""">>>x=42# Global name>>>deffunc():...y=21# Local name...exec(code,{"x":x},{"y":y})...>>>func()z=63>>>zTraceback (most recent call last):...NameError:name 'z' is not defined

The call toexec() is embedded in a function in this example. Therefore, you have a global (module-level) scope and a local (function-level) scope. Theglobals argument provides thex name from the global scope, and thelocals argument provides they name from the local scope.

Note that after runningfunc(), you don’t have access toz because this name was created under the local scope ofexec(), which isn’t available from the outside.

With theglobals andlocals arguments, you can tweak the context in whichexec() runs your code. These arguments are pretty helpful when it comes to minimizing the security risks associated withexec(), but you should still make sure that you’re running code from trusted sources only. In the following section, you’ll learn about these security risks and how to deal with them.

Uncovering and Minimizing the Security Risks Behindexec()

As you’ve learned so far,exec() is a powerful tool that allows you to execute arbitrary code that comes to you as strings. You should useexec() with extreme care and caution because of its ability to runany piece of code.

Typically, the code that feedsexec() is dynamically generated at runtime. This code may have many sources of input, which could include your program user, other programs, a database, a stream of data, and a network connection, among others.

In this scenario, you can’t be entirely sure what the input string will contain. So, the probability of facing an untrusted and malicious source of input code is pretty high.

The security issues associated withexec() are the most common reason why many Python developers recommend avoiding this function altogether. Finding a better, faster, more robust, and safer solution is almost always possible.

However, if you must useexec() in your code, then the generally recommended approach is to use it with explicitglobals andlocals dictionaries.

Another critical issue withexec() is that it breaks a fundamental assumption in programming:the code that you’re currently reading or writing is the code that you’ll be executing when you fire up your program. How doesexec() break this assumption? It makes your programs run new and unknown code that’s dynamically generated. This new code can be hard to follow, maintain, or even control.

In the following sections, you’ll dive into a few recommendations, techniques, and practices that you should apply if you ever need to useexec() in your code.

>>>whileTrue:...exec(input("->> "))...->> print("Hello, World!")Hello, World!

>>>exec("import os",{"__builtins__":{}},{})Traceback (most recent call last):...ImportError:__import__ not found

>>>x=42>>>y=21>>>exec("print(x + y)",{},{})Traceback (most recent call last):...NameError:name 'x' is not defined

>>>exec("print(min([2, 3, 7, 4, 8]))",{},{})2>>>exec("print(len([2, 3, 7, 4, 8]))",{},{})5

>>>exec("print(min([2, 3, 7, 4, 8]))",{"__builtins__":{}},{})Traceback (most recent call last):...NameError:name 'print' is not defined

>>>allowed_builtins={"__builtins__":{"min":min,"print":print}}>>>exec("print(min([2, 3, 7, 4, 8]))",allowed_builtins,{})2>>>exec("print(len([2, 3, 7, 4, 8]))",allowed_builtins,{})Traceback (most recent call last):...NameError:name 'len' is not defined

Puttingexec() Into Action

Up to this point, you’ve learned how the built-inexec() function works. You know that you can useexec() to run either string-based or compiled-code input. You also learned that this function can take two optional arguments,globals andlocals, which allow you to tweak the execution namespace forexec().

Additionally, you’ve learned that usingexec() implies some serious security issues, including allowing users to run arbitrary Python code on your computer. You studied some recommended coding practices that help minimize the security risks associated withexec() in your code.

In the following sections, you’ll code a few practical examples that’ll help you spot those use cases in which usingexec() can be appropriate.

>>>fromtimeitimporttimeit>>>timeit("'-'.join(str(n) for n in range(100))",number=10000)0.1282792080000945

# timeit.py# ...classTimer:"""Class for timing execution speed of small code snippets."""def__init__(self,stmt="pass",setup="pass",timer=default_timer,globals=None):"""Constructor.  See class doc string."""self.timer=timerlocal_ns={}global_ns=_globals()ifglobalsisNoneelseglobals# ...src=template.format(stmt=stmt,setup=setup,init=init)self.src=src# Save for traceback displaycode=compile(src,dummy_src_name,"exec")exec(code,global_ns,local_ns)self.inner=local_ns["inner"]# ...

# calculations.pydefadd(a,b):"""Return the sum of two numbers.    Tests:        >>> add(5, 6)        11        >>> add(2.3, 5.4)        7.7        >>> add("2", 3)        Traceback (most recent call last):        TypeError: numeric type expected for "a" and "b"    """ifnot(isinstance(a,(int,float))andisinstance(b,(int,float))):raiseTypeError('numeric type expected for "a" and "b"')returna+b# ...

$python-mdoctestcalculations.py

# doctest.pyclassDocTestRunner:# ...def__run(self,test,compileflags,out):# ...try:# Don't blink!  This is where the user's code gets run.exec(compile(example.source,filename,"single",compileflags,True),test.globs)self.debugger.set_continue()# ==== Example Finished ====exception=NoneexceptKeyboardInterrupt:# ...

# calculations.pydefadd(a,b):"""Return the sum of two numbers.    Tests:        >>> import os; os.system("ls -l")        0    """ifnot(isinstance(a,(int,float))andisinstance(b,(int,float))):raiseTypeError('numeric type expected for "a" and "b"')returna+b

# settings.conffont_face = ""font_size = 10line_numbers = Truetab_size = 4auto_indent = True

>>>frompathlibimportPath>>>defload_config(config_file):...config_file=Path(config_file)...code=compile(config_file.read_text(),config_file.name,"exec")...config_dict={}...exec(code,{"__builtins__":{}},config_dict)...returnconfig_dict...>>>load_config("settings.conf"){    'font_face': '',    'font_size': 10,    'line_numbers': True,    'tab_size': 4,    'auto_indent': True}

Conclusion

You’ve learned how to use the built-inexec() function to execute Python code from a string or bytecode input. This function provides a quick tool for executing dynamically generated Python code. You also learned how to minimize the security risks associated withexec() and when it’s okay to use the function in your code.

In this tutorial, you’ve learned how to:

• Work with Python’s built-inexec() function
• Use Python’sexec() to runstring-based andcompiled-code input
• Assess and minimize thesecurity risks associated with usingexec()

Additionally, you’ve coded a few practical examples that helped you better understand when and how to useexec() in your Python code.