Repository files navigation

IMPORTANT: Pyctr is prototype software, and under active development. Expect roughedges and bugs, but if you try it, we appreciate early feedback! We'd also lovecontributions (please see our contributing guidelines).

Pyctr is a virtualization and source code transformation engine for Python. Itenables multi-stageprogramming in Python, and is the core functionality of AutoGraph extracted intoa standalone module (i.e., no dependencies on TensorFlow).

Language virtualization (as defined byChafi et. al.)is a methodology in which language constructs are overloadable as virtualmethods. Virtualization enables frameworkdevelopers to add their own overloads for Python constructs which are notnormally overloadable. In this way Pyctr can be seen as an extension of Python'sbuilt in operator overloading support.Python allows one to provide an implementation of__add__to override the meaning of the+ symbol. In an analogous way, Pyctressentially allows developers to define their own__if__function that will be executed in place of Pythonsif statement.

Pyctr aims to virtualize all of Python, allowing DSL and framework developers tooverload Python syntax with custom behavior. Pyctr currently supports manyPython features including, but not limited to,control-flow statements (e.g.if,while, andfor), function calls, andvariable access. We're continually working on support for additional Pythonconstructs.

Pyctr is a Python code transformation library currently distributed as a part ofTensorFlow AutoGraph. AutoGraph is a tool used to convert eager-style Pythoncode, including control-flow, into code that generates TensorFlow graphs. Pyctrinside AutoGraph is a refinement of the code transformation tools generated aspart of the Tangent automatic differentiation project.

This repository contains a version of Pyctr that does not depend on TensorFlowand can be usedto provide AutoGraph-like functionality (reusing Python syntax forframework-specific functionality) for any framework that can be called fromPython.

This document describes how to get started using Pyctr (also see the APIdocumentation). There are two use-cases to consider: users of Pyctr, and usersof systems which use Pyctr. In this document, we refer to users directly usingPyctr as “DSL developers;” they are using Pyctr to repurpose Python as a DSL,and simply providing the domain-specific logic. We refer to the other categoryof users as “end users.”

If you want to skip ahead to looking at some examples, Pyctr provides some intheexamples module.

In using Pyctr, one must first determine which features of Python requirevirtualization. For example, a DSL developer may only require function calls tobe virtualized, in other cases (such as when translating an entire program) itmay be necessary to virtualize everything. Currently, these developers must findwhich transformer contains the necessary constructs and supply that as anargument for Pyctr’s conversion method; soon this will be inferred automaticallyby inspecting a property of the supplied overloads object. Pyctr currently makesthe following transformers available:

• variables: for variable virtualization
• control_flow: for control flow structures such as for/while loops and ifstatements (Note: control_flow requires that the variables transformer hastaken place. Failure to perform these in the correct order may yieldunintended consequences.)
• functions: for function call virtualization
• logical_ops: for and, or, not virtualization

We provide a detailed explanation of the API exposed by each transformer below.

Virtualized methods exposed:

• init(name): this occurs before the first assignment of a variable (typicallyimmediately before, but there are some exceptions). This will appear in thepattern x = overload.init(‘x’)
• assign: this represents variable assignment, and replaces the patternlhs = rhs with overload.assign(lhs, rhs)
• read: this represents a variable read, and replaces the pattern x withoverload.read(x)

Virtualizing variables requires a separate pass of the AST to construct theproper scopes. Pyctr currently handles all global and nonlocal scopes asexpected, but does not handle classes (yet!). The code for this may be found intransformers/virtualization/scoping.py.

For more details, see transformers/virtualizations/variables.py.

Virtualized methods exposed:

• if_stmt(cond, body, orelse, local_writes): this replaces idiomatic ifstatements with functions representing the condition, then branch, and elsebranch, and a call to overload.if_stmt
• while_stmt(cond, body, orelse, local_writes): this replaces idiomatic whileloops with functions representing the condition, loop body, and orelse body,and a call to overload.while_stmt
• for_stmt(target, iter_, body, orelse, local_writes): this replaces idiomaticfor loops with functions representing the loop body and orelse body, and acall to overload.for_stmt

For more details, see transformers/virtualizations/control_flow.py.

Virtualized methods exposed:

• call(func, args, kwargs): this replaces all function calls (other than thosewhitelisted) with a call to overload.call

For more details, see transformers/virtualizations/functions.py.

Virtualized methods exposed:

• and_(a, b): replaces pattern a and b with overload.and_(a, b)
• or_(a, b): replaces pattern a or b with overload.or_(a, b)
• not_(x): replaces pattern not x with overload.not_(x)

Note that and_ and or_ expect a variable number of parameters for b in order tomatch Python’s semantics (i.e., allowing chaining).

For more details, see transformers/virtualizations/logical_ops.py.

Examples of building overloads objects may be found in theexamples module.

A simplified view is that a DSL developer creates an object upon which thenecessary attributes are defined. For example, one may create a module whichconverts logical operations to Z3Py operations (seeexamples/z3py/z3py.py):

def and_(a, b):  return z3.And(a, b)

def or_(a, b):  return z3.Or(a, b)

def not_(x):  return z3.Not(x)

The following code defines one of De Morgan’s laws using idiomatic Pythonlogical operators and converts it to Z3Py:

def demorgan(a, b):  return (a and b) == (not (not a or not b))

converted_demorgan = pyctr.convert(demorgan, z3py, transformers=[logical_ops])

Note thatz3py here represents the module we defined above.

In order to provide an easy entrypoint for users, DSL developers may choose toconstruct a decorator which may be applied to a function for conversion. Thismay look as follows:

def pytorch_to_tf(func):  return pyctr.convert(func, my_overload_object, transformers=my_transformers)

If a DSL developer has chosen to expose a decorator as the entrypoint for Pyctrconversion (as shown above), end users may simply do the following:

@pytorch_to_tfdef my_function(x, y):  return torch.rand(x) * torch.rand(y)

Alternatively, end users may use thePyctr.convert method directly.