Rate this Page
★★★★★

Togglingerror_on_graph_break#

Created On: Sep 03, 2025 | Last Updated On: Sep 03, 2025

Summary:

  • Whenfullgraph=False, we can usetorch._dynamo.error_on_graph_break() for more flexibility indealing with graph breaks.

So far, we have introduced two ways in dealing with graph breaks intorch.compile:

  1. fullgraph=True errors on the first graph break and additionally guarantees that only one graph is traced from the code.

  2. fullgraph=False continues tracing even when encountering graph breaks.

What if we want to disallow graph breaks for most of the code, but there are a few problematic functions where the graph breaks are hard to remove,and we are okay with having those graph breaks? We can usetorch._dynamo.error_on_graph_break() to achieve this.

torch.compile has anerror_on_graph_break setting (initially set toFalse).If a graph break or compiler error occurs in code whileerror_on_graph_break is set toFalse, thentorch.compile will attempt to continue compilation after the graph break/error.Iferror_on_graph_break is set toTrue, thentorch.compile will abort compilation and propagate the error to user code.

A significant difference betweenerror_on_graph_break=True andfullgraph=True is that the formerdoes not guarantee that a single graph will be captured.error_on_graph_breakcan be arbitrarily toggled during compile time by using thetorch._dynamo.error_on_graph_break() context manager/decorator.In comparison, oncefullgraph is set toTrue, it cannot be set back toFalse.Finally,error_on_graph_break has lower precedence thanfullgraph -error_on_graph_break only takes effect whenfullgraph=False.

error_on_graph_break(False) example#

@torch._dynamo.error_on_graph_break(False)defcode_with_a_difficult_graph_break(x):x=x+1torch._dynamo.graph_break()returnx+2definner(x):returncode_with_a_difficult_graph_break(x)# NOTE: fullgraph=False@torch._dynamo.error_on_graph_break(True)@torch.compiledeffn(x):returninner(x)# No error, but there is a graph breakfn(torch.randn(3))
Graph break in user code at /tmp/ipykernel_369/1452578661.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/1452578661.py", line 17, in <module>    fn(torch.randn(3))  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/1452578661.py", line 14, in fn    return inner(x)  File "/tmp/ipykernel_369/1452578661.py", line 8, in inner    return code_with_a_difficult_graph_break(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/1452578661.py", line 4, in code_with_a_difficult_graph_break    torch._dynamo.graph_break()Graph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1452578661.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlGraph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1452578661.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlGraph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1452578661.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.html
tensor([3.0113, 3.9169, 0.5099])

Usingerror_on_graph_break(False) undererror_on_graph_break(True) is helpful for when we want to minimize graph breaks (i.e. follow thefullgraph=True programming model),but there are some sections of code with non-performance-critical graph breaks that are difficult to work around.

error_on_graph_break() can be used as a context manager as well:

# NOTE: fullgraph=False@torch._dynamo.error_on_graph_break(True)@torch.compiledeffn(x):x=x+1withtorch._dynamo.error_on_graph_break(False):torch._dynamo.graph_break()# no errorreturnx+2# No error, but there is a graph breakfn(torch.randn(3))
Graph break in user code at /tmp/ipykernel_369/737485247.py:7Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/737485247.py", line 11, in <module>    fn(torch.randn(3))  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/737485247.py", line 7, in fn    torch._dynamo.graph_break()  # no error
tensor([2.2028, 3.7649, 4.9742])

You can use monkey patching to toggleerror_on_graph_break for code where you cannot edit the source (e.g. framework code):

classThirdPartyModule(torch.nn.Module):defforward(self,x):x=x+1torch._dynamo.graph_break()returnx+2tp_mod=ThirdPartyModule()tp_mod.forward=torch._dynamo.error_on_graph_break(False)(tp_mod.forward)@torch._dynamo.error_on_graph_break(True)@torch.compiledeffn(x):returntp_mod.forward(x)# No error, but there is a graph breakfn(torch.randn(3))
Graph break in user code at /tmp/ipykernel_369/2112598647.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/2112598647.py", line 16, in <module>    fn(torch.randn(3))  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/2112598647.py", line 13, in fn    return tp_mod.forward(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/2112598647.py", line 4, in forward    torch._dynamo.graph_break()Graph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/2112598647.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlGraph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/2112598647.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.html
tensor([3.2821, 4.4465, 2.9630])

error_on_graph_break(True) example#

@torch._dynamo.error_on_graph_break(True)definner2(x):x=x+1torch._dynamo.graph_break()# errorreturnx+2definner(x):returninner2(x)# fullgraph=False, error_on_graph_break=False@torch.compiledeffn(x):x=x+4torch._dynamo.graph_break()# no errorreturninner(x)try:fn(torch.randn(3))exceptExceptionase:print(e)
Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlfrom user code:   File "/tmp/ipykernel_369/2379101916.py", line 15, in torch_dynamo_resume_in_fn_at_14    return inner(x)  File "/tmp/ipykernel_369/2379101916.py", line 8, in inner    return inner2(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/2379101916.py", line 4, in inner2    torch._dynamo.graph_break()  # errorSet TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"
Graph break in user code at /tmp/ipykernel_369/2379101916.py:14Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/2379101916.py", line 18, in <module>    fn(torch.randn(3))  File "/tmp/ipykernel_369/2379101916.py", line 14, in fn    torch._dynamo.graph_break()  # no error

Usingerror_on_graph_break(True) undererror_on_graph_break(False) is helpful for when we want to usetorch.compile flexibly (i.e. follow thefullgraph=False programming model),but there are some sections of the code that are performance-critical and we want to ensure that those sections do not contain graph breaks.

error_on_graph_break nesting behavior#

torch._dynamo.error_on_graph_break() affects theerror_on_graph_break setting of nested calls as well:

definner(x):x=x+1torch._dynamo.graph_break()returnx+2definner2(x):withtorch._dynamo.error_on_graph_break(False):returninner(x)@torch._dynamo.error_on_graph_break(True)@torch.compiledeffn(x):returninner2(x)# no errorfn(torch.randn(3))
Graph break in user code at /tmp/ipykernel_369/1007149706.py:3Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/1007149706.py", line 16, in <module>    fn(torch.randn(3))  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/1007149706.py", line 13, in fn    return inner2(x)  File "/tmp/ipykernel_369/1007149706.py", line 8, in inner2    return inner(x)  File "/tmp/ipykernel_369/1007149706.py", line 3, in inner    torch._dynamo.graph_break()Graph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1007149706.py:3Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlGraph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1007149706.py:3Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.html
tensor([2.7700, 2.0265, 1.2761])

torch._dynamo.error_on_graph_break() can be used under anothertorch._dynamo.error_on_graph_break() region:

definner(x):x=x+1withtorch._dynamo.error_on_graph_break(False):torch._dynamo.graph_break()returnx+2definner2(x):withtorch._dynamo.error_on_graph_break(True):returninner(x)@torch.compiledeffn(x):returninner2(x)# no errorfn(torch.randn(3))
Graph break in user code at /tmp/ipykernel_369/1343774799.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/1343774799.py", line 16, in <module>    fn(torch.randn(3))  File "/tmp/ipykernel_369/1343774799.py", line 13, in fn    return inner2(x)  File "/tmp/ipykernel_369/1343774799.py", line 9, in inner2    return inner(x)  File "/tmp/ipykernel_369/1343774799.py", line 4, in inner    torch._dynamo.graph_break()Graph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1343774799.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlGraph break (user stack suppressed due to duplicate graph break) in user code at /tmp/ipykernel_369/1343774799.py:4Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.html
tensor([2.9136, 2.4328, 3.4279])

Interaction withfullgraph#

fullgraph=True takes higher precedence thanerror_on_graph_break:

@torch._dynamo.error_on_graph_break(False)definner(x):x=x+1torch._dynamo.graph_break()returnx+2@torch.compile(fullgraph=True)deffn(x):returninner(x)try:fn(torch.randn(3))exceptExceptionase:print(e)
Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlfrom user code:   File "/tmp/ipykernel_369/2331424258.py", line 9, in fn    return inner(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 216, in inner    return func(*args, **kwargs)  File "/tmp/ipykernel_369/2331424258.py", line 3, in inner    x = x + 1Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"

fullgraph=True cannot be toggled back tofullgraph=False:

@torch.compile(fullgraph=False)definner(x):x=x+1torch._dynamo.graph_break()returnx+2@torch.compile(fullgraph=True)deffn(x):returninner(x)try:fn(torch.randn(3))exceptExceptionase:print(e)
Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlfrom user code:   File "/tmp/ipykernel_369/262151723.py", line 9, in fn    return inner(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/polyfills/__init__.py", line 276, in getattr_and_trace    return fn(*args[2:], **kwargs)  File "/tmp/ipykernel_369/262151723.py", line 3, in inner    x = x + 1Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"
@torch.compile(fullgraph=True)definner(x):x=x+1torch._dynamo.graph_break()returnx+2@torch.compile(fullgraph=False)deffn(x):returninner(x)try:fn(torch.randn(3))exceptExceptionase:print(e)
Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlfrom user code:   File "/tmp/ipykernel_369/2119173801.py", line 3, in inner    x = x + 1Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"
Graph break in user code at /tmp/ipykernel_369/2119173801.py:3Graph Break Reason: Call to `torch._dynamo.graph_break()`  Explanation: User-inserted graph break. Message: None  Hint: Remove the `torch._dynamo.graph_break()` call.  Developer debug context: Called `torch._dynamo.graph_break()` with args `[]`, kwargs `{}` For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0025.htmlUser code traceback:  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main    return _run_code(code, main_globals, None,  File "/opt/conda/envs/py_3.10/lib/python3.10/runpy.py", line 86, in _run_code    exec(code, run_globals)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel_launcher.py", line 18, in <module>    app.launch_new_instance()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1075, in launch_instance    app.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelapp.py", line 758, in start    self.io_loop.start()  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 211, in start    self.asyncio_loop.run_forever()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever    self._run_once()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once    handle._run()  File "/opt/conda/envs/py_3.10/lib/python3.10/asyncio/events.py", line 80, in _run    self._context.run(self._callback, *self._args)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/utils.py", line 71, in preserve_context    return await f(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 614, in shell_main    await self.dispatch_shell(msg, subshell_id=subshell_id)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 471, in dispatch_shell    await result  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 366, in execute_request    await super().execute_request(stream, ident, parent)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/kernelbase.py", line 827, in execute_request    reply_content = await reply_content  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/ipkernel.py", line 458, in do_execute    res = shell.run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/ipykernel/zmqshell.py", line 663, in run_cell    return super().run_cell(*args, **kwargs)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell    result = self._run_cell(  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell    result = runner(coro)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner    coro.send(None)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes    if await self.run_code(code, result, async_=asy):  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "/tmp/ipykernel_369/2119173801.py", line 12, in <module>    fn(torch.randn(3))  File "/tmp/ipykernel_369/2119173801.py", line 9, in fn    return inner(x)  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/polyfills/__init__.py", line 276, in getattr_and_trace    return fn(*args[2:], **kwargs)  File "/tmp/ipykernel_369/2119173801.py", line 3, in inner    x = x + 1

Summary offullgraph=True/False vserror_on_graph_break#

Here is a table summarizing the differences betweenfullgraph=True/False anderror_on_graph_break:

error_on_graph_break=True

error_on_graph_break=False (default)

fullgraph=True

Graph breaks result in errors. Only the first graph break will be reported.One graph guarantee.

fullgraph cannot be toggled toFalse.error_on_graph_break has no effect.

User code must be fully compatible withtorch.compile. Guarantees no performance hits from graph breaks (because there are no graph breaks).

Ideal for code sensitive to graph breaks: framework/library code or cases where getting maximum performance is required. Prevents downstream user code from inadvertently allowing graph breaks.

Same asfullgraph=True anderror_on_graph_break=True aserror_on_graph_break has no effect whenfullgraph=True.

fullgraph=False (default)

Graph breaks result in errors. Only the first graph break will be reported.No one graph guarantee.

error_on_graph_break can be toggled toFalse.

User code must be fully compatible withtorch.compile. Guarantees no performance hits from graph breaks (because there are no graph breaks).

Ideal for user code sensitive to graph breaks.error_on_graph_break can be toggled toFalse to deal with sections that have graph breaks that are difficult to work around.

Will continue to compile after encountering graph breaks. All graph breaks will be reported.

error_on_graph_break can be toggled toTrue.

Doesn’t require many user code changes to work. Performance may be negatively impacted due to graph breaks.

Ideal for out-of-the-box use cases, on “non-weird” code, or where squeezing maximal performance is not necessary