Autoload Mechanism#
Created On: Sep 12, 2025 | Last Updated On: Sep 13, 2025
TheAutoload mechanism in PyTorch simplifies the integration of a custom backend by enabling automatic discovery and initialization at runtime. This eliminates the need for explicit imports or manual initialization, allowing developers to seamlessly integrate a new accelerator or backend into PyTorch.
Background#
TheAutoload Device Extension proposal in PyTorch is centered on improving support for various hardware backend devices, especially those implemented as out-of-the-tree extensions (not part of PyTorch’s main codebase). Currently, users must manually import or load these device-specific extensions to use them, which complicates the experience and increases cognitive overhead.
In contrast, in-tree devices (devices officially supported within PyTorch) are seamlessly integrated—users don’t need extra imports or steps. The goal of autoloading is to make out-of-the-tree devices just as easy to use, so users can follow the standard PyTorch device programming model without explicit loading or code changes. This would allow existing PyTorch applications to run on new devices without any modification, making hardware support more user-friendly and reducing barriers to adoption.
For more information about the background ofAutoload, please refer to itsRFC.
Design#
The core idea ofAutoload is to Use Python’s plugin discovery (entry points) so PyTorch automatically loads out-of-tree device extensions when torch is imported—no explicit user import needed.
For more instructions of the design ofAutoload, please refer toHow it works.
Implementation#
This tutorial will takeOpenReg as a new out-of-the-tree device and guide you through the steps to enable and use theAutoload mechanism.
Entry Point Setup#
To enableAutoload, register the_autoload function as an entry point insetup.py file.
1setup( 2packages=find_packages(), 3package_data=package_data, 4ext_modules=ext_modules, 5cmdclass={ 6"clean":BuildClean,# type: ignore[misc] 7}, 8include_package_data=False, 9entry_points={10"torch.backends":[11"torch_openreg = torch_openreg:_autoload",12],13},14)
Backend Setup#
Define the initialization hook_autoload for backend initialization intorch_openreg. This hook will be automatically invoked by PyTorch during startup.
1def_autoload():2# It is a placeholder function here to be registered as an entry point.3pass45
Result#
After setting up the entry point and backend, build and install your backend. Now, we can use the new accelerator without explicitly importing it.
>>>importtorch>>>importtorch_openreg>>>torch.tensor(1,device="openreg")tensor(1, device='openreg:0')
>>>importtorch# Automatically import torch_openreg>>>>>>torch.tensor(1,device="openreg")tensor(1, device='openreg:0')
