Differentiable programming is aprogramming paradigm in which a numeric computer program can bedifferentiated throughout viaautomatic differentiation.^{[1][2][3][4][5]} This allows forgradient-based optimization of parameters in the program, often viagradient descent, as well as other learning approaches that are based on higher-order derivative information. Differentiable programming has found use in a wide variety of areas, particularlyscientific computing andmachine learning.^[5] One of the early proposals to adopt such a framework in a systematic fashion to improve upon learning algorithms was made by theAdvanced Concepts Team at theEuropean Space Agency in early 2016.^[6]

Most differentiable programming frameworks work by constructing a graph containing the control flow anddata structures in the program.^[7] Attempts generally fall into two groups:

• Static,compiled graph-based approaches such asTensorFlow,^{[note 1]}Theano, andMXNet. They tend to allow for goodcompiler optimization and easier scaling to large systems, but their static nature limits interactivity and the types of programs that can be created easily (e.g. those involvingloops orrecursion), as well as making it harder for users to reason effectively about their programs.^[7] A proof-of-concept compiler toolchain called Myia uses a subset of Python as a front end and supports higher-order functions, recursion, and higher-order derivatives.^[8][9][10]
• Operator overloading, dynamic graph-based approaches such asPyTorch,NumPy'sautograd package, andPyaudi. Their dynamic and interactive nature lets most programs be written and reasoned about more easily. However, they lead tointerpreter overhead (particularly when composing many small operations), poorer scalability, and reduced benefit from compiler optimization.^[9][10]

The use of just-in-time compilation has emerged recently^[when?] as a possible solution to overcome some of the bottlenecks of interpreted languages. The C++heyoka and Python packageheyoka.py make large use of this technique to offer advanced differentiable programming capabilities (also at high orders). A package for theJulia programming language – Zygote – works directly on Julia'sintermediate representation.^[7][11][5]

A limitation of earlier approaches is that they are only able to differentiate code written in a suitable manner for the framework, limiting their interoperability with other programs. Newer approaches resolve this issue by constructing the graph from the language's syntax or IR, allowing arbitrary code to be differentiated.^[7][9]

Differentiable programming has been applied in areas such as combiningdeep learning withphysics engines inrobotics,^[12] solvingelectronic-structure problems with differentiabledensity functional theory,^[13] differentiableray tracing,^[14]differentiable imaging,^[15][16]image processing,^[17] andprobabilistic programming.^[5]

Differentiable programming is making significant strides in various fields beyond its traditional applications. In healthcare and life sciences, for example, it is being used for deep learning in biophysics-based modelling of molecular mechanisms, in areas such as protein structure prediction and drug discovery. These applications demonstrate the potential of differentiable programming in contributing to significant advancements in understanding complex biological systems and improving healthcare solutions.^[18]

• Differentiable function
• Machine learning

• Differential calculus
• Programming paradigms

• Articles with short description
• Short description is different from Wikidata
• All articles with vague or ambiguous time
• Vague or ambiguous time from April 2025