- Notifications
You must be signed in to change notification settings - Fork527
A high-performance, zero-overhead, extensible Python compiler with built-in NumPy support
License
exaloop/codon
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Codon is a high-performance Python implementation that compiles to native machine code withoutany runtime overhead. Typical speedups over vanilla Python are on the order of 10-100x or more, ona single thread. Codon's performance is typically on par with (and sometimes better than) that ofC/C++. Unlike Python, Codon supports native multithreading, which can lead to speedups many timeshigher still.
Think of Codon as Python reimagined for static, ahead-of-time compilation, built from the groundup with best possible performance in mind.
- 💡No learning curve: Be as close to CPython as possible in terms of syntax, semantics and libraries
- 🚀Top-notch performance: Atleast on par with low-level languages like C, C++ or Rust
- 💻Hardware support: Full, seamless support for multicore programming, multithreading (no GIL!), GPU and more
- 📈Optimizations: Comprehensive optimization framework that can target high-level Python constructsand libraries
- 🔋Interoperability: Full interoperability with Python's ecosystem of packages and libraries
❌Drop-in replacement for CPython: Codon is not a drop-in replacement for CPython. There are someaspects of Python that are not suitable for static compilation — we don't support these in Codon.There are ways to use Codon in larger Python codebases via itsJIT decoratororPython extension backend. Codon also supportscalling any Python module via itsPython interoperability.See also"Differences with Python" in the docs.
❌New syntax and language constructs: We try to avoid adding new syntax, keywords or other languagefeatures as much as possible. While Codon does add some new syntax in a couple places (e.g. to expressparallelism), we try to make it as familiar and intuitive as possible.
Download and install Codon with this command:
/bin/bash -c"$(curl -fsSL https://exaloop.io/install.sh)"
After following the prompts, thecodon
command will be available to use. For example:
- To run a program:
codon run file.py
- To run a program with optimizations enabled:
codon run -release file.py
- To compile to an executable:
codon build -release file.py
- To generate LLVM IR:
codon build -release -llvm file.py
Many more options are available and described inthe docs.
Alternatively, you canbuild from source.
Codon supports much of Python, and many Python programs will work with few if any modifications.Here's a simple scriptfib.py
that computes the 40th Fibonacci number...
fromtimeimporttimedeffib(n):returnnifn<2elsefib(n-1)+fib(n-2)t0=time()ans=fib(40)t1=time()print(f'Computed fib(40) ={ans} in{t1-t0} seconds.')
... run through Python and Codon:
$ python3 fib.pyComputed fib(40) = 102334155 in 17.979357957839966 seconds.$ codon run -release fib.pyComputed fib(40) = 102334155 in 0.275645 seconds.
You can import and use any Python package from Codon viafrom python import
. For example:
frompythonimportmatplotlib.pyplotaspltdata= [x**2forxinrange(10)]plt.plot(data)plt.show()
(Just remember to set theCODON_PYTHON
environment variable to the CPython shared library,as explained in thethe Python interoperability docs.)
Codon supports native multithreading viaOpenMP. The@par
annotationin the code below tells the compiler to parallelize the followingfor
-loop, in this case usinga dynamic schedule, chunk size of 100, and 16 threads.
fromsysimportargvdefis_prime(n):factors=0foriinrange(2,n):ifn%i==0:factors+=1returnfactors==0limit=int(argv[1])total=0@par(schedule='dynamic',chunk_size=100,num_threads=16)foriinrange(2,limit):ifis_prime(i):total+=1print(total)
Note that Codon automatically turns thetotal += 1
statement in the loop body into an atomicreduction to avoid race conditions. Learn more in themultitheading docs.
Codon also supports writing and executing GPU kernels. Here's an example that computes theMandelbrot set:
importgpuMAX=1000# maximum Mandelbrot iterationsN=4096# width and height of imagepixels= [0for_inrange(N*N)]defscale(x,a,b):returna+ (x/N)*(b-a)@gpu.kerneldefmandelbrot(pixels):idx= (gpu.block.x*gpu.block.dim.x)+gpu.thread.xi,j=divmod(idx,N)c=complex(scale(j,-2.00,0.47),scale(i,-1.12,1.12))z=0jiteration=0whileabs(z)<=2anditeration<MAX:z=z**2+citeration+=1pixels[idx]=int(255*iteration/MAX)mandelbrot(pixels,grid=(N*N)//1024,block=1024)
GPU programming can also be done using the@par
syntax with@par(gpu=True)
. See theGPU programming docs for more details.
Codon includes a feature-complete, fully-compiled native NumPy implementation. It uses the sameAPI as NumPy, but re-implements everything in Codon itself, allowing for a range of optimizationsand performance improvements.
Here's an example NumPy program that approximates
importtimeimportnumpyasnprng=np.random.default_rng(seed=0)x=rng.random(500_000_000)y=rng.random(500_000_000)t0=time.time()# pi ~= 4 x (fraction of points in circle)pi= ((x-1)**2+ (y-1)**2<1).sum()* (4/len(x))t1=time.time()print(f'Computed pi~={pi:.4f} in{t1-t0:.2f} sec')
... run through Python and Codon:
$ python3 pi.pyComputed pi~=3.1417 in 2.25 sec$ codon run -release pi.pyComputed pi~=3.1417 in 0.43 sec
Codon can speed up NumPy code through general-purpose and NumPy-specific compiler optimizations,including inlining, fusion, memory allocation elision and more. Furthermore, Codon's NumPyimplementation works with its multithreading and GPU capabilities, and can even integrate withPyTorch. Learn more in theCodon-NumPy docs.
Please seedocs.exaloop.io for in-depth documentation.
About
A high-performance, zero-overhead, extensible Python compiler with built-in NumPy support