Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork10.9k
Numerical software on Windows
This is a survey of the build dependencies of numerical software on Windows.
Fast numerics need optimized libraries forBLAS andLAPACK. They also need a Fortrancompiler to make use of standard Fortran numerical libraries.
Most numerical programming development happens in Unix, and most open-sourcesoftware is cross-platform. Building many numerical libraries needs aUnix-like build system such asmake
andautoconf
.
A selected list:
- numpy and scipy numerical libraries for Python
- R language for statistical computing
- GNU Octave numerical computinglanguage
- Sage mathematics software system
- Scilab software for numerical computation
- Julia dynamic programming language for technicalcomputing
All of these need to compile Fortran code and use BLAS / LAPACK libraries -see below.
SeeVisual Studio page on Wikipedia.This is the platform compiler for Windows. Express edition is free todownload. Compiles C and C++, but Microsoft does not make a Fortran compiler.
SeeMSVC andPython for a listof MS Visual C versions and download links.
Free x86 and AMD64 (x86-64) VC90 c-compilers for Python-2.7 are now available from Microsoft.
Cygwin is a Unix-like environment on Windows. Itincludes the full GNU compiler collection and the Unix build tools. gcccompilers link against a C runtime in a Cygwin-specific DLL providing a POSIXcompatibility layer. This can cause problems when different libraries compileagainst different versions of the DLL. It is easy to run into problems withmultiple copies of theDLL
MinGW is a port of of the GNU compiler collection to Windows that linksagainst the Microsoft C runtime libraries. It is 32-bit only.
SeeMingGW-w64 sourceforge page andsection in MinGW wikipedia article
Another port of of the GNU compiler collection to Windows implementingmultilib (32- and 64- bit compilation from the same compiler).
Much younger than the MinGW project, and under rapid development.
SeeIntel C compilers. Seem tobe somewhere upward of $1000 for Windows C compiler.
See theIntel developer licensepage.
The license that appears to apply to the Intel run-times, linked statically ordynamically, is theIntel EULA for software developmentproducts. Seethis discussion for more detail.
Relevant clauses for us are:
- Distribution of the Redistributables is also subject to the followinglimitations: [...] (d) will provide the Redistributables subject to alicense agreement that prohibits disassembly and reverse engineering ofthe Redistributables except in cases when you provide Your Productsubject to an open source license that is not an Excluded License, forexample, the BSD license, or the MIT license, (e) will indemnify, holdharmless, and defend Intel and its suppliers from and against any claimsor lawsuits, including attorney's fees, that arise or result from Yourmodifications, derivative works or Your distribution of Your Product.
"Redistributables" does cover run-time DLLs, and probably also applies tostatically linked object code - seediscussion.
With this license, distributing our binaries under a BSD license, we don'thave to add the disassembly clause to the license agreement, but we do have topay Intel's legal fees if someone sues Intel because of a problem in Numpy /Scipy.
Also see below for discussion of Intel Fortran compiler.
See:mingwpy discussion of BLAS / LAPACK.
The options for a 64-bit Fortran compiler on Windows are:
- Cygwin gfortran
- MinGW-w64 gfortran
- Intel Fortran. "From$849". Fromthis forumdiscussion,even libraries that are statically linked to the Intel Fortran runtime arecovered by the same end-user license, and specifically, the developerdistributing the binaries is liable for Intel's legal fees if Intel getssued as a result of using the binary (see above).
- Portland Group Visual Fortran.Free license offer for US andGerman academics until June 30th 2014. Thelicense for run-timefiles forces us to add a clause toour license forbidding disassembly of their run-times, and makes us liablefor their legal fees if they are sued, making it very unattractive for opensource development. Standard cost appears to be $140 for a single seatAcademic license.
- Absoft (from $289 academic, $697commercial).
- gfortran fromFortran tools forWindows
- NAG Fortran builder
- Lahey Fortran - from $175 foran educational license for Lahey / GNU Fortran.
- Simply Fortran - GNU Fortran with anIDE and package manager. It is free to download, but will disable some ofthe features of the IDEafter 30 daysunless you register. Registration costs $99. I (MB) emailed thesupport email on April 7 2016 to ask if SimplyFortran was able to link withthe MSVC 2015 runtime (the MSVC version used to build Python 3.5), but havenot yet had a reply.
- (At some point)PGI LLVM Fortrancompiler.Announced in November 2015, this "multi-year" project is expected to appearas a source code release by"late2016".
If all Fortran code is compatible with Fortran 77, it is also possible tocompile the Fortran code with a C compiler by converting the Fortran code to Cwithf2c.
Various problems can arise - seeOpenBLAS withVS andMSVC and MinGW DLLs for examples.
Applications and DLLs have to make sure that they do not use different MSVCruntime libraries in the same process.
MinGW-w64 lacks intrinsic support for choosing MSVC run-times as a gcc option.For example, Python 3.3 was compiled with MSVC 10 (VS 2010). A Pythonextension will be in the same process as Python, and so will also need to linkagainst the MSVC 10 run-times.
The workaround is to create and patch the gcc specs file for the particularMSVC runtime you want to link against.
- Cygwin contains a full range of Unix build tools.
- msys is "a collection of GNU utilitiessuch as bash, make, gawk and grep to allow building of applications andprograms which depend on traditionally UNIX tools to be present. It isintended to supplement MinGW and the deficiencies of the cmd shell." MSYScan also be used with MinGW-w64. It contains make and autoconf and a minimalbash shell.
- msys2 is "an updated, modernversion of MSYS, both of which are Cygwin (POSIX compatibility layer) forkswith the aim of better interoperability with native Windows software." Itcan install MinGW-w64 from its own package manager.
Another option is using cross-compilers likeMXE. MXE doesMinGW builds from Linux.
If a program tries to load a DLL calledmylibrary.dll
, Windows has todecide which directories it should search formylibrary.dll
.
Applications and libraries often need to ship DLLs, and they need to make surethe application / libraries finds the correct version.
One place that Windows looks for DLLs is on the system $PATH (the%PATH%
variable incmd.exe
, or$env:PATH
in Powershell). It is not safe ingeneral to put our ownmylibrary.dll
into some system directory likeWINDOWS/system
because another application may depend on a differentversion of the DLL that is also calledmylibrary.dll
.
For reference here is thestandard DLL searchalgorithm.See thisexploration of DLL searchsettings for moreexplanation.
Alternative strategies for finding the right DLL are:
Putting the required DLL in the same directory as the calling executable,assuming the standard search order is in place.
Using the
LOAD_WITH_ALTERED_SEARCH_PATH
flag to the DLL loading callLoadLibraryEx
. In this case you get thealternate DLL searchalgorithm,which looks in the directory containing the loading DLL instead of thedirectory containing the calling executable.Setting an extra DLL search directory withSetDllDirectory.The Microsoft documentation on theLoadLibraryExcall says this about
SetDllDirectory
:However, be aware that using SetDllDirectory effectively disables safe DLLsearch mode while the specified directory is in the search path and it isnot thread safe. If possible, it is best to use AddDllDirectory to modify adefault process search path.
See thestandard DLL searchalgorithmfor more detail on safe DLL search mode.
Adding a specific directory to the search path withAddDllDirectory (dead link).This only works for updated Windows 7 and later.
Usingprivateassembliesto specify a set of DLLs specific to the application.
SeeBuilding numpy / scipy (dead link).
Standard Windows compile uses:
- MinGW or Cygwin for build tools
- ATLAS for BLAS / LAPACK
- MinGW or Cygwin for Fortran
Standard numpy and scipy binary releases on Windows use pre-compiled ATLASlibraries and are 32-bit only because of the difficulty of compiling ATLAS on64-bit Windows. Because of the problem of locating DLLs, ATLAS is linkedstatically.
Seebuilding R fromsourceandR Windowstoolset.
In the Windows toolset page, a typical quote:
We have found that the build process for R is quite sensitive to the choiceof tools: please follow our instructions exactly, even to the choice ofparticular versions of the tools
The R build depends on a customized set ofWindows tools forR. These includecopies ofCygwin command linetoolsand acopy of the MinGW-w64toolchain
So, R uses:
- MinGW / Cygwin copies for build tools
- Copies of reference BLAS
- MinGW for Fortran
SeeOctave for WindowsandOctave-forge Windowsdirectory.
OctaveWindows binary releases on the main sourceforgesiteare behind compared with Mac (3.6.4 vs 3.8.0). The top of the Octave Windowspage says:
GNU Octave is primarily developed on GNU/Linux and other POSIX conformalsystems. The ports of GNU Octave to Windows use different approaches to getmost of the original Octave and adapt it to Microsoft Windows idiosyncrasies(eg: dynamic libraries, file paths, permissions, environment variables, GUIsystem, etc). Bear this in mind and don't panic if you get unexpectedresults.
There is acurrent Octave build forwindows (dead link)using theMXE cross compiler.
Tools, BLAS and Fortran use are a little confusing across the range of Octavebuilds:
- There appear to be Octave builds with MinGW, Cygwin and MSVC. The currentMinGW builds look like they are 32-bit. Approach seems to be to specify acustom MSYS / MinGW environment viainstall.cmdscripts to get dependencies. MSVC builds apparently need MSYS - seethisemailthread.
- BLAS is both of (choice of) OpenBLAS and ATLAS
- MinGW builds use gfortran. It's not clear to me (MB) how the MSVC buildsdeal with Fortran. They may use MinGW gfortran - see links inthisthread
Sage can only be built on Cygwin on Windows. Uses:
- Cygwin for build chain
- Blas from ATLAS
- Fortran from gfortran
Seecompiling scilab forWindows
The instructions include this command to checkout requirements for Windows:
svn export --force --username anonymous --password Scilab svn://svn.scilab.org/scilab/trunk/Dev-Tools/SE/Prerequirements/Windows scilab
This fetches a large number of Windows binary packages that appear to includecompiled reference BLAS, ATLAS, OpenBLAS, MKL, Python, java.
So:
- MSVC for build chain
- One of reference BLAS, ATLAS, OpenBLAS, MKL
- Intel Fortran XE2011orf2cfor Fortran sources.
Julia usesMinGW-builds -binary builds of dual 32- / 64-bit MinGW compilers.
- MinGW-builds / MSYS2 or MinGW-builds / MSYS for Unix build tools
- OpenBLAS compiled as part of the build process
- Fortran bygfortran