To build lxml from source, you need libxml2 and libxslt properlyinstalled,including the header files. These are likely shipped inseparate-dev or-devel packages likelibxml2-dev, whichyou must install before trying to build lxml.
Contents
The lxml.etree and lxml.objectify modules are written inCython.Since we distribute the Cython-generated .c files with lxml releases,however, you do not need Cython to build lxml from the normal releasesources. We even encourage you tonot install Cython for a normalrelease build, as the generated C code can vary quite heavily betweenCython versions, which may or may not generate correct code for lxml.The pre-generated release sources were tested and therefore are knownto work.
So, if you want a reliable build of lxml, we suggest to a) use asource release of lxml and b) disable or uninstall Cython for thebuild.
Only if you are interested in building lxml from a checkout of thedeveloper sources (e.g. to test a bug fix that has not been releaseyet) or if you want to be an lxml developer, then you do need aworking Cython installation. You can usepip to install it:
pip install -r requirements.txt
https://github.com/lxml/lxml/blob/master/requirements.txt
lxml currently requires at least Cython 0.20, later release versionsshould work as well.
The lxml package is developed in a repository onGithub usingMercurial and thehg-git plugin. You can retrieve the currentdeveloper version using:
hg clone git://github.com/lxml/lxml.git lxml
This will create a directorylxml and download the source into it,including the complete development history. Don't be afraid, thedownload is fairly quick. You can also browse thelxml repositorythrough the web.
Clone the source repository as described above (or downloadthesource tar-ball and unpack it) and then type:
python setup.py build
or:
python setup.py bdist_egg # requires 'setuptools' or 'distribute'
To (re-)build the C sources with Cython, you must additionally pass theoption--with-cython:
python setup.py build --with-cython
If you want to test lxml from the source directory, it is better to build itin-place like this:
python setup.py build_ext -i --with-cython
or, in Unix-like environments:
make inplace
To speed up the build in test environments (e.g. on a continuousintegration server), set theCFLAGS environment variable todisable C compiler optimisations (e.g. "-O0" for gcc, that'sminus-oh-zero), for example:
CFLAGS="-O0" make inplace
If you get errors about missing header files (e.g.Python.h orlibxml/xmlversion.h) then you need to make sure the developmentpackages of Python, libxml2 and libxslt are properly installed. OnLinux distributions, they are usually called something likelibxml2-dev orlibxslt-devel. If these packages wereinstalled in non-standard places, try passing the following option tosetup.py to make sure the right config is found:
python setup.py build --with-xslt-config=/path/to/xslt-config
If this doesn't help, you may have to add the location of the headerfiles to the include path like:
python setup.py build_ext -i -I /usr/include/libxml2
where the file is in/usr/include/libxml2/libxml/xmlversion.h
To use lxml.etree in-place, you can place lxml'ssrc directoryon your Python module search path (PYTHONPATH) and then importlxml.etree to play with it:
# cd lxml# PYTHONPATH=src pythonPython 2.7.2Type "help", "copyright", "credits" or "license" for more information.>>> from lxml import etree>>>
To make sure everything gets recompiled cleanly after changes, you canrunmake clean or delete the filesrc/lxml/etree.c.
The source distribution (tgz) and the source repository contain a testsuite for lxml. You can run it from the top-level directory:
python test.py
Note that the test script only tests the in-place build (see distutilsbuilding above), as it searches thesrc directory. You can use thefollowing one-step command to trigger an in-place build and test it:
make test
This also runs the ElementTree and cElementTree compatibility tests. To callthem separately, make sure you have lxml on your PYTHONPATH first, then run:
python selftest.py
and:
python selftest2.py
If the tests give failures, errors, or worse, segmentation faults, we'd reallylike to know. Please contact us on themailing list, and please specifythe version of lxml, libxml2, libxslt and Python you were using, as well asyour operating system type (Linux, Windows, MacOS-X, ...).
This is the procedure to make an lxml egg orwheel for your platform.It assumes that you havesetuptools ordistribute installed, as wellas thewheel package.
First, download the lxml-x.y.tar.gz release. This contains the pregeneratedC files so that you can be sure you build exactly from the release sources.Unpack them andcd into the resulting directory. Then, to build a wheel,simply run the command
python setup.py bdist_wheel
or, to build a statically linked wheel with all of libxml2, libxslt andfriends compiled in, run
python setup.py bdist_wheel --static-deps
The resulting .whl file will be written into thedist directory.
To build an egg file, run
python setup.py build_egg
If you are on a Unix-like platform, you can first build the extension modulesusing
python setup.py build
and thencd into the directorybuild/lib.your.platform to callstrip on any.so file you find there. This reduces the size ofthe binary distribution considerably. Then, from the package root directory,call
python setup.py bdist_egg
This will quickly package the pre-built packages into an egg file anddrop it into thedist directory.
Apple regularly ships new system releases with horribly outdatedsystem libraries. This is specifically the case for libxml2 andlibxslt, where the system provided versions are too old to build lxml.
While the Unix environment in MacOS-X makes it relatively easy toinstall Unix/Linux style package management tools and new software, itactually seems to be hard to get libraries set up for exclusive usagethat MacOS-X ships in an older version. Alternative distributions(like macports) install their libraries in addition to the systemlibraries, but the compiler and the runtime loader on MacOS still seesthe system libraries before the new libraries. This can lead toundebuggable crashes where the newer library seems to be loaded butthe older system library is used.
Apple discourages static building against libraries, which would helpworking around this problem. Apple does not ship static librarybinaries with its system and several package management systems followthis decision. Therefore, building static binaries requires buildingthe dependencies first. Thesetup.py script does thisautomatically when you call it like this:
python setup.py build --static-deps
This will download and build the latest versions of libxml2 andlibxslt from the official FTP download site. If you want to usespecific versions, or want to prevent any online access, you candownload bothtar.gz release files yourself, place them into asubdirectorylibs in the lxml distribution, and callsetup.pywith the desired target versions like this:
python setup.py build --static-deps \ --libxml2-version=2.7.8 \ --libxslt-version=1.1.27 \sudo python setup.py install
Instead ofbuild, you can use any target, likebdist_egg ifyou want to use setuptools to build an installable egg.
Note that this also works withpip. Since you can't passcommand line options in this case, you have to use an environmentvariable instead:
STATIC_DEPS=true pip install lxml
To install the package into the system Python package directory,run the installation with "sudo":
STATIC_DEPS=true sudo pip install lxml
TheSTATICBUILD environment variable is handled equivalently totheSTATIC_DEPS variable, but is used by some other extensionpackages, too.
Most operating systems have proper package management that makes installingcurrent versions of libxml2 and libxslt easy. The most famous exception isMicrosoft Windows, which entirely lacks these capabilities. To work aroundthe limits of this platform, lxml's installation can download pre-builtpackages of the dependencies and build statically against them. Assumingyou have a proper C compiler setup to build Python extensions, this shouldwork:
python setup.py bdist_wininst --static-deps
It should create a windows installer in thepkg directory.
Andreas Pakulat proposed the following approach.
In casedpkg-buildpackage tells you that some dependecies are missing, youcan either install them manually or runapt-getbuild-dep lxml.
That will give you .deb packages in the parent directory which can beinstalled usingdpkg-i.