Repository files navigation

Vcflib and related tools are the workhorses in bioinformatics for processing the VCF variant calling format. See

A spectrum of free software tools for processing the VCF variant call format: vcflib, bio-vcf, cyvcf2, hts-nim and slivar.Garrison E, Kronenberg ZN, Dawson ET, Pedersen BS, Prins P (2022), PLoS Comput Biol 18(5): e1009123.https://doi.org/10.1371/journal.pcbi.1009123

(bibtex reference at the end of this page)

November 2024: a fresh release with, mostly, pangenome related fixes.

January 2023: this is vcflib's firstHumpty Dumpty release:vcfcreatemulti is the natural companion tovcfwave.Often variant callers are not perfect.vcfwave with its companion toolvcfcreatemulti can take an existing VCF file that contains multiple complex overlapping and even nested alleles and, unlike Humpty Dumpty, take them apart and put them together again.Thereby, hopefully, creating sane VCF output that is useful for analysis and getting rid of false positives.

We created these tools by including the state-of-the-artbiWFA wavefront aligner.The tools are particularly useful for the output from structural variation callers and pangenome genotypers, such as used by the Human Pangenome Reference Consortium (HPRC) because of overlapping ALT segments.

See alsoRELEASE_NOTES.md

May 2022: thevcflib paper has been published on PLoS Computational Biology!

April 2022: vcflib has just gone pangenome!

By introducing the wavefront algorithm we can now realign long sequences and reduce call complexity (and FPs!) introduced by pangenome variant callers using the newvcfwave tool.

See alsoRELEASE_NOTES.md

TheVariant Call Format(VCF)is a flat-file, tab-delimited textual format thatdescribes reference-indexed variations between individuals. VCFprovides a common interchange format for the description of variationin individuals and populations of samples, and has become thede facto standard reporting format for a wide array of genomicvariant detectors.

vcflib provides methods to manipulate and interpret sequence variationdescribed by VCF. It is both:

• an API for parsing and operating on records of genomic variation asit can be described by the VCF format
• a collection of command-line utilities for executing complexmanipulations on VCF files

vclib is both a library (with an API) and a collection of usefultools. The API provides a quick and extremely permissive method toread and write VCF files. Extensions and applications of the libraryprovided in the included utilities (*.cpp) comprise the vast bulk ofthe library's utility.

We have also added infrastructure to write Python bindings. Seebelow.

Short index:

• Install
• Usage
• TOOLS
  • Filter
  • Metrics
  • Phenotype
  • Genotype
  • Transformation
  • Statistics
  • Scripts
  • Python bindings
• Link library
• Build from source
• Development
• LICENSE
• Credit work
  • Bibtex reference

For latest updates seeRELEASE NOTES.

Conda installs in user land without root access

conda install -c bioconda vcflib

Homebrew installs on Linux and Mac OSX

brew install brewsci/bio/vcflib

For Debian and Ubuntu

apt-get install libvcflib-tools libvcflib-dev

We develop against guix and vcflib is packaged as

guix package -i vcflib

See also the Guix shell below.

Users are encouraged to drive the utilities in the library in astreaming fashion, using Unix pipes to fully utilize resources onmulti-core systems. Piping provides a convenient method to interfacewith other libraries (vcf-tools, BedTools, GATK, htslib,bio-vcf, bcftools,freebayes) which interface via VCFfiles, allowing the composition of an immense variety of processingfunctions. Examples can be found in the scripts,e.g../scripts.

See alsovcflib.md.

The vcflib source repository contains a number of additional scripts.Click on the link to see the source code.

vcflib has rudimentary python bindings, but the are easy to build up on. Seepyvcflib.

VCFLIB uses the cmake build system, after a recursive checkout of the sources make the files in the 'build' directory with:

git clone --recursive https://github.com/vcflib/vcflib.gitcd vcflibmkdir -p build&&cd buildcmake -DCMAKE_BUILD_TYPE=Debug -DZIG=OFF -DOPENMP=OFF ..cmake --build.cmake --install.

and to run the tests

ctest. --verbose

Executables are built into the./build directory in the repository.

Note, if you have an existing repo update submodules with

git submodule update --init --recursive --progresscd buildcmake --build. --target clean

The most interesting one, perhaps, is wfa2lib. Bundling is no longer the default, but it is a submodule that will install an optimized libwfa2.so. Set up compilation with

cmake -DCMAKE_BUILD_TYPE=Release -DWFA_GITMODULE=ON  ..cmake --build.ctest.cmake --install.

Build dependencies can be viewed in the github-CI scripts (see badges above), as well asguix.scm used by us to create the build environment (for instructions see the header of guix.scm). Essentially:

• cmake
• C++ compiler
• htslib
• tabixpp
• WFA2 (wfa2lib)
• pybind11 (for testing)

For include files add

• libhts-dev
• libtabixpp-dev
• libtabixpp0

And for some of the VCF executables

• zig 0.13 (disable with cmake -DZIG=OFF)
• python
• perl

To use Zig compiled tools, such as vcfcreatemulti, make sure a recentzig compiler is in thePATH.Thezig dependency is for the recently upgradedvcfcreatemulti that can be run after vcfwave to combine overlapping records. It is unfortunate that zig is not part of some large Linux distributions.

Note you need to add the full path before running cmake and make. From the build dir this should work:

cd buildexport PATH=$(pwd)/../zig-linux-x86_64-0.13.0:$PATHzig versioncmake  -DCMAKE_BUILD_TYPE=Debug -DWFA_GITMODULE=1 ..make VERBOSE=1

More onzig can be found in the souce codeREADME.

You may encounter something like libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by ...vcflib/build/pyvcflib.cpython-310-x86_64-linux-gnu.so). This is caused by the python pyvcflib not running against a python interpreter compiled with the same libstdc++ dependency. That is a bit nasty to solve. For now make sure to match C++ compilers for the python module.

Check out htslib in tabixpp (recursively) and

cmake -DHTSLIB_LOCAL:STRING=./htslib/ ..cmake --build .

The standard build createsbuild/vcflib.a. Take a hint from thecmake file that builds all the vcflib tools.

Distros, such as Debian, should build with something like

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DZIG=OFF ..

See the CMakeLists.txt header for more.

Seevcfecho.cpp for basic usage.Variant.h andVariant.cppdescribe methods available in the API. vcflib is incorporated intoseveral projects, such asfreebayes, which may providea point of reference for prospective developers. Note vcflib containssubmodules (git repositories) comprising some dependencies. A fullGuix development environment we use is definedhere.

vcflib uses different test systems. The most important one is thedoctest because itdoubles as documentation. For an example seevcf2tsv.md which can be run from thecommand line with

cdtestpython3 -m doctest -o NORMALIZE_WHITESPACE -o REPORT_UDIFF pytest/vcf2tsv.md

We also added support for python bindings and unit tests. Seerealign.py for an example.

The developers are on the vcflibmatrix channel. Please do not use the github issue tracker for support issues!

To contribute code to vcflib send a github pull request. We may askyou to add a working test case as described in 'adding tests'.

This software is distributed under the free softwareMIT LICENSE.

Citations are the bread and butter of Science. If you are using thissoftware in your research and want to support our future work, pleasecite the following publication:

Please cite:

A spectrum of free software tools for processing the VCF variant call format: vcflib, bio-vcf, cyvcf2, hts-nim and slivar.Garrison E, Kronenberg ZN, Dawson ET, Pedersen BS, Prins P (2022), PLoS Comput Biol 18(5): e1009123.https://doi.org/10.1371/journal.pcbi.1009123

@article{10.1371/journal.pcbi.1009123,doi ={10.1371/journal.pcbi.1009123},author ={Garrison, Erik AND Kronenberg, Zev N. AND Dawson, Eric T. AND Pedersen, Brent S. AND Prins, Pjotr},journal ={PLOS Computational Biology},publisher ={Public Library of Science},title ={A spectrum of free software tools for processing the VCF variant call format: vcflib, bio-vcf, cyvcf2, hts-nim and slivar},year ={2022},month ={05},volume ={18},url ={https://doi.org/10.1371/journal.pcbi.1009123},pages ={1-15}}

Below the prepublished version of our paper

@article {Garrison2021.05.21.445151,author ={Garrison, Erik and Kronenberg, Zev N. and Dawson, Eric T. and Pedersen, Brent S. and Prins, Pjotr},title ={Vcflib and tools for processing the VCF variant call format},elocation-id ={2021.05.21.445151},year ={2021},doi ={10.1101/2021.05.21.445151},publisher ={Cold Spring Harbor Laboratory},URL ={https://www.biorxiv.org/content/early/2021/05/23/2021.05.21.445151},eprint ={https://www.biorxiv.org/content/early/2021/05/23/2021.05.21.445151.full.pdf},journal ={bioRxiv}}