Repository files navigation

NEWS 2023-01-11: Development occurs on themain branch.

The SPIR-V Tools project provides an API and commands for processing SPIR-Vmodules.

The project includes an assembler, binary module parser, disassembler,validator, and optimizer for SPIR-V. Except for the optimizer, all are basedon a common static library. The library contains all of the implementationdetails, and is used in the standalone tools whilst also enabling integrationinto other code bases directly. The optimizer implementation resides in itsown library, which depends on the core library.

The interfaces have stabilized:We don't anticipate making a breaking change for existing features.

SPIR-V is defined by the Khronos Group Inc.See theSPIR-V Registry for the SPIR-V specification,headers, and XML registry.

The official releases for SPIRV-Tools can be found on LunarG'sSDK download page.

For convenience, here are also links to the latest builds (HEAD).Those are untested automated builds. Those are not official releases, norare guaranteed to work. Official releases builds are in the Vulkan SDK.

More downloads

SeeCHANGES for a high level summary of recent changes, by version.

SPIRV-Tools project version numbers are of the formvyear.index and withan optional-dev suffix to indicate work in progress. For example, thefollowing versions are ordered from oldest to newest:

• v2016.0
• v2016.1-dev
• v2016.1
• v2016.2-dev
• v2016.2

Use the--version option on each command line tool to see the softwareversion. An API call reports the software version as a C-style string.

The official releases for SPIRV-Tools can be found on LunarG'sSDK download page.

You can find either the prebuilt, and QA tested binaries, or download theSDK Config, which lists the commits to use to build the release from scratch.

GitHub releases are deprecated, and we will not publish new releases untilfurther notice.

• Support for SPIR-V 1.0, through 1.5
  • Based on SPIR-V syntax described by JSON grammar files in theSPIRV-Headers repository.
  • Usually, support for a new version of SPIR-V is ready within days afterpublication.
• Support for extended instruction sets:
  • GLSL std450 version 1.0 Rev 3
  • OpenCL version 1.0 Rev 2
• Assembler only does basic syntax checking. No cross validation ofIDs or types is performed, except to check literal arguments toOpConstant,OpSpecConstant, andOpSwitch.
• Where tools expect binary input, a hex stream may be provided instead. Seespirv-dis --help.

Seedocs/syntax.md for the assembly language syntax.

The validator checks validation rules described by the SPIR-V specification.

Khronos recommends that tools that create or transform SPIR-V modules use thevalidator to ensure their outputs are valid, and that tools that consume SPIR-Vmodules optionally use the validator to protect themselves from bad inputs.This is especially encouraged for debug and development scenarios.

The validator has one-sided error: it will only return an error when it hasimplemented a rule check and the module violates that rule.

The validator is incomplete.See theCHANGES file for reports on completed work, andtheValidatorsub-project for plannedand in-progress work.

Note: The validator checks some Universal Limits, from section 2.17 of the SPIR-V spec.The validator will fail on a module that exceeds those minimum upper bound limits.The validator has been parameterized to allow larger values, for use when targetinga more-than-minimally-capable SPIR-V consumer.

Seetools/val/val.cpp or runspirv-val --help for the command-line help.

The optimizer is a collection of code transforms, or "passes".Transforms are written for a diverse set of reasons:

• To restructure, simplify, or normalize the code for further processing.
• To eliminate undesirable code.
• To improve code quality in some metric such as size or performance.Note: These transforms are not guaranteed to actually improve anygiven metric. Users should always measure results for their own situation.

As of this writing, there are 67 transforms including examples such as:

• Simplification
  • Strip debug info
  • Strip reflection info
• Specialization Constants
  • Set spec constant default value
  • Freeze spec constant to default value
  • FoldOpSpecConstantOp andOpSpecConstantComposite
  • Unify constants
  • Eliminate dead constant
• Code Reduction
  • Inline all function calls exhaustively
  • Convert local access chains to inserts/extracts
  • Eliminate local load/store in single block
  • Eliminate local load/store with single store
  • Eliminate local load/store with multiple stores
  • Eliminate local extract from insert
  • Eliminate dead instructions (aggressive)
  • Eliminate dead branches
  • Merge single successor / single predecessor block pairs
  • Eliminate common uniform loads
  • Remove duplicates: Capabilities, extended instruction imports, types, anddecorations.
• Normalization
  • Compact IDs
  • Canonicalize IDs
  • CFG cleanup
  • Flatten decorations
  • Merge returns
  • Convert AMD-specific instructions to KHR instructions
• Code improvement
  • Conditional constant propagation
  • If-conversion
  • Loop fission
  • Loop fusion
  • Loop-invariant code motion
  • Loop unroll
• Other
  • Graphics robust access
  • Upgrade memory model to VulkanKHR

Additionally, certain sets of transformations have been packaged intohigher-level recipes. These include:

• Optimization for size (spirv-opt -Os)
• Optimization for performance (spirv-opt -O)

For the latest list with detailed documentation, please refer toinclude/spirv-tools/optimizer.hpp.

For suggestions on using the code reduction options, please refer to thiswhite paper.

Note: The linker is still under development.

Current features:

• Combine multiple SPIR-V binary modules together.
• Combine into a library (exports are retained) or an executable (no symbolsare exported).

See theCHANGES file for reports on completed work, and theGeneralsub-project forplanned and in-progress work.

Note: The reducer is still under development.

The reducer simplifies and shrinks a SPIR-V module with respect to auser-suppliedinterestingness function. For example, given a largeSPIR-V module that cause some SPIR-V compiler to fail with a givenfatal error message, the reducer could be used to look for a smallerversion of the module that causes the compiler to fail with the samefatal error message.

To suggest an additional capability for the reducer,file anissue with"Reducer:" as the start of its title.

Note: The fuzzer is still under development.

The fuzzer applies semantics-preserving transformations to a SPIR-V binarymodule, to produce an equivalent module. The original and transformed modulesshould produce essentially identical results when executed on identical inputs:their results should differ only due to floating-point round-off, if at all.Significant differences in results can pinpoint bugs in tools that processSPIR-V binaries, such as miscompilations. Thismetamorphic testing approachis similar to the method used by theGraphicsFuzzproject for fuzzing of GLSL shaders.

To suggest an additional capability for the fuzzer,file anissue with"Fuzzer:" as the start of its title.

Note: The diff tool is still under development.

The diff tool takes two SPIR-V files, either in binary or text format andproduces a diff-style comparison between the two. The instructions between thesrc and dst modules are matched as best as the tool can, and output is produced(in src id-space) that shows which instructions are removed in src, added in dstor modified between them. The order of instructions are not retained.

Matching instructions between two SPIR-V modules is not trivial, and thus anumber of heuristics are applied in this tool. In particular, without debuginformation, match functions is nontrivial as they can be reordered. As such,this tool is primarily useful to produce the diff of two SPIR-V modules derivedfrom the same source, for example before and after a modification to the shader,before and after a transformation, or SPIR-V produced from different tools.

• Utility filters
• Build targetspirv-tools-vimsyntax generates filespvasm.vim.Copy that file into your$HOME/.vim/syntax directory to get SPIR-V assembly syntaxhighlighting in Vim. This build target is not built by default.

The SPIR-V Tools project is maintained by members of the The Khronos Group Inc.,and is hosted athttps://github.com/KhronosGroup/SPIRV-Tools.

Consider joining thepublic_spirv_tools_dev@khronos.org mailing list, viahttps://www.khronos.org/spir/spirv-tools-mailing-list/.The mailing list is used to discuss development plans for the SPIRV-Tools as an open source project.Once discussion is resolved,specific work is tracked via issues and sometimes in one of theprojects.

(To provide feedback on the SPIR-Vspecification, file an issue on theSPIRV-Headers GitHub repository.)

Seedocs/projects.md to see how we use theGitHub Projectfeatureto organize planned and in-progress work.

Contributions via merge request are welcome. Changes should:

• Be provided under theApache 2.0.
• You'll be prompted with a one-time "click-through"Khronos Open Source Contributor License Agreement(CLA) dialog as part of submitting your pull request orother contribution to GitHub.
• Include tests to cover updated functionality.
• C++ code should follow theGoogle C++ Style Guide.
• Code should be formatted withclang-format.kokoro/check-format/build.shshows how to download it. Note that we currently useclang-format version 5.0.0 for SPIRV-Tools. Settings are defined bythe included.clang-format file.

We intend to maintain a linear history on the GitHubmain branch.

Example of getting sources, assuming SPIRV-Tools is configured as a standalone project:

git clone https://github.com/KhronosGroup/SPIRV-Tools.git   spirv-toolscd spirv-tools# Check out sources for dependencies, at versions known to work together,# as listed in the DEPS file.python3 utils/git-sync-deps

For some kinds of development, you may need the latest sources from the third-party projects:

git clone https://github.com/KhronosGroup/SPIRV-Headers.git spirv-tools/external/spirv-headersgit clone https://github.com/google/googletest.git          spirv-tools/external/googletestgit clone https://github.com/google/effcee.git              spirv-tools/external/effceegit clone https://github.com/google/re2.git                 spirv-tools/external/re2git clone https://github.com/abseil/abseil-cpp.git          spirv-tools/external/abseil_cppgit clone https://github.com/microsoft/mimalloc.git         spirv-tools/external/mimalloc

Some tests depend on theEffcee library for stateful matching.Effcee itself depends onRE2, and RE2 depends onAbseil.

• If SPIRV-Tools is configured as part of a larger project that already usesEffcee, then that project should include Effcee before SPIRV-Tools.
• Otherwise, SPIRV-Tools expects Effcee sources to appear inexternal/effcee,RE2 sources to appear inexternal/re2, and Abseil sources to appear inexternal/abseil_cpp.

SPIRV-Tools may be configured to use themimalloc library to improve memoryallocation performance.

In the CMake build, usage of mimalloc is controlled by theSPIRV_TOOLS_USE_MIMALLOCoption. This variable defaults onON when building for Windows andOFF when buildingfor other platforms. Enabling this option on non-Windows platforms is supported and isexpected to work normally, but this has not been tested as thoroughly and extensively asthe Windows version. In the future, theSPIRV_TOOLS_USE_MIMALLOC option may default toON for non-Windows platforms as well.

In order to avoid unexpectedly changing allocation behavior of applications that linkSPIRV-Tools libraries statically, mimalloc is disabled by default on static libraries.The option 'SPIRV_TOOLS_USE_MIMALLOC_IN_STATIC_BUILD' can be used to force the usage ofmimalloc on static libraries.

Note: mimalloc is currently only supported when building with CMake. When using Bazel,mimalloc is not used.

• example: demo code of using SPIRV-Tools APIs
• external/googletest: Intended location for thegoogletest sources, not provided
• external/effcee: Location ofEffcee sources, if theeffcee libraryis not already configured by an enclosing project.
• external/re2: Location ofRE2 sources, if there2 library is not alreadyconfigured by an enclosing project.(The Effcee project already requires RE2.)
• external/abseil_cpp: Location ofAbseil sources, if Abseil isnot already configured by an enclosing project.(The RE2 project already requires Abseil.)
• external/mimalloc: Intended location formimalloc sources, not provided
• include/: API clients should add this directory to the include search path
• external/spirv-headers: Intended location forSPIR-V headers, not provided
• include/spirv-tools/libspirv.h: C API public interface
• source/: API implementation
• test/: Tests, using thegoogletest framework
• tools/: Command line executables

The project contains a number of tests, used to drive developmentand ensure correctness. The tests are written using thegoogletest framework. Thegoogletestsource is not provided with this project. There are two ways to enabletests:

• If SPIR-V Tools is configured as part of an enclosing project, then theenclosing project should configuregoogletest before configuring SPIR-V Tools.
• If SPIR-V Tools is configured as a standalone project, then download thegoogletest source into the<spirv-dir>/external/googletest directory beforeconfiguring and building the project.

Note: Prebuilt binaries are available from thedownloads page.

Firstget the sources.Then build using CMake, Bazel, Android ndk-build, or the Emscripten SDK.

You can build the project usingCMake:

cd<spirv-dir>mkdir build&&cd buildcmake [-G<platform-generator>]<spirv-dir>

Once the build files have been generated, build using the appropriate buildcommand (e.g.ninja,make,msbuild, etc.; this depends on the platformgenerator used above), or use your IDE, or use CMake to run the appropriate buildcommand for you:

cmake --build. [--config Debug]# runs `make` or `ninja` or `msbuild` etc.

The SPIR-V fuzzer,spirv-fuzz, can only be built via CMake, and is disabled bydefault. To build it, clone protobuf and use theSPIRV_BUILD_FUZZER CMakeoption, like so:

# In <spirv-dir> (the SPIRV-Tools repo root):git clone --depth=1 --branch v3.13.0.1 https://github.com/protocolbuffers/protobuf external/protobuf# In your build directory:cmake [-G<platform-generator>]<spirv-dir> -DSPIRV_BUILD_FUZZER=ONcmake --build. --config Debug

You can also add-DSPIRV_ENABLE_LONG_FUZZER_TESTS=ON to build additionalfuzzer tests.

You can also useBazel to build the project.

bazel build :all

The SPIRV-Tools core library can be built to a WebAssemblynode.jsmodule. The resultingSpirvTools WebAssembly module only exports methods toassemble and disassemble SPIR-V modules.

First, make sure you have theEmscripten SDK.Then:

cd<spirv-dir>./source/wasm/build.sh

The resulting node package, with JavaScript and TypeScript bindings, iswritten to<spirv-dir>/out/web.

Note: This builds the package locally. It doesnot publish it tonpm.

To test the result:

node ./test/wasm/test.js

For building and testing SPIRV-Tools, the following tools should beinstalled regardless of your OS:

• CMake: if using CMake for generating compilationtargets, you need to install CMake Version 2.8.12 or later.
• Python 3: for utility scripts and running the testsuite.
• Bazel (optional): if building the source with Bazel,you need to install Bazel Version 7.4.0 on your machine. Other versions mayalso work, but are not verified.
• Emscripten SDK (optional): if building theWebAssembly module.

SPIRV-Tools is regularly tested with the following compilers:

On Linux

• GCC version 9.4
• Clang version 10.0

On MacOS

• AppleClang 15.0

On Windows

• Visual Studio 2022

Note: Other compilers or later versions may work, but they are not tested.

The following CMake options are supported:

• SPIRV_BUILD_FUZZER={ON|OFF}, defaultOFF - Build the spirv-fuzz tool.
• SPIRV_COLOR_TERMINAL={ON|OFF}, defaultON - Enables color console output.
• SPIRV_SKIP_TESTS={ON|OFF}, defaultOFF- Build only the library andthe command line tools. This will prevent the tests from being built.
• SPIRV_SKIP_EXECUTABLES={ON|OFF}, defaultOFF- Build only the library, notthe command line tools and tests.
• SPIRV_USE_SANITIZER=<sanitizer>, default is no sanitizing - On UNIXplatforms with an appropriate version ofclang this option enables the useof the sanitizers documentedhere.This should only be used with a debug build.
• SPIRV_WARN_EVERYTHING={ON|OFF}, defaultOFF - On UNIX platforms enablemore strict warnings. The code might not compile with this option enabled.For Clang, enables-Weverything. For GCC, enables-Wpedantic.SeeCMakeLists.txt for details.
• SPIRV_WERROR={ON|OFF}, defaultON - Forces a compilation error on anywarnings encountered by enabling the compiler-specific compiler front-endoption. No compiler front-end options are enabled when this option is OFF.

Additionally, you can pass additional C preprocessor definitions to SPIRV-Toolsvia settingSPIRV_TOOLS_EXTRA_DEFINITIONS. For example, by setting it to/D_ITERATOR_DEBUG_LEVEL=0 on Windows, you can disable checked iterators anditerator debugging.

SPIR-V Tools supports building static librarieslibSPIRV-Tools.a andlibSPIRV-Tools-opt.a for Android. Using the Android NDK r25c or later:

cd <spirv-dir>export ANDROID_NDK=/path/to/your/ndk   # NDK r25c or latermkdir build && cd buildmkdir libsmkdir app$ANDROID_NDK/ndk-build -C ../android_test     \                      NDK_PROJECT_PATH=.      \                      NDK_LIBS_OUT=`pwd`/libs \                      NDK_APP_OUT=`pwd`/app

Occasionally the entries inDEPS will need to be updated. This is done ondemand when there is a request to do this, often due to downstream breakages.To updateDEPS, runutils/roll_deps.sh and confirm that tests pass.The script requires Chromium'sdepot_tools.

The internals of the library use C++17 features, and are exposed via both a Cand C++ API.

In order to use the library from an application, the include path should pointto<spirv-dir>/include, which will enable the application to include theheader<spirv-dir>/include/spirv-tools/libspirv.h{|pp} then linking againstthe static library in<spirv-build-dir>/source/libSPIRV-Tools.a or<spirv-build-dir>/source/SPIRV-Tools.lib.For optimization, the header file is<spirv-dir>/include/spirv-tools/optimizer.hpp, and the static library is<spirv-build-dir>/source/libSPIRV-Tools-opt.a or<spirv-build-dir>/source/SPIRV-Tools-opt.lib.

• SPIRV-Tools CMake target: Creates the static library:
  • <spirv-build-dir>/source/libSPIRV-Tools.a on Linux and OS X.
  • <spirv-build-dir>/source/libSPIRV-Tools.lib on Windows.
• SPIRV-Tools-opt CMake target: Creates the static library:
  • <spirv-build-dir>/source/libSPIRV-Tools-opt.a on Linux and OS X.
  • <spirv-build-dir>/source/libSPIRV-Tools-opt.lib on Windows.

The interfaces are still under development, and are expected to change.

There are five main entry points into the library in the C interface:

• spvTextToBinary: An assembler, translating text to a binary SPIR-V module.
• spvBinaryToText: A disassembler, translating a binary SPIR-V module totext.
• spvBinaryParse: The entry point to a binary parser API. It issues callbacksfor the header and each parsed instruction. The disassembler is implementedas a client ofspvBinaryParse.
• spvValidate implements the validator functionality.Incomplete
• spvValidateBinary implements the validator functionality.Incomplete

The C++ interface is comprised of three classes,SpirvTools,Optimizer andLinker, all in thespvtools namespace.

• SpirvTools providesAssemble,Disassemble, andValidate methods.
• Optimizer provides methods for registering and running optimization passes.
• Linker provides methods for combining together multiple binaries.

Command line tools, which wrap the above library functions, are provided toassemble or disassemble shader files. It's a convention to name SPIR-Vassembly and binary files with suffix.spvasm and.spv, respectively.

The assembler reads the assembly language text, and emits the binary form.

The standalone assembler is the executable calledspirv-as, and is located in<spirv-build-dir>/tools/spirv-as. The functionality of the assembler is implementedby thespvTextToBinary library function.

• spirv-as - the standalone assembler
  • <spirv-dir>/tools/as

Use option-h to print help.

The disassembler reads the binary form, and emits assembly language text.

The standalone disassembler is the executable calledspirv-dis, and is located in<spirv-build-dir>/tools/spirv-dis. The functionality of the disassembler is implementedby thespvBinaryToText library function.

• spirv-dis - the standalone disassembler
  • <spirv-dir>/tools/dis

Use option-h to print help.

The output includes syntax colouring when printing to the standard output stream,on Linux, Windows, and OS X.

The linker combines multiple SPIR-V binary modules together, resulting in a singlebinary module as output.

This is a work in progress.The linker does not support OpenCL program linking options related to mathflags. (See section 5.6.5.2 in OpenCL 1.2)

• spirv-link - the standalone linker
  • <spirv-dir>/tools/link

The optimizer processes a SPIR-V binary module, applying transformationsin the specified order.

This is a work in progress, with initially only few available transformations.

• spirv-opt - the standalone optimizer
  • <spirv-dir>/tools/opt

Warning: This functionality is under development, and is incomplete.

The standalone validator is the executable calledspirv-val, and is located in<spirv-build-dir>/tools/spirv-val. The functionality of the validator is implementedby thespvValidate library function.

The validator operates on the binary form.

• spirv-val - the standalone validator
  • <spirv-dir>/tools/val

The reducer shrinks a SPIR-V binary module, guided by a user-suppliedinterestingness test.

This is a work in progress, with initially only shrinks a module in a few ways.

• spirv-reduce - the standalone reducer
  • <spirv-dir>/tools/reduce

Runspirv-reduce --help to see how to specify interestingness.

The fuzzer transforms a SPIR-V binary module into a semantically-equivalentSPIR-V binary module by applying transformations in a randomized fashion.

This is a work in progress, with initially only a few semantics-preservingtransformations.

• spirv-fuzz - the standalone fuzzer
  • <spirv-dir>/tools/fuzz

Runspirv-fuzz --help for a detailed list of options.

The control flow dumper prints the control flow graph for a SPIR-V module as aGraphViz graph.

This is experimental.

• spirv-cfg - the control flow graph dumper
  • <spirv-dir>/tools/cfg

Warning: This functionality is under development, and is incomplete.

The diff tool produces a diff-style comparison between two SPIR-V modules.

• spirv-diff - the standalone diff tool
  • <spirv-dir>/tools/diff`

• spirv-lesspipe.sh - Automatically disassembles.spv binary files for theless program, on compatible systems. For example, set theLESSOPENenvironment variable as follows, assuming bothspirv-lesspipe.sh andspirv-dis are on your executable search path:

   export LESSOPEN='| spirv-lesspipe.sh "%s"'

  Then you page through a disassembled module as follows:

  less foo.spv

  • Thespirv-lesspipe.sh script will pass through any extra arguments tospirv-dis. So, for example, you can turn off colours and friendly IDnaming as follows:

    export LESSOPEN='| spirv-lesspipe.sh "%s" --no-color --raw-id'

• vim-spirv - A vim plugin whichsupports automatic disassembly of.spv files using the:edit command andassembly using the:write command. The plugin also provides additionalfeatures which include; syntax highlighting; highlighting of all ID's matchingthe ID under the cursor; and highlighting errors where theInstructionoperand ofOpExtInst is used without an appropriateOpExtInstImport.

• 50spirv-tools.el - Automatically disassembles '.spv' binary files whenloaded into the emacs text editor, and re-assembles them when saved,provided any modifications to the file are valid. This functionalitymust be explicitly requested by defining the symbolSPIRV_TOOLS_INSTALL_EMACS_HELPERS as follows:

  cmake -DSPIRV_TOOLS_INSTALL_EMACS_HELPERS=true ...

  In addition, this helper is only installed if the directory /etc/emacs/site-start.dexists, which is typically true if emacs is installed on the system.

  Note that symbol IDs are not currently preserved through a load/edit/save operation.This may change if the ability is added to spirv-as.

Tests are only built when googletest is found.

Usectest -j <num threads> to run all the tests. To run tests using all threads:

ctest -j$(nproc)

To run a single test target, usectest [-j <N>] -R <test regex>. For example,you can run allopt tests with:

ctest -R'spirv-tools-test_opt'

Usebazel test :all to run all tests. This will run tests in parallel by default.

To run a single test target, specify:my_test_target instead of:all. Test targetnames get printed when you runbazel test :all. For example, you can runopt_def_use_test with:

on linux:

bazeltest --cxxopt=-std=c++17 :opt_def_use_test

on windows:

bazeltest --cxxopt=/std:c++17 :opt_def_use_test

See theprojects pagesfor more information.

• The disassembler could emit helpful annotations in comments. For example:
  • Use variable name information from debug instructions to annotatekey operations on variables.
  • Show control flow information by annotatingOpLabel instructions withthat basic block's predecessors.
• Error messages could be improved.

This is a work in progress.

• The linker could accept math transformations such as allowing MADs, or othermath flags passed at linking-time in OpenCL.
• Linkage attributes can not be applied through a group.
• Check decorations of linked functions attributes.
• Remove dead instructions, such as OpName targeting imported symbols.

Full license terms are inLICENSE

Copyright (c) 2015-2016 The Khronos Group Inc.Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at    http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.