Repository files navigation

Illinois Extended Reality testbed or ILLIXR (pronounced like elixir) isthe first fully open-source Extended Reality (XR) system and testbed.The modular, extensible, andOpenXR-compatible ILLIXR runtimeintegrates state-of-the-art XR components into a complete XR system.The testbed is part of the broaderILLIXR consortium,an industry-supported community effort to democratize XR systemsresearch, development, and benchmarking.

You can find the complete ILLIXR systemhere.

ILLIXR also provides its components in standalone configurations to enable architects andsystem designers to research each component in isolation.The standalone components are packaged together in the as of thev3.1.0 release of ILLIXR.

ILLIXR's modular and extensible runtime allows adding new components and swapping differentimplementations of a given component.ILLIXR currently contains the following components:

• Perception

  • Eye Tracking
    1. RITNet **
  • Scene Reconstruction
    1. ElasticFusion **
    2. KinectFusion **
  • Simultaneous Localization and Mapping
    1. OpenVINS **
  • Cameras and IMUs
    1. ZED Mini
    2. Intel RealSense

• Visual

  • Chromatic aberration correction
  • Computational holography for adaptive multi-focal displays **
  • Lens distortion correction
  • Asynchronous Reprojection (TimeWarp)

• Aural

  • Audio encoding **
  • Audio playback **

(** Source is hosted in an external repository under theILLIXR project.)

We continue to add more components (new components and new implementations).

Many of the current components of ILLIXR were developed by domain experts and obtained frompublicly available repositories.They were modified for one or more of the following reasons: fixing compilation, adding features,or removing extraneous code or dependencies.Each component not developed by us is available as a forked GitHub repository forproper attribution to its authors.

Apaper with details on ILLIXR, including its components, runtime, telemetry support,and a comprehensive analysis of performance, power, and quality on desktop and embedded systems.

Atalk presented at NVIDIA GTC'21 describing ILLIXR and announcing the ILLIXR consortium:Video.Slides.

Ademo of an OpenXR application running with ILLIXR.

For more up-to-date list of related papers, demos, and talks, please visitillixr.org.

TheILLIXR consortium is an industry-supported community effort to democratizeXR systems research, development, and benchmarking.Visit ourwebsite for more information.

The ILLIXR consortium is also holding a biweekly consortium meeting. For past meetings, for more information, past meeting recordings, and request for presenting, please visithere. Please join ourDiscord for announcement.

We request that you cite our followingpaper when you use ILLIXR for a publication.We would also appreciate it if you send us a citation once your work has been published.

@ARTICLE {illixr,author = {M. Huzaifa and R. Desai and S. Grayson and X. Jiang and Y. Jing and J. Lee and F. Lu and Y. Pang and J. Ravichandran and F. Sinclair and B. Tian and H. Yuan and J. Zhang and S. V. Adve},journal = {IEEE Micro},title = {ILLIXR: An Open Testbed to Enable Extended Reality Systems Research},year = {2022},volume = {42},number = {04},issn = {1937-4143},pages = {97-106},abstract = {We present Illinois Extended Reality testbed (ILLIXR), the first fully open-source XR system and research testbed. ILLIXR enables system innovations with end-to-end co-designed hardware, compiler, OS, and algorithms, and driven by end-user perceived Quality-of-Experience (QoE) metrics. Using ILLIXR, we provide the first comprehensive quantitative analysis of performance, power, and QoE for a complete XR system and its individual components. We describe several implications of our results that propel new directions in architecture, systems, and algorithms research for domain-specific systems in general, and XR in particular.},keywords = {x reality;pipelines;measurement;visualization;cameras;runtime;headphones},doi = {10.1109/MM.2022.3161018},publisher = {IEEE Computer Society},address = {Los Alamitos, CA, USA},month = {jul}}

For more information, see ourGetting Started page.

The ILLIXR project started inSarita Adve’s research group,co-led by PhD candidate Muhammad Huzaifa, at the University of Illinois at Urbana-Champaign.Other major contributors includeRishi Desai,Doug Friedel,Samuel Grayson,Xutao Jiang,Ying Jing,Jae Lee,Fang Lu,Yihan Pang,Joseph Ravichandran,Giordano Salvador,Finn Sinclair,Boyuan Tian,Henghzhi Yuan,andJeffrey Zhang.

ILLIXR came together after many consultations with researchers and practitioners in many domains:audio,graphics,optics,robotics,signal processing,andextended reality systems.We are deeply grateful for all of these discussions and specifically to the following:Wei Cu,Aleksandra Faust,Liang Gao,Matt Horsnell,Amit Jindal,Steve LaValle,Steve Lovegrove,Andrew Maimone,Vegard Øye,Martin Persson,Archontis Politis,Eric Shaffer,Paris Smaragdis,Sachin Talathi,andChris Widdowson.

Our OpenXR implementation is derived fromMonado.We are particularly thankful to Jakob Bornecrantz and Ryan Pavlik.

The development of ILLIXR was supported bythe Applications Driving Architectures (ADA) Research Center(a JUMP Center co-sponsored by SRC and DARPA),the Center for Future Architectures Research (C-FAR, a STARnet research center),a Semiconductor Research Corporation program sponsored by MARCO and DARPA,andby a Google Faculty Research Award.The development of ILLIXR was also aided by generous hardware and software donationsfrom ARM and NVIDIA.Facebook Reality Labs provided theOpenEDS Semantic Segmentation Dataset.

Wesley Darvin came up with the name for ILLIXR.

ILLIXR is available as open-source software under the permissiveUniversity of Illinois/NCSA Open Source License.As mentioned above, ILLIXR largely consists of components developed by domain experts andmodified for the purposes of inclusion in ILLIXR.However, ILLIXR does contain software developed solely by us.The NCSA license is limited to only this software.The external libraries and software included in ILLIXR each have their own licenses andmust be used according to those licenses:

¹ Current as of April 14, 2024.

² Also known as the Simplified BSD License.

³ This software is covered by multiple open source licenses, see the link for details.

The optionalOpen-VINS plugin (available from a secondary repository) is licensed underGPL v3.0.

Note that ILLIXR's extensibility allows the source to be configured and compiled using onlypermissively licensed software.

Whether you are a computer architect, a compiler writer, a systems person, work on XR related algorithmsor applications, or just anyone interested in XR research, development, or products,we would love to hear from you and hope you will contribute!You can jointheILLIXR consortium,Discord,ormailing list,or send us anemail,or just send us a pull request!