Repository files navigation

Unified Communication X (UCX) is anaward winning,optimized production proven-communication framework for modern, high-bandwidthand low-latency networks.

UCX exposes a set of abstract communication primitives that utilize the best ofavailable hardware resources and offloads. These include RDMA (InfiniBand and RoCE),TCP, GPUs, shared memory, and network atomic operations.

Please visit ourdocumentation sitefor more details.

Please review our"Membership Voluntary Consensus Standard" and"Export Compliant Contribution Submissions" policies.

• Using UCX
• Known issues
• Architecture
• Supported Transports
• Supported CPU Architectures
• Licenses
• Our Community
• Contributor Agreement and Guidelines
• Publications

Building UCX is typically a combination of running "configure" and "make".If using a release tarball execute the following commands to install theUCX system from within the directory at the top of the tree:

$ ./contrib/configure-release --prefix=/where/to/install$ make -j8$ make install

If directly cloning the git repository use:

$ ./autogen.sh$ ./contrib/configure-release --prefix=/where/to/install$ make -j8$ make install

NOTE: Compiling support for various networks or other specific hardware mayrequire additional command line flags when running configure.

$ ./autogen.sh$ ./contrib/configure-devel --prefix=$PWD/install-debug

*** NOTE: Developer builds of UCX typically include a large performancepenalty at run-time because of extra debugging code.

$ contrib/buildrpm.sh -s -b

$ dpkg-buildpackage -us -uc

$ make docs

Wiki page

Wiki page

Start server:

$ ./src/tools/perf/ucx_perftest -c 0

Connect client:

$ ./src/tools/perf/ucx_perftest<server-hostname> -t tag_lat -c 1

NOTE the-c flag sets CPU affinity. If running both >commands on same host, make sure you set the affinity to different CPU cores.

$ make -C test/gtesttest

• UCX version 1.8.0 has a bug that may cause data corruption when TCP transportis used in conjunction with shared memory transport. It is advised to upgradeto UCX version 1.9.0 and above. UCX versions released before 1.8.0 don't havethis bug.

• UCX may hang with glibc versions 2.25-2.29 due to known bugs in thepthread_rwlock functions. When such hangs occur, one of the UCX threads getsstuck in pthread_rwlock_rdlock (which is called by ucs_rcache_get), eventhough no other thread holds the lock. A related issue is reported inglibc Bug 23844.If this issue occurs, it is advised to use glibc version provided with yourOS distribution or build glibc from source using versions less than 2.25 orgreater than 2.29.

• Due to compatibility flaw when using UCX with rdma-core v22 settingUCX_DC_MLX5_RX_INLINE=0 is unsupported and will make DC transport unavailable.This issue is fixed in rdma-core v24 and backported to rdma-core-22.4-2.el7 rpm.Seeucx issue 5749 for moredetails.

• Infiniband
• Omni-Path
• RoCE
• Cray Gemini and Aries
• CUDA
• ROCm
• Shared Memory
  • posix, sysv,cma,knem, andxpmem
• TCP/IP

NOTE: UCX >= 1.12.0 requires rdma-core >= 28.0 or MLNX_OFED >= 5.0 forInfiniband andRoCE transports support.

• x86_64
• Power8/9
• Arm v8

UCX is licensed as:

• BSD3

• Project Website
• ReadTheDocs
• Github
• Software Releases
• Mailing List
• Twitter

In order to contribute to UCX, please sign up with an appropriateContributor Agreement.

All contributors have to comply with"Membership VoluntaryConsensus Standard" and"Export CompliantContribution Submissions" policies.

Follow theseinstructionswhen submitting contributions and changes.

To reference UCX in a publication, please use the following entry:

@inproceedings{shamis2015ucx,title={UCX: an open source framework for HPC network APIs and beyond},author={Shamis, Pavel and Venkata, Manjunath Gorentla and Lopez, M Graham and Baker, Matthew B and Hernandez, Oscar and Itigin, Yossi and Dubman, Mike and Shainer, Gilad and Graham, Richard L and Liss, Liran and others},booktitle={2015 IEEE 23rd Annual Symposium on High-Performance Interconnects},pages={40--43},year={2015},organization={IEEE}}

To reference the UCX website:

@misc{openucx-website,title ={{The Unified Communication X Library}},key ={{{The Unified Communication X Library}},    howpublished = {{\url{http://www.openucx.org}}}}