Repository files navigation

A library for manifold-constrained optimization in TensorFlow.

To install the latest development version from GitHub:

pip install git+https://github.com/master/tensorflow-riemopt.git

To install a package from PyPI:

pip install tensorflow-riemopt

The core package implements concepts in differential geometry, such asmanifolds and Riemannian metrics with associated exponential and logarithmicmaps, geodesics, retractions, and transports. For manifolds, where closed-formexpressions are not available, the library provides numerical approximations.

importtensorflow_riemoptasriemoptS=riemopt.manifolds.Sphere()x=S.projx(tf.constant([0.1,-0.1,0.1]))u=S.proju(x,tf.constant([1.,1.,1.]))v=S.proju(x,tf.constant([-0.7,-1.4,1.4]))y=S.exp(x,v)u_=S.transp(x,y,u)v_=S.transp(x,y,v)

• manifolds.Cholesky - manifold of lower triangular matrices with positive diagonal elements
• manifolds.Euclidian - unconstrained manifold with the Euclidean metric
• manifolds.Grassmannian - manifold ofp-dimensional linear subspaces of then-dimensional space
• manifolds.Hyperboloid - manifold ofn-dimensional hyperbolic space embedded in then+1-dimensional Minkowski space
• manifolds.Poincare - the Poincaré ball model of the hyperbolic space
• manifolds.Product - Cartesian product of manifolds
• manifolds.SPDAffineInvariant - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
• manifolds.SPDLogCholesky - SPD manifold with the Log-Cholesky metric
• manifolds.SPDLogEuclidean - SPD manifold with the Log-Euclidean metric
• manifolds.SpecialOrthogonal - manifold of rotation matrices
• manifolds.Sphere - manifold of unit-normalized points
• manifolds.StiefelEuclidean - manifold of orthonormalp-frames in then-dimensional space endowed with the Euclidean metric
• manifolds.StiefelCanonical - Stiefel manifold with the canonical metric
• manifolds.StiefelCayley - Stiefel manifold the retraction map via an iterative Cayley transform

Constrained optimization algorithms work as drop-in replacements for Kerasoptimizers for sparse and dense updates in both Eager and Graph modes.

• optimizers.RiemannianSGD - Riemannian Gradient Descent
• optimizers.RiemannianAdam - Riemannian Adam and AMSGrad
• optimizers.ConstrainedRMSProp - Constrained RMSProp

• layers.ManifoldEmbedding - constrainedkeras.layers.Embedding layer

• ANTHEM - Choudhary, Nurendra, Nikhil Rao, Sumeet Katariya, Karthik Subbian, and Chandan K. Reddy. "ANTHEM: Attentive Hyperbolic Entity Model for Product Search." In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022.
• SPDNet - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
• LieNet - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
• GrNet - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
• Hyperbolic Neural Network - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
• Poincaré GloVe - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.

If you find TensorFlow RiemOpt useful in your research, please cite:

@misc{smirnov2021tensorflow,      title={TensorFlow RiemOpt: a library for optimization on Riemannian manifolds},      author={Oleg Smirnov},      year={2021},      eprint={2105.13921},      archivePrefix={arXiv},      primaryClass={cs.MS}}

TensorFlow RiemOpt was inspired by many similar projects:

• Manopt, a matlab toolbox for optimization on manifolds
• Pymanopt, a Python toolbox for optimization on manifolds
• Geoopt: Riemannian Optimization in PyTorch
• Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds

The code is MIT-licensed.