Part of the book series:Lecture Notes in Computer Science ((LNISA,volume 11832))
Included in the following conference series:
1132Accesses
Abstract
Theoretical analysis in [1] suggested that adversarially trained generative models are naturally inclined to learn distribution with low support. In particular, this effect is caused by the limited capacity of the discriminator network. To verify this claim, [2] proposed a statistical test based on the birthday paradox that partially confirmed the analysis. In this paper, we continue this line of work and develop a parameter-free and straightforward method to estimate the support size of an arbitrary decoder-based generative model. Our approach considers the decoder network from a geometric viewpoint and evaluates the support size as the volume of the manifold containing the generative model samples. Additionally, we propose a method to measure non-uniformity of a generative model that can provide additional insight into the model’s behavior. We then apply these tools to perform a quantitative comparison of common generative models.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 5719
- Price includes VAT (Japan)
- Softcover Book
- JPY 7149
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
\(ReLU(x) = \max (0, x)\).
References
Arora, S., Ge, R., Liang, Y., Ma, T., Zhang, Y.: Generalization and equilibrium in generative adversarial nets (GANs). arXiv preprintarXiv:1703.00573 (2017)
Arora, S., Zhang, Y.: Do GANs actually learn the distribution? An empirical study. arXiv preprintarXiv:1706.08224 (2017)
Ben-Israel, A.: The change-of-variables formula using matrix volume. SIAM J. Matrix Anal. Appl.21(1), 300–312 (1999)
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2172–2180 (2016)
Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. arXiv preprintarXiv:1605.08803 (2016)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Klambauer, G., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a nash equilibrium. arXiv preprintarXiv:1706.08500 (2017)
Kingma, D.P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., Welling, M.: Improved variational inference with inverse autoregressive flow. In: Advances in Neural Information Processing Systems, pp. 4743–4751 (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprintarXiv:1312.6114 (2013)
Lucic, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: Are GANs created equal. A large-scale study. arXiv e-prints (2017)
Metz, L., Poole, B., Pfau, D., Sohl-Dickstein, J.: Unrolled generative adversarial networks. arXiv preprintarXiv:1611.02163 (2016)
Odena, A., et al.: Is generator conditioning causally related to GAN performance? arXiv preprintarXiv:1802.08768 (2018)
Oord, A.V.D., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks. arXiv preprintarXiv:1601.06759 (2016)
Sajjadi, M.S., Bachem, O., Lucic, M., Bousquet, O., Gelly, S.: Assessing generative models via precision and recall. arXiv preprintarXiv:1806.00035 (2018)
Salakhutdinov, R., Larochelle, H.: Efficient learning of deep Boltzmann machines. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 693–700 (2010)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, pp. 2234–2242 (2016)
Spivak, M.: Calculus on Manifolds. W.A. Benjamin Inc., Massachusetts (1965)
Theis, L., Oord, A.V.D., Bethge, M.: A note on the evaluation of generative models. arXiv preprintarXiv:1511.01844 (2015)
Wu, Y., Burda, Y., Salakhutdinov, R., Grosse, R.: On the quantitative analysis of decoder-based generative models. arXiv preprintarXiv:1611.04273 (2016)
Acknowledgements
The authors would like to thank Ilya Tolstikhin for fruitful discussions. The work was partly supported by Ministry of Education and Science of the Russian Federation (grant 14.756.31.0001). Dmitry Vetrov was also partly supported by Samsung Research, Samsung Electronics.
Author information
Authors and Affiliations
National Research University Higher School of Economics, Moscow, Russia
Kirill Struminsky
Skolkovo Institute of Science and Technology, Moscow, Russia
Kirill Struminsky
Samsung-HSE Laboratory, National Research University Higher School of Economics, Moscow, Russia
Dmitry Vetrov
- Kirill Struminsky
You can also search for this author inPubMed Google Scholar
- Dmitry Vetrov
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toKirill Struminsky.
Editor information
Editors and Affiliations
RWTH Aachen University, Aachen, Germany
Wil M. P. van der Aalst
University of Ljubljana, Ljubljana, Slovenia
Vladimir Batagelj
National Research University Higher School of Economics, Moscow, Russia
Dmitry I. Ignatov
Krasovskii Institute of Mathematics and Mechanics, Yekaterinburg, Russia
Michael Khachay
National Research University Higher School of Economics, Moscow, Russia
Valentina Kuskova
University of Oslo, Oslo, Norway
Andrey Kutuzov
National Research University Higher School of Economics, Moscow, Russia
Sergei O. Kuznetsov
National Research University Higher School of Economics, Moscow, Russia
Irina A. Lomazova
Lomonosov Moscow State University, Moscow, Russia
Natalia Loukachevitch
LORIA, Vandœuvre-lès-Nancy, France
Amedeo Napoli
University of Florida, Gainesville, FL, USA
Panos M. Pardalos
Ca Foscari University of Venice, Venice, Italy
Marcello Pelillo
National Research University Higher School of Economics, Nizhny Novgorod, Russia
Andrey V. Savchenko
Kazan Federal University, Kazan, Russia
Elena Tutubalina
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Struminsky, K., Vetrov, D. (2019). A Simple Method to Evaluate Support Size and Non-uniformity of a Decoder-Based Generative Model. In: van der Aalst, W.,et al. Analysis of Images, Social Networks and Texts. AIST 2019. Lecture Notes in Computer Science(), vol 11832. Springer, Cham. https://doi.org/10.1007/978-3-030-37334-4_8
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-030-37333-7
Online ISBN:978-3-030-37334-4
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative