Throughout the ages, man has continuously been involved with the process of optimization. In its earliest form, optimization consisted of unscientific rituals and prejudices like pouring libations and sacrificing animals to the gods, consulting the oracles, observing the positions of the stars, and watching the flight of birds. When the circumstances were appropriate, the timing was thought to be auspicious (or optimum) for planting the crops or embarking on a war. As the ages advanced and the age of reason prevailed, unscientific rituals were replaced by rules of thumb and later, with the development of mathematics, mathematical calculations began to be applied.

1. 1.

   See Sect. A.8 for more details on vector and matrix norms. Appendix A also deals with other aspects of linear algebra that are important to optimization.

2. 2.

   The above notation for a set will be used consistently throughout the book.

Authors and Affiliations

1. Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC, Canada

   Andreas Antoniou & Wu-Sheng Lu

Corresponding author

Correspondence toAndreas Antoniou.

Reprints and permissions

© 2021 Springer Science+Business Media, LLC, part of Springer Nature

Policies and ethics