SciPyOptimizers
Optimizers in SciPy
Optimizers are a set of procedures defined in SciPy that either find the minimum value of a function, or the root of an equation.
Optimizing Functions
Essentially, all of the algorithms in Machine Learning are nothing more than a complex equation that needs to be minimized with the help of given data.
Roots of an Equation
NumPy is capable of finding roots for polynomials and linear equations, but it can not find roots fornon linear equations, like this one:
x + cos(x)
For that you can use SciPy'soptimize.root function.
This function takes two required arguments:
fun - a function representing an equation.
x0 - an initial guess for the root.
The function returns an object with information regarding the solution.
The actual solution is given under attributex of the returned object:
Example
Find root of the equationx + cos(x):
from math import cos
def eqn(x):
return x + cos(x)
myroot = root(eqn, 0)
print(myroot.x)
Note: The returned object has much more information about the solution.
Example
Print all information about the solution (not justx which is the root)
Minimizing a Function
A function, in this context, represents a curve, curves havehigh points andlow points.
High points are calledmaxima.
Low points are calledminima.
The highest point in the whole curve is calledglobal maxima, whereas the rest of them are calledlocal maxima.
The lowest point in whole curve is calledglobal minima, whereas the rest of them are calledlocal minima.
Finding Minima
We can usescipy.optimize.minimize() function to minimize the function.
Theminimize() function takes the following arguments:
fun - a function representing an equation.
x0 - an initial guess for the root.
method - name of the method to use. Legal values:
'CG'
'BFGS'
'Newton-CG'
'L-BFGS-B'
'TNC'
'COBYLA'
'SLSQP'
callback - function called after each iteration of optimization.
options - a dictionary defining extra params:
{
"disp": boolean - print detailed description
"gtol": number - the tolerance of the error
}Example
Minimize the functionx^2 + x + 2 withBFGS:
def eqn(x):
return x**2 + x + 2
mymin = minimize(eqn, 0, method='BFGS')
print(mymin)
