Aneural network is a group of interconnected units calledneurons that send signals to one another. Neurons can be eitherbiological cells ormathematical models. While individual neurons are simple, many of them together in a network can perform complex tasks. There are two main types of neural networks.
In the context of biology, a neural network is a population of biologicalneurons chemically connected to each other bysynapses. A given neuron can be connected to hundreds of thousands of synapses.[1]Each neuron sends and receiveselectrochemical signals calledaction potentials to its connected neighbors. A neuron can serve anexcitatory role, amplifying and propagating signals it receives, or aninhibitory role, suppressing signals instead.[1]
Populations of interconnected neurons that are smaller than neural networks are calledneural circuits. Very large interconnected networks are calledlarge scale brain networks, and many of these together formbrains andnervous systems.
Signals generated by neural networks in the brain eventually travel through the nervous system and acrossneuromuscular junctions tomuscle cells, where they cause contraction and thereby motion.[2]
In machine learning, a neural network is an artificial mathematical model used to approximate nonlinear functions. While early artificial neural networks were physical machines,[3] today they are almost always implemented insoftware.
Neurons in an artificial neural network are usually arranged into layers, with information passing from the first layer (the input layer) through one or more intermediate layers (the hidden layers) to the final layer (the output layer).[4]The "signal" input to each neuron is a number, specifically alinear combination of the outputs of the connected neurons in the previous layer. The signal each neuron outputs is calculated from this number, according to itsactivation function. The behavior of the network depends on the strengths (orweights) of the connections between neurons. A network is trained by modifying these weights throughempirical risk minimization orbackpropagation in order to fit some preexisting dataset.[5]
The termdeep neural network refers to neural networks that have more than three layers, typically including at least two hidden layers in addition to the input and output layers.
Neural networks are used to solve problems inartificial intelligence, and have thereby found applications in many disciplines, includingpredictive modeling,adaptive control,facial recognition,handwriting recognition,general game playing, andgenerative AI.
The theoretical base for contemporary neural networks was independently proposed byAlexander Bain in 1873[6] andWilliam James in 1890.[7] Both posited that human thought emerged from interactions among large numbers of neurons inside the brain. In 1949,Donald Hebb describedHebbian learning, the idea that neural networks can change and learn over time by strengthening a synapse every time a signal travels along it.[8] In 1956,Svaetichin discovered the functioning of second order retinal cells (Horizontal Cells), which were fundamental for the understanding of neural networks.
Artificial neural networks were originally used to model biological neural networks starting in the 1930s under the approach ofconnectionism. However, starting with the invention of theperceptron, a simple artificial neural network, byWarren McCulloch andWalter Pitts in 1943,[9] followed by the implementation of one in hardware byFrank Rosenblatt in 1957,[3]artificial neural networks became increasingly used for machine learning applications instead, and increasingly different from their biological counterparts.