Aneural Turing machine (NTM) is arecurrent neural network model of aTuring machine. The approach was published byAlex Graves et al. in 2014.[1] NTMs combine the fuzzypattern matching capabilities ofneural networks with thealgorithmic power ofprogrammable computers.
An NTM has a neural network controller coupled toexternal memory resources, which it interacts with through attentional mechanisms. The memory interactions are differentiable end-to-end, making it possible to optimize them usinggradient descent.[2] An NTM with along short-term memory (LSTM) network controller can infer simple algorithms such as copying, sorting, and associative recall from examples alone.[1]
The authors of the original NTM paper did not publish theirsource code.[1] The first stable open-source implementation was published in 2018 at the 27th International Conference on Artificial Neural Networks, receiving a best-paper award.[3][4][5] Other open source implementations of NTMs exist but as of 2018 they are not sufficiently stable for production use.[6][7][8][9][10][11][12] The developers either report that thegradients of their implementation sometimes becomeNaN during training for unknown reasons and cause training to fail;[10][11][9] report slow convergence;[7][6] or do not report the speed of learning of their implementation.[12][8]
Differentiable neural computers are an outgrowth of Neural Turing machines, withattention mechanisms that control where the memory is active, and improve performance.[13]
