Repository files navigation

This project is awork-in-progress, which provides aflexible framework toexperiment and run neural network of heterogeneous topologies.A preliminary release will be visible soon.neuron is written in Scala which adopts the so-called"define-by-run" scheme.

• template vs. module
• neural networkoperators
• autoencoders (w or w/o tiled weight)
• activation functions: logistic, tanh, ReLU, softplus
• metrics (L1, L2, Mahalanobis, Softmax)
• regularization: weight decay, activation sparsity, dropout, maxout
• data parallel framework: atomic parameters + distributed states
• optimization: LBFGS, SGD, SAGD, SGD with momentum
• recursive neural network

The simplest example to train a regularized multi-layer perceptronto predict handwritten digits from MNIST dataset. It takes around ten minutes.

packageneuron.examplesimportneuron.core._importneuron.math._objectMLP_MNISTextendsWorkspacewithOptimizable {defmain(args:Array[String]):Unit= {// set @MLP=784-200-10, @weight_decay=1E-4      nn= (newRegularizedLinearNN(200,10,1E-4)**newSingleLayerNeuralNetwork(200)**newRegularizedLinearNN(784,200,1E-4)).create()// nn is declared in trait @Optimizable// load standard MNIST training dataval (xData, yData)=LoadData.mnistDataM("std","train")// generate random weight and initializevaltheta0= nn.getRandomWeights("get random weights").toWeightVector()      nn.setWeights("set weight", theta0);// full-batch training (@maxIter=200, @distance=SoftMaxDistance)val (_, theta)= trainx(xData, yData, theta0,200,SoftMaxDistance)// load standard MNIST testing dataval (xDataTest, yDataTest)=LoadData.mnistDataM("std","t10k")// estimate accuracyvalaccuracy= (yDataTest.argmaxCol() countEquals nn(xDataTest,null).argmaxCol())/ xDataTest.cols.toDouble      println(accuracy)    }}/* Accuracy: 0.9806*/

Also have a look at

• Basics: explains the most fundamental ideas for the use of neuron, and why they featured neuron as a good choice for prototyping neural networks that leverage flexibility, simplicity and efficiency.
• Auto-Encoder: a special family of unsupervised neural network.
• Examples: we have more examples under foldersrc/main/scala/neuron/tutorials/
• Scaladoc: TBA

• How is neuron different from other deep learning libraries (such as theano, torch7, etc), besides it is Scala based?

  We argue that not only the number of parameters contributes to the representation ability of neural network, but also its infrastructure (network topology, train strategy, etc.) Neuron focuses on fast prototyping novel network architecture. Using Scala, we attempt to make the implementation of neural network in a mixed functional and imperative way ... though neuron is not at the mature shape to be industrial proven.

• How is the speed of neruon?

  Neuron is currently backed bybreeze for numerical computation, which should be fast. And the extra cost for data control flow is minimized. Neuron provides convenientdata parallelization.

• Breeze andNak: a set of libraries for machine learning and numerical computing
• UFLDL Tutorial: a Stanford course, find solutions atGithub
• ScalaSTM: a lightweight software transactional memory for Scala

The MIT License (MIT)

Copyright (c) 2014 - 2015 Jianbo Ye

Permission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the "Software"), to dealin the Software without restriction, including without limitation the rightsto use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included inall copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS INTHE SOFTWARE.