Repository files navigation

The original RNNLIB is hosted athttp://sourceforge.net/projects/rnnlwhile this "fork" is created to repeat results for theonline handwriting prediction and synthesis reported inhttp://arxiv.org/abs/1308.0850. The later by now is Alex Graves's classicpaper on LSTM networks showing of what RNN can learn about thestructure present in the sequential input.

Building rnnlib requires the following:

• C++11 compiler
• fortran for building OpenBLAS
• cmake
• libcurl
• automake
• libtool
• texinfo

In addition, the following python packages are needed for the auxiliary scripts in the 'utils' directory:

• SciPy
• PyLab
• PIL

And this package is needed to create and manipulate netcdf data files with python, and to run the experiments in the 'examples' directory:

• ScientificPython (NOT Scipy)

To build RNNLIB do

$ cmake -DCMAKE_BUILD_TYPE=Release .$ cmake --build .

Cmake run creates the binary files 'rnnlib', 'rnnsynth' and 'gradient_check' in the current directory.

It is recommended that you add the directory containing the 'rnnlib' binary to your path,as otherwise the tools in the 'utilities' directory will not work.

Project files for the integrated development environments can be generated by cmake. Run cmake --helpto get list of supported IDEs.

Step in to examples/online_prediction and go through few steps below to prepare thetraining data, train the model and eventually plot the results of the synthesis

Start by registering and downloading pen strokes data fromhttp://www.iam.unibe.ch/~fkiwww/iamondb/data/lineStrokes-all.tar.gzText lables for strokes can be found herehttp://www.iam.unibe.ch/~fkiwww/iamondb/data/ascii-all.tar.gzThen unzip ./lineStrokes and ./ascii under examples/online_prediction.Data format in the downloaded files can not be used as isand requires further preprocessing to convert pen coordinates to offsets fromprevious point and merge them into the single file of netcdf format.

Run ./build_netcdf.sh to split dataset to training and validation sets.The same script does all necessary preprocessing including normalisationof the input and makes corresponding online.nc and online_validation.ncfiles for use with rnnlib .

Each point in the input sequences from online.nc consists of three numbers:the x and y offset from the previous point, and the binary end-of-stroke feature.

To gain some confidence that the build is fine run the gradient check:

gradient_check --autosave=false check_synth2.config

The training goes in two steps. First it is done without weights regularizationand then repeated again with adaptive weight noise (MDL in rnnlib terms) from thebest network recorded by step one. Training with MDL from the beginning will havetoo slow convergence rate.

rnnlib --verbose=false synth1d.config

Where synth1d.config is 1st step configuration file that defines network topology:3 LSTM hidden layers of 400 cells, 20 gaussian mixtures as output layer, 10 mixturesfor character warping window layerSomewhere between training epoch 10-15 it will find optimal solution and will do"early stopping" w/o improvement for 20 epoch. "Early" here takes 3 days on IntelSandybridge CPU. Normally training can be stopped as long as loss starts rising upfor 2-3 consequent epochs.The best solution found is stored in synth1d@.best_loss.save file

Best loss error from step 1 is expected to be around -1080 nats and it can be furtherimproved (ca. 10%) by using weights regularisation. Loss error goes up and down during thetraining unlike in Step 1. Therefore one must be more patient to declare early stopping andwait for 20 epochs with loss worse then the best result so far. Rnnlib has implementationof MDL regulariser which is used in this step. The command line is as following:

rnnlib --mdl=true --mdlOptimiser=rmsprop from_step1.best_loss.save

Handwriting synthesis is done by rnnsynth binary using network parameters obtained bystep 2:

rnnsynth from_step2.best_loss.save

The character sequence is given to stdin and output is written to stdout. The output sequenceis the same as input where each data point has x,y offsets and end-of-stroke flag.

Rnnsynth output is the sequence of x,y offsets and end-of-stroke flags. To visualise it onecan use show_pen.m Octave script:

octave:>show_pen('/tmp/trace1')

Where /tmp/trace1 contains stdout from rnnsynth.

Configuration options are exlained inhttp://sourceforge.net/p/rnnl/wiki/Home/. Since thenthere are few things added:

• lstm1d as hiddenType layer type - optimised LSTM layer when input dimension is 1d
• rmsprop optimizer type
• mixtures=N where N is number of gaussians in the output layer
• charWindowSize=N where N is the number of gaussians in the character window layer
• skipConnections=true|false - whether to add skip connections; default is true

Please create github issues to discuss the problems