- Notifications
You must be signed in to change notification settings - Fork14
Recursive Self-Organizing Map/Neural Gas.
License
stephantul/somber
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
somber (Somber Organizes Maps By Enabling Recurrence) is a collection of numpy/python implementations of various kinds ofSelf-Organizing Maps (SOMS), with a focus on SOMs for sequence data.
To the best of my knowledge, the sequential SOM algorithms implemented in this package haven't been open-sourced yet. If you do find examples, please let me know, so I can compare and link to them.
The package currently contains implementations of:
- Regular Som (SOM) (Kohonen, various publications)
- Recursive Som (RecSOM) (Voegtlin, 2002)
- Neural Gas (NG) (Martinetz & Schulten, 1991)
- Recursive Neural Gas (Voegtlin, 2002)
- Parameterless Som (Berglund & Sitte, 2007)
Because these various sequential SOMs rely on internal dynamics for convergence, i.e. they do not fixate on some external label like a regular Recurrent Neural Network, processing in a sequential SOM is currently strictly online. This means that every example is processed separately, and weight updates happen after every example. Research into the development of batching and/or multi-threading is currently underway.
If you need a fast regular SOM, check outSOMPY, which is a direct port of the MATLAB Som toolbox.
Care has been taken to make SOMBER easy to use, and function like a drop-in replacement for sklearn-like systems.The non-recurrent SOMs take as input[M * N]
arrays, where M is the number of samples and N is the number of features.The recurrent SOMs take as input[M * S * N]
arrays, where M is the number of sequences, S is the number of items per sequence, and N is the number of features.
Color clustering is a kind ofHello, World
for Soms, because it nicely demonstrates how SOMs create a continuous mapping.The color dataset comes from this niceblog
importnumpyasnpfromsomberimportSomX=np.array([[0.,0.,0.], [0.,0.,1.], [0.,0.,0.5], [0.125,0.529,1.0], [0.33,0.4,0.67], [0.6,0.5,1.0], [0.,1.,0.], [1.,0.,0.], [0.,1.,1.], [1.,0.,1.], [1.,1.,0.], [1.,1.,1.], [.33,.33,.33], [.5,.5,.5], [.66,.66,.66]])color_names= ['black','blue','darkblue','skyblue','greyblue','lilac','green','red','cyan','violet','yellow','white','darkgrey','mediumgrey','lightgrey']# initializes=Som((10,10),learning_rate=0.3)# train# 10 updates with 10 epochs = 100 updates to the parameters.s.fit(X,num_epochs=10,updates_epoch=10)# predict: get the index of each best matching unit.predictions=s.predict(X)# quantization error: how well do the best matching units fit?quantization_error=s.quantization_error(X)# inversion: associate each node with the exemplar that fits best.inverted=s.invert_projection(X,color_names)# Mapping: get weights, mapped to the grid points of the SOMmapped=s.map_weights()importmatplotlib.pyplotaspltplt.imshow(mapped)
In this example, we will show that the RecursiveSOM is able to memorize short sequences which are generated by a markov chain.We will also demonstrate that the RecursiveSOM can generate sequences which are consistent with the sequences on which it has been trained.
importnumpyasnpfromsomberimportRecursiveSomfromstringimportascii_lowercase# Dumb sequence generator.defseq_gen(num_to_gen,probas):symbols=ascii_lowercase[:probas.shape[0]]identities=np.eye(probas.shape[0])seq= []ids= []r=0choices=np.arange(probas.shape[0])forxinrange(num_to_gen):r=np.random.choice(choices,p=probas[r])ids.append(symbols[r])seq.append(identities[r])returnnp.array(seq),ids# Transfer probabilities.# after an A, we have a 50% chance of B or C# after B, we have a 100% chance of A# after C, we have a 50% chance of B or C# therefore, we will never expect sequential A or B, but we do expect# sequential C.probas=np.array(((0.0,0.5,0.5), (1.0,0.0,0.0), (0.0,0.5,0.5)))X,ids=seq_gen(10000,probas)# initialize# alpha = contribution of non-recurrent part to the activation.# beta = contribution of recurrent part to activation.# higher alpha to beta ratios=RecursiveSom((10,10),learning_rate=0.3,alpha=1.2,beta=.9)# train# show a progressbar.s.fit(X,num_epochs=100,updates_epoch=10,show_progressbar=True)# predict: get the index of each best matching unit.predictions=s.predict(X)# quantization error: how well do the best matching units fit?quantization_error=s.quantization_error(X)# inversion: associate each node with the exemplar that fits best.inverted=s.invert_projection(X,ids)# find which sequences are mapped to which neuron.receptive_field=s.receptive_field(X,ids)# generate some data by starting from some position.# the position can be anything, but must have a dimensionality# equal to the number of weights.starting_pos=np.ones(s.num_neurons)generated_indices=s.generate(50,starting_pos)# turn the generated indices into a sequence of symbols.generated_seq=inverted[generated_indices]
See issues for TODOs/enhancements. If you use SOMBER, feel free to send me suggestions!
- Stéphan Tulkens
MIT
About
Recursive Self-Organizing Map/Neural Gas.
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
Packages0
Uh oh!
There was an error while loading.Please reload this page.