A Study of Self- Organizing Maps(SOM) Neural Network Using Matlab

Kohonen's Self-Organizing Map ( SOM ) is one of the most popular artificial neural network algorithms. Bing unsupervised neural network , Self Organizing Maps( SOM ) have applications in different fields such as speech recognition, image processing and so on . This project includes a study of Self Organizing Maps neural network  using  MATLAB The structure, characteristics, implementation, applications and testing of this neural network for styles of one dimension and two dimensions have been considered It includes finding out the functions used for topology, and then finding out the distance function for this network throughout illustrative examples. Neural network implementation consists of three stages: Initialization (creation), Training, and Simulating. Explanation of the neighborhood concept is done. MATLAB software is used to perform how creating, training and simulating of a Self Organizing Map. Creation process consists of choosing a network parameters, plotting the results, then illustrating and identifying all functions used to create a Self Organizing Map. Training consists of weight initialization and weight vector creation. Simulating means testing the neural network using the initialization parameters and training vector created in the past two stages. The Study includes also four tests: The first test is used manual calculation procedure of the network mathematically.  The other three tests are procedures for different applications using MATLAB language. It becomes evident from the graphs of the results that it’s essential to have the weight vectors for the coordination field greater than the density of input vectors in the case of employing the network to seclude the training styles in output cells. Keywords: Neural Network, Simulating, Self- Organizing Maps, Competitive layer, training DOI: 10.7176/RHSS/10-6-01 Publication date: March 31 st 2020


Introduction
1.1-Motivation for the work: neural nets are a powerful tool for modeling problems for which the explicit from of the relationships among certain variables is not known The need of knowledge for the specific feature and applications of the different types of neural networks. Self organized map is one of the most important and widely used neural network. Self organized map is a simple equivalent way for neural system of human to save information in a certain style 1.2-Literature survey: 1. Timo Honkela, and Ari M. Vepsäläinen (1991): (Interpreting imprecise expressions: experiments with Kohonen's Self-Organizing Maps and associative memory.) Kohonen's Self-Organizing Map (SOM) is used to model the impreciseness of natural language interpretation. The author designed the study and conducted the practical experiments related to the SOM. Ari M. Vepsäläinen provided his expertise of associative memories 2. Timo Honkela (1993): (Neural nets that discuss: a general model of communication based on selforganizing maps) The basic idea is to provide a model of adaptation that leads to intersubjectivity in interpretation of natural language expressions and to coherence in communication.
3. Timo Honkela, Ville Pulkki, and Teuvo Kohonen (1995): (Contextual relations of words in Grimm tales analyzed by self-organizing map.) deals with the analysis of contextual relations of words. The work was based on the principle of selforganizing semantic maps presented by Ritter and Kohonen in 1989 in Biological Cybernetics. In their article, artificially generated, syntactically correct and meaningful short phrases were used as the input to the SOM. the method was refined and the input consisted of segments of text from a natural corpus. 4. Timo Honkela (1997): (Self-Organizing Maps of words for natural language processing applications. presents how the word category maps can be used in natural language processing (NLP) applications. The main emphasis is on the WEBSOM method, and the concrete results presented in the publication are based on the close collaboration of the WEBSOM team. also discusses the nature of the found clusters and the general principles of using the SOM, e.g., in graded classification, following the argumentations set by Ritter and Kohonen in their article in the journal Biological Cybernetics in 1989. 5. Timo Honkela (1997)( Learning to understand -general aspects of using Self-Organizing Maps in natural language processing.) the introduction outlines some limits of the traditional symbolic approaches such as relying on fixed categorizations, problems related to handling language change, and, moreover, subjectivity and context-sensitivity of interpretation. One chapter summarizes the basic principles of using the SOM in natural language interpretation. Finally, the last chapter contains some epistemological considerations. 6. Samuel Kaski, Timo Honkela, Krista Lagus, and Teuvo Kohonen (1996): (Creating an order in digital libraries with self-organizing maps. ) Studies on the WEBSOM full-text analysis method are described presents a fully unsupervised method with improvements in the document encoding. The original idea of using a two-stage SOM architecture was due to the author. Samuel Kaski was the main originator of the idea of using table lookups for the efficient encoding of the documents and the subsequent convolution. The original document maps were limited to some 10 000 documents at maximum. To facilitate considerably larger maps for up to one million documents, Professor Kohonen later suggested some new methods for shortcut computation. Many ideas and details, as well as implementations and experiments were developed jointly in a team consisting of Samuel Kaski, .
1.3 -Aim of the project Study the features and applications of the Self organized map neural network Using Matlab software 1.4 -Outline of the project This Study contains five chapters: After this Introduction ,chapter two involves the theoretical background of Neural Networks and Self Organized Map(som) Based Implementation and application are found in chapter three and chapter four respectively . Chapter five includes the conclusion and future works Neural Network & Self Organizing Maps (SOM)

Artificial Neural Networks
An artificial neural network is an information processing system that has certain performance characteristics in common with biological neural networks. Artificial neural networks have been developed as generalizations of mathematical models of Human cognition or neural biology. A neural network is characteristics by It's pattern of connections between the neurons (called architecture). Its method of determining the weights on the connections (call its training. Or learning algorithm). Its activation function.
A neural network consists of a large number of simple processing elements called neurons, units, cells, or nodes. Each neuron is connected to other neurons by means of directed communication links, each with an associated weight. The weights represent information being used by the net to solve a problem.
Each neuron has an internal state. Called its activation or activity level. Which is a function of the inputs it has received. Typically, a neuron sends its activation as a signal to several other neurons. It is important to note that a neuron can send only one signal at a time. Although that signal is broadcast to several other neurons. There is a close analogy between the structure of a biological neuron (i.e. a brain or nerve cell) and the processing element (or artificial neuron).
A biological Neuron has three types of components that are of particular interest in understanding an artificial neuron: its dendrites, soma and axon. The many dendrites receive signals from other neurons. The signals are electric impulses that are transmitted across synaptic gap by means of a chemical process.
The action of chemical transmitter modifies the incoming signal (typically by scaling the frequency of the signals that are received) in a manner similar to the action of the weights in an artificial neural network.
Some features of artificial neural networks that are suggested by biological neurons are:-Information processing is local (Although other means of transmission, such as the action of hormones, may suggest means of overall process control). Memory is distributed: Long term memory resides in the neuron's synapses or weights. Short-term memory corresponds to the signals sent by the neurons. A Synapses' strength may be modified by experience. Neurotransmitter's for synapses may be excitatory or inhibitory. It's convenient to visualize neurons as arranged in layers. Typically neurons in the same layer behave in the same manner. Key factors in determining the behavior of a neuron are its activation function and the pattern of weighted connections over which it sends and receives signals.
The arrangement of neurons into layers and the connection patterns within and between layers the net architecture.
Neural nets are often classified as single layer (with no hidden layer) or multi layer (with hidden layer) in determining the number of layers. The input units are not counted as a layer, because they perform no computation.  Training is accomplished by presenting a sequence of training vectors, or patterns. Each with an associated target output vector. The weights are then adjusted according to a learning algorithm. This process is known as supervised training.
In this neural networks the output is bivalent element say, either 1 (if the input vector belongs to the category) or -1(if it doesn't belong) Unsupervised Training Self-organizing neural nets group similar input vectors together without the use training data to specify what a typical member of each group looks like or to which group each factor belongs. A sequence of input vectors is provided. But no target vectors are specified. The net modifies the weights so that the most similar input vectors are assigned to the same output (Or cluster unit) the neural net will produce an exemplar (representative) for each cluster formed. Self-organizing nets.

Activation functions
The basic operation of artificial neurons involves summing its weighted input signal and applying an output. 1--Identity function σ=steepness parameter

Fig2-5: Competitive Layer
2.5 Topology preserving maps in the brain The brain is capable of creating maps in its neural structures for the characteristics received as neural signals from perception systems represented by one or more dimensions. Every entity in our surroundings has many characteristics that brain can percept like, color, location, texture.
How can the brain process the multi dimensional signals? And how this signal can be represented by multi dimension in the biological neural structures? 2.6 Mapping of the visual field on the Cortex The researchers have proved that Human being's brain cortex is distributed to functional groups which consist of billions of neurons connected together by billions of synapses.
These functional groups may be arranged in organized order, some of the most important of them are: Somatosensory Cortex, Auditory Cortex, Visual Cortex, Motor Cortex, for example the visual cortex is divided to several cell groups; each of them has its certain perceptional function for the visual signal, and it's located in the Visual Cortex at the rear part of the brain.
The visual data is diagramed in the Visual Cortex (Brain Cortex) as two dimensional small areas as in the figure (2-1). From the figure (2-1) we note the followings:-The adjacent area in the visual field (retina) is processed by adjacent areas in the brain cortex. The surface of visual cortex in the brain is specialized to process the signals coming from center of visual field, and it being processed in higher resolution.
Then the cortex is discriminated by its harmony and unison in saving information This way of representing the information carries two characteristics. In every phase of representation, the upcoming information will be represented in a proper context or in its proper spot.
The cells related to information closed to each others are saved in vicinity to each others in away which is able to interact with each others through synopses.  Kunihiko Fukushima and his colleagues at NHK Laboratories in Tokyo have developed a series of specialized neural nets for character recognition. One example of such a net called a neogognitron . D .Boltzmann Machine A number of researchers have been involved in the development of nondeterministic neural nets, that is, nets in which weights or activations are changed on the bases of a probability density function.

Self Organizing Maps Neural Networks
The self organizing map developed by Kohonen. Which groups the input data into clusters. A common use for unsupervised learning adaptive resonance theory nets. In this form of learning the units that update their weights do so by forming a new weight vector that is linear combination of the old weight vector and the current input vector.
Typically, the unit whose weight vector was closest to the input vector is allowed to learn. The weight update for output (or cluster) unit (j) is given is Where (p) is the input vector, (Wj) is the weight vector for unit j (which also the jth column of the weight matrix) and (α) the learning rate, decreases as learning proceeds.
Self organizing network is a simple equivalent way for neural system of human to save information in a certain style.
The basic goal of (SOM) is transferring the input factors which has a random dimensional or multiple dimensional to one dimensional or two dimensional. Thus the neurons in competitive layers recognize like one dimensional or two dimensional Self-organizing feature maps (SOFM) learn to classify input vectors according to how they are grouped in the input space. They differ from competitive layers in that neighboring neurons in the self-organizing map learn to recognize neighboring sections of the input space. Thus, self-organizing maps learn both the distribution (as do competitive layers) and topology of the input vectors they are trained on.
The neurons in the layer of an SOFM are arranged originally in physical positions according to a topology function. The functions gridtop, hextop or randtop can arrange the neurons in a grid, hexagonal, or random topology. Distances between neurons are calculated from their positions with a distance function. There are four distance functions, dist, boxdist, linkdist and mandist. Link distance is the most common. These topology and distance functions are described in detail later in this section. Input layer and output layer (also called competitive layer). Every neuron in the input layer connected to each neuron in output layer such (Feed forward connection) it means that the input data spread out through network from input layers toward output layers.
The input layers don't perform any process, only receive input vectors from output world and send out to processor elements in the output layers for this network.  Step1: While stopping condition is false do steps 2-8 Step 2: For each input vector P do step 3-5 Step 3: For each j compute for input Step 4: Find index j such that D(j) is a minimum Step 5: For all units j within a specified neighborhood of j and for all i Wij(new) = wij(old)+ α (Pi-wij(old))----------9 Step 6: Update learning rate α(t+1)= α (t)*0.5------------10 where t is the discrete-time index of the variables Step 7: Reduce radius of topological neighborhood at specified times Step 8: Test stopping condition which Represents by the no of steps or nets stability and this done by Comparing the average distance(AD) of the net and with the value of (Tolerance distance(TD) during initializing if the average distance less or equal to tolerance distance If AD <=TD then nets stability else next step It means the training completed but if the Average distance is more than Tolerance distance return to the next step and the average distance founded by: 2.12 Determining the size of the network To determine the size of the network we should know that the increment in the number of the neurons causes decrement in the error rate resulting in the net in addition it will get better classification for the inputs.

 
In the other hand a large number of neurons consuming more time in training process.

Disadvantage of SOM
The disadvantage of SOM is the net performs too much processing because at the first steps of training much of neurons adjust their weights in a correlated scheme. For each step of training (Input one set of train) the net calculates the distances of this set and all neurons' weight in the competitive layers.
2.14 Advantage of SOM One the main characteristics of SOM is the capability of connecting neurons to a group of weights in a way that achieves geometric order (Topology of net) and it can be assured for a net organization by calculating the Euclidian distances among the neurons.
Then the measured distances must comply with the original topology of this net. The net called organized if the Euclidian distances among neurons are proportional to the geometric distances among neurons.

Matlab Based Implementation of SOM
The processing in som includes four main elements Initialization: it's the process of giving random small values to the weights in the Network.
Research on Humanities and Social Sciences www.iiste.org ISSN 2224-5766 (Paper) ISSN 2225-0484 (Online) Vol. 10, No.6, 2020 11 Competition: After every input into the network the cells in the output layer calculate the activation level according to a certain function depending on competition basis, the winner cell is the one which has the higher activation level.
Cooperation: The winner cell determines the space which contains the neighbor cells in order to amend the weights of these cells.
Adaptive: increase the level of activation for winning cell resulting from competition function through modifying this cell weight to improve it's response for the input vector itself.  The various training vectors are seen as fuzzy gray spots around the perimeter of this figure(3-1). The initialization for newsom is midpoint. Thus, the initial network neurons are all concentrated at the black spot at (1, 0.5). Tuning-phase neighborhood distance Thus, feature maps, while learning to categorize their input, also learn both the topology and distribution of their input.

Neural
We can train the network for 1000 epochs with net.trainParam.epochs = 1000; net = train(net,P); This training produces the following plot.

.3 Simulating
When simulating a network, the negative distances between each neuron's weight vector and the input vector are calculated (negdist) to get the weighted inputs. The weighted inputs are also the net inputs (netsum). The net inputs compete (compet) so that only the neuron with the most positive net input will output a 1 p = [1;0]; a = sim(net,p) a = (5,1) 1 We can see that the neurons have started to move toward the various training groups. Additional training is required to get the neurons closer to the various groups.
As noted previously, self-organizing maps differ from conventional competitive learning in terms of which neurons get their weights updated. Instead of updating only the winner, feature maps update the weights of the winner and its neighbors.
The result is that neighboring neurons tend to have similar weight vectors and to be responsive to similar input vectors Applications and Results

Mathematical Test
Kohonen self-organizing map (SOM) to cluster four vectors Let the vectors to be clustered be Research on Humanities and Social Sciences www.iiste.org ISSN 2224-5766 (Paper) ISSN 2225-0484 (Online) Vol.10, No.6, 2020

Fig 4-3: Plot of 1000Two-element vectors
Research on Humanities and Social Sciences www.iiste.org ISSN 2224-5766 (Paper) ISSN 2225-0484 (Online) Vol. 10, No.6, 2020 Now we train the map on the 1000 vectors for 1 epoch and re-plot the network weights. After training, note that the layer of neurons has begun to self-organize so that each neuron now classifies a different region of the input space, and adjacent (connected) neurons respond to adjacent regions.
We can now use SIM to classify vectors by giving them to the network and seeing which neuron responds. The neuron indicated by "a" responded with a "1", so p belongs to that class. p = [0.5; 0.3]; a = sim(net,p) a = (24, 1) 1

A geometric test
The cluster units in a Khonen self-organizing map can viewed as having a position (given by their weight vector) for input patterns with two components, this position is easy to represent graphically. The topological relationships between cluster units in Kohonen Self-organizing maps are often indicated by drawing lines connecting the units. In this test we assume a linear structure. The initial weights are chosen randomly. With each component having a value between -1 and 1. There are 50 cluster units. The 100 input vectors are chosen randomly from within a circle of radius 0.5 (centered at the origin). The initial learning rate is 0.5 it is reduced linearly to 0.01 over 100 epochs. Throughout training the winning unit and its nearest neighbor unit on either side (units J, J+1, and J-1) are allowed to learn. Fig.4-6 shows the training pattern, Figs 4-7, 4-8, 4-9, 4-10, 4-11 show the cluster units initially and after 10, 20, 30 and 100 epochs respectively. Not only have the cluster units moved to represent the training inputs (i.e. all of the weight vectors for the cluster units now fall within the unit circle). But the curve connecting the cluster units, has smoothed out somewhat as training progresses. An even smoother curve can be obtained by starting with a larger radius and gradually reducing it to 0. This would involve using more training epochs.

Conclusion
A self-organizing map learns to categorize input vectors. It also learns the distribution of input vectors Feature maps allocate more neurons to recognize parts of the input space where many input vectors occur and allocate fewer neurons to parts of the input space where few input vectors occur. Self-organizing maps also learn the topology of their input vectors. Neurons next to each other in the network learn to respond to similar vectors. The layer of neurons can be imagined to be a rubber net that is stretched over the regions in the input space where input vectors occur. Self-organizing maps allow neurons that are neighbors to the winning neuron to output values. Thus the transition of output vectors is much smoother than that obtained with competitive layers, where only one neuron has an output at a time.
5.2 Suggestions for future works 1 .The segmentation of an image can be carried out by different techniques that are based mostly on the discontinuity and similarity of the grey levels of an image. Neural networks try to simulate a structure similar to the one that is believed the human brain has Two-dimensional layers of cellular modules that are densely interconnected between them. Modeling most neural networks in the brain, especially in the cortex. .The image has been transformed into a matrix, and segmented if needed by grey level selection; the algorithm for SOM will have as input region. 2. The WEBSOM method organizes a document collection on a map display that provides an overview facilitates interactive browsing. By virtue of the Self-Organizing Map algorithm, documents can be mapped onto a twodimensional grid so that related documents appear close to each other. If the word forms are first organized into categories on a word category map, an encoding of the documents can be achieved that explicitly expresses the similarity of the word meanings. The encoded documents may then be organized by the SOM to produce a document map. The visualized document map provides a general view of the document collection. In the WEBSOM project insightful methods have been developed for information retrieval based on the selforganizing map in the WEBSOM similar documents become mapped close to each other on the map, like the books on the shelves of a well-organized library the self-organized document map offers a general idea of the underlying document a space. The user may view any area of the map in detail by simple clicking the map image with the mouse The WEBSOM browsing interface is implemented as a set of HTML documents that can be viewed using a graphical WWW browser.