Servo - January 2008

would simply subtract a small amount from the learning rate after each iteration through the data set. In other words, if we start with a learning rate of .05 and we have 48 data points which we iterate through 30 times, we might subtract .001 from the learning rate after each iteration, leaving us with a final learning rate of .02. We don’t subtract after each data point, only after each run-through the entire set.

Once we have a learning rate specified, we determine how much to change the prototype vector by multiplying the learning rate by the difference between both components of the two vectors. For example, if the learning rate is .05, our data point is (122, 180) and our prototype is (84, 203), we would adjust it as follows:

Delta = learning rate * (point 1 x – point 2 x)

Delta = .05 * (122 – 84)

Delta = 1.9

New prototype x value = 84 + 1.9 = 85.9

Delta = .05 * (180 - 203)

Delta = -1.15

New prototype y value = 203 -1.15 = 201.85

Then we round the floating point value off to the nearest integer to keep our memory consumption limited. The new values for our prototypes would be (86, 202). As you can see, the prototype vector has moved a little bit closer to the data point.

In code, it looks like this:

– a function for rounding

long round(float x){
long x2;
       if (x>=0){
               x2 = (long)(x+0.5);

}
else {

       }
 return x2;
}

x2 = (long)(x-0.5);

– updating both coordinates of the winning prototype

temp1 = (long)x - (long)prototypes[winner][0];
temp2 = c*temp1;
prototypes[winner][0] = round( prototypes[winner][0]+
temp2);

temp1 = (long)y - (long)prototypes[winner][1];
temp2 = c*temp1;
prototypes[winner][1] = round( prototypes[winner][1]+
temp2);

We repeat this update process for each data point a set

DIFFERENT BITS

FIGURE 4. Graph of the Self-Organizing Map sample values and the final prototype values.

number of iterations. In my example, 10 iterations was more than enough. Now the prototype vectors have moved to positions representing clusters of data.

If we feed a new input vector into the program, it will recognize it as one of the four categories. I set my circuit up to output to one of four LEDs, depending on which category it was sensing. At this point, you can decide if you want to halt learning, or allow the program to continue the learning process in a more restricted fashion. In my code, I chose to halt learning, but the code to continue learning with a limited learning rate of .005 is included in the comments.

You might be wondering how it is possible to map specific physical outputs to outputs of the Self-Organizing

Map when we don’t know ahead of time how it is going to shape up. This question is resolved by a different type of neural network called an Oustar which combines a competitive layer like our Self-Organizing Map with a supervised output layer like the feedforward neural networks we looked at in the September issue. By adding a teacher, we are able to decide which clusters of data will be represented by which output. This is useful in situations where you know what relationships you want the network to learn in advance.

One of the things I find fascinating about the Self-Organizing Map is the idea that the program can have a kind of emergent knowledge all its own; something