DIFFERENT BITS
FIGURE 3. XOR feedforward
neural network architecture.
on how far off the output was from
what we wanted. Then we propagate
this error back to the hidden neurons
and adjust the weights between the
inputs and hidden neurons based on
how much each hidden connection
was responsible for the incorrect
answer.
Let’s walk through one example
from each layer, and then we will
see what this looks like in terms of
microcontroller C code.
Our first calculation is called delta
output (delta just means change in
math-speak):
delta output = output activation *
(1.0 – output activation) * (desired
output – output activation)
Next, we calculate delta weight
for each weight connected to the
output neuron:
delta weight = hidden activation
* delta output
Finally, we adjust the value of
16 SERVO 09.2007
the weight in question:
output weight = output weight +
(learning constant * delta weight)
We do this for each weight
connected to the output neuron. Then
we have to adjust the weights of the
connections between the input and
hidden neurons. This is a little bit
trickier because it is more difficult to
figure out how much each weight is to
blame for our error.
For each hidden neuron, we calculate an error sum based on the delta
output we computed earlier, and the
weight of that neuron’s connection to
the output neuron:
error sum = output weight *
delta output
Next, we compute a delta hidden
for each hidden neuron just like our
delta output above:
delta hidden = (hidden activation
* ( 1.0 – hidden activation))
* error sum
Finally, for each connection
feeding into the hidden neuron, we
calculate a delta weight and adjust that
connection’s weight value:
delta weight = input value
* delta hidden
hidden weight = hidden weight +
(learning constant * delta weight)
Microcontroller
Implementation
a. Circuit
That is really all there is to it.
Constant tweaking over a few
thousand iterations and you have
evolved yourself an XOR gate! The
CCS (Custom Computer Services;
www.ccsinfo.com) C implementation
is pretty straightforward and similar
to the perceptron implementation
from last time. The trickiest part is
keeping all the array indexes straight
and having a clear picture of the
network layout.
