Making Things See, written by Greg Borenstein.
So, I took a chance and ordered the book. I also
purchased a Xbox 360 gaming system with the Kinect, and
the adapter cable that powers the sensor and allows it to
connect to a PC with a standard USB plug. The book uses
an open source collection of drivers written by Primesense
for the Kinect called OpenNI, and a skeletal tracking
software called NITE. The book also introduces the reader
to the Processing programing language for the PC and
using it to communicate with the Arduino microcontroller
through the USB serial port.
Since I had already learned how to do some of the
basic things with an Arduino — like reading switches, pots,
and controlling LEDs and transistors — I had some
understanding of the Arduino version of the Processing
language. I still had a lot to learn about the PC version and
serial communications, though.
Borenstein’s book has proven to be an excellent
resource for me. It is filled with lots of good software, and
the complex functions of the language and the sensor are
explained in simple examples at first, then each chapter and
project builds on the previous one. It took about a day to
install the OpenNI drivers, the NITE software, and the
Processing language on the PC, and to finally get all the
libraries in the correct folders and the drivers talking to each
other. Once everything was working, the fun really began.
The Kinect sensor is very cool. It uses two cameras: an
RGB camera that can see what humans would think of as
normal video, plus it has an infrared projector that paints
the scene with thousands of tiny dots that the second
infrared camera can see. The infrared camera is basically
looking for the brightness of the projected dots as the
infrared light bounces off of objects. The brighter the dot,
the closer the object; the dimmer the dot, the farther away
the object is from the camera. This is all done with some
very cool and complex math and triangulation that is
explained in the book.
One of the early examples in the book shows how to
measure distance using the depth sensor function. In fact,
after running that example and doing a bit of my own
coding, I was able to read the distance of an obstacle, and
cause a servo (controlled by an Arduino) to turn right or left
with a wave of my hand. That was an exciting moment
since I realized that with just a few lines of code, I could
allow a mobile robot to steer around obstructions using
simple infrared vision and have it check for a minimum
distance in inches or centimeters.
I did several more of the examples, and eventually
could control LEDs and lights waving my hands by
measuring the virtual space between me and the camera.
I wondered what else this thing could do.
As I moved along in the book, Chapter 4 introduces
you to skeletal tracking, and how the software libraries can
find the position of a body and most all of the joints. Once
you enter the code and run the examples, you see an
overlay of the skeletal lines on your body on the screen, and
you can see how the software calculates your movements
in real time.
In Chapter 6, there is a program that combines the
user calibration function that learns your body position,
skeletal tracking, and serial communications to an Arduino.
Plus, there’s an easy build for a simple robot arm using two
servos, two Popsicle sticks, and some hot glue. Basically,
this creates a shoulder and an elbow when assembled.
Once I built this simple project and got it working, I
wanted more! I found a stuffed animal at a flea market and
brought it home for a bit of surgery. I mounted two servos
on a wood frame to act as the shoulders and gently forced
the assembly into the body. With just a bit of modification
to the example program on the PC (plus the Arduino code),
I had this little test bot mimicking my human arm
When I raised an arm, “he” would raise an arm. I
named him Minky the Monkey and we actually went to the
2012 Maker Fair together and had a lot of fun doing
demos. Unfortunately after two days of demos, both of
Minky's servos died from the restrictions of his body fur and
a few children grabbing his arms while the servos were
seeking positions. (Kids are the ultimate durability testers.)
I did learn a few things about the need to calibrate the
software for each user, which is a bit of a pain since people
54 SERVO 08.2014
Cal’s shoulder and hip mounts. Cal’s hips use four 2-1/2” long
#8 machine screws to narrow his hips in respect to his
shoulders. The original toy skeleton wasn't really meant to
move in the way that Cal does, so I had to make some width
adjustments. Also, the numbers on the servos designate
which pin on the Arduino controls them. This helps a lot
when I'm writing his software.