Jun 212012
 
kinect-xl

A recent project of mine involves research and development with an XBOX 360 Kinect Sensor. Being a python guy, I started searching for python bindings to some OSX-supported framework. When you just get started looking into this area it can be a little confusing. There are a number of layers to the software stack to enable one to accomplish anything meaningful. This is just a short and general blog post outlining the basics of what I have discovered thus far, to help anyone else that might also be getting started.

At the lowest level, you need a driver. Something that can talk to the USB device that is the Kinect sensor. When you purchase the XBOX Kinect for Windows version of the sensor, and you are going to be developing on windows, much of this whole stack is provided to you by way of the Kinect SDK. But for the open source folks with the standard XBOX 360 sensor, you need to piece together your own solution.

Two drivers that I have discovered thus far:

I had started OpenKinect (libfreenect) because it comes with a python wrapper included. There were a few dependencies (I will talk about specific build steps in just a moment), but once I got this installed I was able to fire up the included  glview app and see both depth and rgb data streaming in from my sensor. The role of these drivers is to provide simply the basic streams. That is, the depth, rgb, audio, and a few other sensor data streams. If your goal is to start tracking players, seeing skeletons, and registering gestures, the drivers are not enough. You would be required to make your own solution from this raw data at this phase in the game.

You would now want to look into middleware that can take the raw data and provide to you an API with higher level information. This would include finding users in the scene for you, tracking their body features, and giving you various events to watch for as the data streams.

Being that my goal was to have python bindings, I found my options to be much more limited than if I were going to be developing in C++. Wrappers have to exist for the framework you want. This is where my research really started ramping up. I spent a few days dealing wtih compiling issues, as well as having an actual bad power adapter that had to be exchanged. But all said and done, here is what I have settled on thus far…

  1. Driver: PrimeSense Sensor
  2. OpenNI Framework
  3. NITE middleware for OpenNI
  4. PyOpenNI python bindings

Install Details

Install homebrew (package manager)

http://mxcl.github.com/homebrew/

Install build tools

Install python2.7

Suggestion: virtualenv Environment

This is not a requirement. But I recommend using virtualenv to set up an environment that specifically uses python2.7 so that you don’t have to fight with mixed dependencies and versions.

Create a virtualenv called “kinect”

Install libusb (patched version)

There is a special patched version of the libusb library, in the form of a homebrew formula.

Now copy platform/osx/homebrew/libusb-freenect.rb -> /usr/local/Library/Formula/

Install SensorKinect drivers

Then uncompress Bin/SensorKinect093-Bin-MacOSX-v*tar.bz2

Install OpenNI framework
  1. Go here: http://www.openni.org/Downloads/OpenNIModules.aspx
  2. Download Unstable Binary for MacOSX
  3. sudo ./install.sh
Install NITE middleware (for OpenNI)
  1. Go here: http://www.openni.org/Downloads/OpenNIModules.aspx
  2. Download Unstable MIDDLEWARE of NITE for OSX
  3. sudo ./install.sh
Install PyOpenNI

Be aware that on OSX, PyOpenNI requires a framework build of python 2.7+ and that you must build it for x86_64 specifically. Also, I was having major problems with cmake properly finding the python includes location. I had to suggest a fix, so please see here for the necessary corrections. I have referenced a patched fork of the repository below.

copy the lib/openni.so module to the python2.7 site-packages

Examples

Once you have everything installed, you can try out the examples that are included both in the NITE source location that you downloaded and also in the PyOpenNI source location:

  1. NITE/Samples
  2. PyOpenNI/examples
I also tried out ofxKinect (github.com/ofTheo/ofxKinect) on the side, which is an addon for  OpenFrameworks. This is kind of a separate path than the OpenNI stack. I would say its more like an advanced offering of libfreenect. Using the included example, I recorded a 3D point cloud that is built on the fly from the RGB and depth data:

 

  2 Responses to “Getting started with XBOX 360 Kinect on OSX – opensource+python”

  1. My brother suggested I might like this website. He was entirely right.
    This publish actually made my day. You can not imagine simply
    how a lot time I had spent for this info! Thanks!

  2. Right here is the perfect webpage for everyone who wants to understand this topic.

    You understand a whole lot its almost hard to argue with you (not that I personally would want to…HaHa).

    You certainly put a fresh spin on a subject that has been written
    about for many years. Great stuff, just
    great!

 Leave a Reply

(required)

(required)


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">