Creative Kinect hacking

After a user defined the plane of interaction by selecting three points on the surface, the device was then able to pick up anything that crossed that plane. “We had built a similar system just a few weeks earlier using a LIDAR scanner, before the Kinect was released,” he reveals. “The Kinect performs almost as well, but the LIDAR scanner costs $5,000, while the Kinect is $120.”

It isn’t just the basic hardware that’s affordable, either. Most Kinect hackers make use of the free, open-source creative programming toolkit openFrameworks, for which various library scripts are available to suit every conceivable need. Another popular library of creative code is Cinder, which operates in a similar, community-developed way.

Besides the Kinect itself, other equipment required varies: Memo Akten has dabbled with high-speed industrial cameras, infrared illuminators and other pieces of kit, and is currently working on a top-secret installation using two motion control rigs. “For an upcoming project, I hope to get an army of quadcopters,” he smiles. “So I’m expanding the arsenal.”

Akten points out that early adopters were likely to be the most technical – those who wrote the open-source drivers – followed by the second phase of hackers working with C++. Then designers with less coding experience joined the party. “More wrappers got written, Kinect support was added to vvvv, Max/MSP, Processing and so on, and suddenly you didn’t need to be that technical to get involved,” he explains. “Now there are loads of standalone apps and wrappers that enable you to plug Kinect into almost any software. All you really need is an idea.”

It’s an empowering thought, and certainly one of the main reasons behind why there are so many diverse and interesting creative applications of the technology out there. “The issue isn’t so much the ‘how’ anymore, it’s the ‘what’,” adds Orme. “If a strong project’s gnawing away at you, as a designer you find a means to accomplish it.”

While it helps to have at least some background or prior knowledge in programming, for Soltis a willingness to learn is much more crucial: “You might need to read and adapt example code, find and use a software library, interface with a device, or learn to program on a whole new platform,” he reels off. “Existing skills might help you learn more easily, but you’ll still need to grapple with new concepts and tools.”

Of course, for users interacting with a Kinect-based interface, the experience can be unfamiliar too – even if they already use it for gaming. “Don’t make assumptions about how people will use your application,” is Nick Hardeman’s advice to designers. “They will most likely act in ways you never anticipated. The most informed decisions come from viewing user interaction.

Figure out your goal, make a rough prototype to test certain aspects and watch people use it without instruction.”

If there’s one thing that all Kinect hackers agree on, it’s that they’ve still barely scratched the surface of the device’s creative potential. “The technology has only been available to the average person for a short period of time,” Hardeman points out. “The continued interest by the community in the Kinect will lead to a higher fidelity, refined product that is more robust. A device that gives more information about the environment – possibly a temperature map, and a higher resolution depth and RGB image, for instance, would open up plenty more opportunities.”

“We’re already hearing murmurs about Microsoft integrating some form of Kinect into its laptops for use in its OS, which presents a whole set of rich scenarios,” adds Orme, but he’s quick to point out that – as with all creative tools – it’s what you do with it that counts. “Being able to crack the lid off something doesn’t necessarily expose a world of opportunities,” he concludes. “The Kinect is a peripheral controller at the end of the day, and knowing how to get coordinates off a mouse controller doesn’t mean you’ve got a project.”