#natural user interfaces

[corntoole]

**Hands-on video with the Leap motion controller** I can't wait to try this out. I'm seriously thinking about joining the developer program. via [engadget](http://www.engadget.com/2012/05/25/leap-motion-gesture-control-technology-hands-on/)

[ooeygui-blog]

Haptic Feedback and Natural User Interfaces

"Kinect-style," or Natural User Interface (NUI) style interfaces are typified by the user's lack of "connectedness" to the computer or system he is interacting with.  The biggest advantage of this approach is that it allows the user to be completely uninhibited in his or her movements.  Un-burdened by connected sensors and devices, he can utilize all six degrees of freedom afforded him by three dimensional space.

Haptic feedback-based interfaces are a contrasting approach, relying heavily on a user's sense of touch (haptics) to communicate with the user and allow him to interact with the system.  The basis for this approach lies in the specialized group of receptors in our muscles, tendons, and joints known as proprioceptors.  These carry signals to the brain that contain data about an objects size, shape, hardness, texture, and other touch-sensory information we use when interacting with our surroundings.  An example of a device using such feedback is the PHANTOM (R) interface from SensAble Technologies. 

This device simulates touching with a sylus connected to a lamp-like arm that gives force feedback by exerting pressure on the stylus.  It allows the user to feel texture, temperature and weight.  The disadvantage of most haptic feedback interfaces is that movement is restricted, in this case to three degrees of freedom.  By being hampered down by contact with physical device, the user does not have the freedom of movement afforded by Natural User Interfaces like the Kinect.

This leads us to an obvious question.  Can these two approaches be fused?  Can the tactile feedback afforded by haptic interfaces be experienced alongside the unhampered movement and flexibility offered by a NUI environment? It takes some creativity and imagination to get past the obvious barrier that one approach is grounded in tactile contact with physical devices and the other is defined by its lack thereof.  But things like the "touchable hologram" in this video suggest that it is in fact possible:

No physical contact with a device is necessary, but interaction with the hologram is accompanied by tactile feedback provided by moving air.  The limitations of this are obvious, however, as the tactile feedback afforded by streams of air is limited and does not communicate such information as size (as when you close your hand around an object), texture, or hardness of an object.  

My vote for the best balance between freedom of movement and tactile feedback goes to those based on magnetic levitation.  These maintain the user's ability to move in six degrees of freedom while still providing tactile information about object collisions by preventing the controller from moving through virtual obstacles.  These can also be designed to adjust feedback based on the texture of a surface, and temperature changes would be easy to implement as well.  One of the more impressive designs, from the Microdynamic Systems Laboratory at CMU has a floating handle that allows for movement in all six degrees of freedom, and motion can be restricted (for virtual collisions) in all three dimensions.  Check out the video below:

Still, there is still much improvement to be made to haptic interfaces.  Commercial haptic interfaces in particular are still very limited, constrained in most cases to a single style of vibration, as with force feedback used in some phones and gaming systems.  This severely hinders the ability of the feedback to simulate feedback that would be encountered in the real world.  The Samsung Anycall Haptic cell phone is a step in the right direction, with 22 different force feedback vibrations, ranging from a button push to the clicks on an old-style volume knob.  Cost is another area that could be significantly improved, since most high quality haptic feedback systems are far too expensive even for many large industrial operations businesses, much less the general populace.

Current commercial products have merely scratched the surface of haptic feedback in their interfaces, but exciting research efforts suggest the future is bright for developments in this area.

[sectorpublic-blog]

Microsoft had a big hit with its Kinect extension for Xbox, with a big community of people 'hacking' it for new capabilities. But there's more "natural user interfaces" where that came from. Fast Company reports that one project Microsoft Research is working on is "Skinput," a system allows you to control devices by hitting specific points on your arm. Not a device on your arm. Just your arm itself.