#gestureui

[timestechnow]

Top 7 Trends Driving the Automotive Human Machine Interface Market in 2025

Automotive Human Machine Interface (HMI) Market is rapidly embracing multi-modal interfaces, which combine touch, voice.

[voiceandgesture]

Still pushing pixels by hand? You shouldn't be. Imagine combining this technology (demonstrated in 1979!) with the subvocal speech recognition tools developed by NASA. Then imagine repetitive motion injuries disappearing. “Put-that-there”: Voice and gesture at the graphics interface by Richard A. Bolt, MIT Media Lab (PDF)

[voiceandgesture]

More than touch: understanding how people use skin as an input surface for mobile computing Abstract: This paper contributes results from an empirical study of on-skin input, an emerging technique for controlling mobile devices. Skin is fundamentally different from off-body touch surfaces, opening up a new and largely unexplored interaction space. We investigate characteristics of the various skin-specific input modalities, analyze what kinds of gestures are performed on skin, and study what are preferred input locations. Our main findings show that (1) users intuitively leverage the properties of skin for a wide range of more expressive commands than on conventional touch surfaces; (2) established multi-touch gestures can be transferred to on-skin input; (3) physically uncomfortable modalities are deliberately used for irreversible commands and expressing negative emotions; and (4) the forearm and the hand are the most preferred locations on the upper limb for on-skin input. We detail on users' mental models and contribute a first consolidated set of on-skin gestures. Our findings provide guidance for developers of future sensors as well as for designers of future applications of on-skin input.

[voiceandgesture]

Consumed endurance: a metric to quantify arm fatigue of mid-air interactions Abstract: Mid-air interactions are prone to fatigue and lead to a feeling of heaviness in the upper limbs, a condition casually termed as the gorilla-arm effect. Designers have often associated limitations of their mid-air interactions with arm fatigue, but do not possess a quantitative method to assess and therefore mitigate it. In this paper we propose a novel metric, Consumed Endurance (CE), derived from the biomechanical structure of the upper arm and aimed at characterizing the gorilla-arm effect. We present a method to capture CE in a non-intrusive manner using an off-the-shelf camera-based skeleton tracking system, and demonstrate that CE correlates strongly with the Borg CR10 scale of perceived exertion. We show how designers can use CE as a complementary metric for evaluating existing and designing novel mid-air interactions, including tasks with repetitive input such as mid-air text-entry. Finally, we propose a series of guidelines for the design of fatigue-efficient mid-air interfaces.

[voiceandgesture]

MouStress: Detecting stress from mouse motion Abstract: Stress causes and exacerbates many physiological and mental health problems. Routine and unobtrusive monitoring of stress would enable a variety of treatments, from break-taking to calming exercises. It may also be a valuable tool for assessing effects (frustration, difficulty) of using interfaces or applications. Custom sensing hardware is a poor option, because of the need to buy/wear/use it continuously, even before stress-related problems are evident. Here we explore stress measurement from common computer mouse operations. We use a simple model of arm-hand dynamics that captures muscle stiffness during mouse movement. We show that the within-subject mouse-derived stress measure is quite strong, even compared to concurrent physiological sensor measurements. While our study used fixed mouse tasks, the stress signal was still strong even when averaged across widely varying task geometries. We argue that mouse sensing "in the wild" may be feasible, by analyzing frequently-performed operations of particular geometries.

[voiceandgesture]

FingerSense: A way to think with your hands

What if we stopped creating new gestures and started using more body parts? FingerSense from Qeexo can tell the difference between your finger, fingernail, and thumb, and can differentiate between ends of a stylus, without using Bluetooth the way Pencil does.

FingerSense Overview (by Qeexo)

[voiceandgesture]

In dance, there is a practice called "marking". When dancers mark, they execute a dance phrase in a simplified, schematic or abstracted form. Based on our interviews with professional dancers in the classical, modern, and contemporary traditions, it is fair to assume that most dancers mark in the normal course of rehearsal and practice. When marking, dancers user their body-in-motion to represent some aspect of the full-out phrase they are thinking about. Their stated reason for marking is that it saves energy, avoids strenuous movement such as jumps, and sometimes it facilitates review of specific aspects of a phrase, such as tempo, movement sequence, or intention, all without the mental and physical complexity involved in creating a phrase full-out. It facilitates real-time reflection.

If you're a designer of gestural interfaces, you likely spend a lot of time marking, using your body to think about the interactions you have with a screen or camera or any machine space. This paper can give you a framework for thinking about marking and argues for the importance of going beyond 'imagining' interaction.

The opportunity to reconceptualize things is something that mental simulations does not offer. It is a major reason "externalizing" what is in mind is a more powerful strategy than working with things in the mind alone.

[voiceandgesture]

Imagine interactive electronic books that never need batteries or charging.

... a new energy harvesting technology that generates electrical energy from a user's interactions with paper-like materials. The energy harvesters are flexible, light, and inexpensive, and they utilize a user's gestures such as tapping, touching, rubbing and sliding to generate energy. The harvested energy is then used to actuate LEDs, e-paper displays and other devices to create interactive applications for books and other printed media.

Paper Generators: Harvesting Energy from Touching, Rubbing & Sliding (by DisneyResearchHub)