Research around safe Kinesthetic Haptics for Virtual Reality, at NVIDIA research and the Cornell Organic Robotics Lab.
Safe Kinesethetic Haptics – that is, experiences that interact with the muscle spindles, not the receptors underneath the surface of the skin – have always been the holy grail of Human-Computer Interaction. For a peripheral to resist muscle activity – for example push back on a finger contracting on it – it needs to apply large forces, up to hundreds of Newtons in short bursts. If the user changes their hand posture or lets go, this force breaks them immediately.
Our goal was to find ways to make safe interactions felt mostly in the muscle spindles. The team at the Cornell Organic Robotics Lab, led by Prof. Robert Shepherd, manufactured some devices which inflate rapidly. Unlike balloons, inflating these results in a relatively rigid, predictable shape. We connected these interaction devices to NVIDIA’s VR Funhouse. Below are two papers describing two techniques of Kinesthetic Interaction:
Stretchable Transducers for Kinesthetic Interactions in Virtual Reality [SIGGRAPH 2017]
In SIGGRAPH 2017, we made a technical demo previewing the IEEE paper below:
Fluidic Elastomer Actuators for Haptic Interactions in Virtual Reality [IEEE Letters on Robotics and Automation 2018]
With Robert Shepherd, Jose Barreiros, Bryan Peele, Omer Shapira, David Luebke, Benjamin Mac Murray, Josef Spjut
Paper: PDF, ACM Digital Library, ResearchGate, NVIDIA Research
Elsewhere: Josef Spjut, Jose Barrerios
Media: Road to VR, TechRadar, VRFocus, UploadVR, InAVate, Studio Daily, RealoVirtual, Cornell News, TheVRSoldier
In this research, we created a “skin” to wrap a VR controller. The skin is composed of a textured layer gripping the controller, a layer of “bladders” which can inflate to the desired shape, along with air conduits, and a top layer – also textured – to interact with the skin.
Inflation of the bladders (8) takes a fraction of a second – and deflation happens with inwards pressure from the silicon sheath itself.
A Variable Shape and Variable Stiffness Controller for Haptic Virtual Interactions [IEEE RoboSoft 2018]
With Benjamin C. Mac Murray, Bryan N. Peele, Patricia Xu, Josef Spjut, Omer Shapira, David Luebke, Robert F. Shepherd
Paper: PDF, IEEEXplore, NVIDIA Research
Press: Road to VR
In this paper, we looked at a device that instead of pushing fingers away from a surface, changes the sensation of “stiffness” of an object. To be exact, what’s changing in this example is the stiffness of the material: when measured at a point, lower stiffness means the material is behaving at a certain level of springyness. But as as an aggregate sensation (over many points) – this means the material moves around a bit, giving an impression of different physical properties. The paper discusses the choice of support structure, which affects the surface compliance and flexibility.
(A) Representative pressure profile of user-triggered inflation and deflation. (B) Sequence of demonstration interactions: [from left to right] interaction with a compliant handle corresponding to a virtual foam sword, rapid compressions of the unpressurized handle create successive pressure spike signaling to the controller to initiate a inflate the handle, interaction with the pressurized handle corresponding to a virtual metal sword, and successive compressions of the pressurized handle trigger deflation as the virtual object returns to a foam sword.
