Shared publicly  - 
 
Using Wearable Computers for Passive Haptic Learning and Rehabilitation
Posted by +Thad Starner, Georgia Tech Professor & Technical Lead - Google Project Glass

Ever try to learn a musical instrument only to give up because it took too much time?  What if a wearable computer could teach you the "muscle memory" needed to play a melody without the endless practice at the piano?  At the Georgia Institute of Technology, we developed Mobile Music Touch (MMT), a wireless glove that enables users to learn to play piano melodies while performing everyday tasks.  We call this process Passive Haptic Learning.  

The MMT device looks like a fingerless workout glove with a small box on the back containing a Bluetooth radio and microcontroller.  Vibration motors are embedded in the glove at the base of each finger and the thumb.  The song being learned can be loaded onto a user’s mobile phone and is played as they go about their business.  As each note is played, the glove taps the finger corresponding to the appropriate key on a piano keyboard.  The result: in as little as 30 minutes, the user learns the "muscle memory" of the first phrases of the song, even if their attention has been devoted to another task.

To test just how passive learning can be, we distracted users with various tasks.  Participants engaged in a variety of activities including reading emails, taking a graduate entrance exam, performing a memory test, watching a movie, and even completing a scavenger hunt while the MMT tapped their fingers. Even though they were focused on the distractor tasks, those who used the MMT showed significant learning and rehearsal effects compared to controls.  Surprisingly, only the vibration was needed for the glove to be effective-- on average, participants who only "felt" the music performed as well as those who heard the audio concurrently.

Dr. Debbie Backus of the Shepherd Center in Atlanta saw a demonstration of MMT and suggested it might be used for rehabilitation.  Working with Tanya Markow, a PhD student at the time, we designed a study to test whether the MMT glove might help people with tetraplegia due to partial spinal cord injury.  We recruited ten participants who were over a year past injury, the point at which rehabilitative improvements are thought to taper off.  Five participants used the glove for eight weeks and showed significant improvements on standardized tests of hand sensation and dexterity compared to the control subjects.  One participant stated he could "open doors better, and it’s easier to turn the key to start my car and use a letter opener now."  A pilot subject reported that he could button his own buttons after the experiment.  

While this work is still preliminary, the experiment suggests that Passive Haptic Rehabilitation may be possible with wearable devices.  We’re excited about this prospect and look forward to seeing the possibilities of haptics-based wearable computers unfold. 
-----

For more information, see:

K. Huang, T. Starner, E. Do, G. Weiberg, D. Kohlsdorf, C. Ahlrichs,
and R. Leibrandt.  "Mobile music touch: mobile tactile stimulation for
passive learning."  ACM CHI, pp. 791-800, 2010.
(http://www.cc.gatech.edu/~thad/starner-chi2010-mobile-music-touch.pdf)

T. Markow.  "Mobile Music Touch: using haptic stimulation for passive
rehabilitation and learning."  PhD Dissertation, Georgia Institute of
Technology, May 2012
(http://smartech.gatech.edu/handle/1853/43665)

"Mobile Music Touch: Learning to Play Piano Melodies Without Attention" (http://youtu.be/uEdr6iY6F-w)
70
31
Jeb Bradwell's profile photoOplerno's profile photoHolly Gentry's profile photoKim André Sand's profile photo
7 comments
 
oh wow! I love this invention!
 
I'm sure Duke Ellington is rolling over in his grave right now.
 
What a fantastic tool ! rehabilitation time could be cut in half for serious hand injuries! Learning to play piano at age 50 could actually happen!
 
So true +Vince Hiller  and the joy and confidence alone would be huge for individuals with those obstacles... a new gained freedom on life I'd say my friend. Thanks for the comment, much appreciated.
Add a comment...