Thesis Proposal - idea 1

Thesis Statement

    This project is to create a computer game which is interacted with the player’s facial expression. This game is designed not only for entertainment, but also for medical treatment such as facial physical therapy/speech therapy. In short, this project is to create a serious game called facial aerobics.

Thesis Description

    In contemporary art, computer game is a type of interactive artwork. In the recent decades, lots of new generation of computer games have being developed, from the changes of game type to the transformation of participation experience. Due to the evolution of game type and interface design, the goal of computer game has been become not only for entertainment, but has additional purposes such as education and training, which is well-known as “Serious Game”.
Thanks to the electronic technology evolution, lots of new interfaces are developed. Human body is one of the new forms in this extension. A player uses his entire body (real or shadow), or part of his body (hands or head etc.) as an interface, to interact with the game world. Through this new experience of interaction, designers intent the players enjoy the game not only mentally, but also physically. In this design document, I would like to use a human face as an interface to navigate a game. Using existing face tracking software to capture a player’s facial expression, to transform the visual feedback from the reality into a meaningful command to the game world. As a result, this game can achieve multiple goals: entertainment and medical treatment purposes.

Facial Expression Recognition Toolkits

    Below are some of the facial expression recognition toolkits available in the market:

• Facerobot from Softimage
  http://www.softimage.com/products/face_robot/default.aspx


• Eyematic from FaceStation
  http://www.digitalmediadesigner.com/articles/viewarticle.jsp?id=10880


• FaceLAB from Seeing Machines
  http://seeingmachines.com


• The MAKE controller kit from makingthings.com
  http://makezine.com/controller


• Cyclops – Realtime Video Analyzer from Cycling74
  http://www.cycling74.com/products/cyclops


• Attentive Interaction Design Toolkit from Jacky Lee, MIT media Lab
  http://web.media.mit.edu/~jackylee/attention.htm


• OpenCV - Face tracking software from Intel
  http://sourceforge.net/projects/opencvlibrary

Where item 4 goes with item 5, and item 6 goes with item 7.

    In this project, I am going to use Attentive Interaction Design Toolkit as my interface mockup.

Interface Mockup using Attentive Interaction Design Toolkit

Facial Physical Therapy

Introduction

    In psychology, facial expression is a kind of non-verbal communication. It reveals the innermost feelings of human beings. In 1972, Ekman et al. suggests six emotions can be detected in faces: happiness, surprise, anger, sadness, fear and disgust/contempt.1
For the people who lose the ability to control their facial muscles, they lose their communication ability as well. In other words, facial paralysis is a disability of communication, both physiological and psychosocial. Clinically, facial paralysis has two types: Flaccid paralysis/paresis (complete facial paralysis), and Synkinesis/mass action (varying degrees of motor weakness accompany by abnormal, asynchronous facial movements).2 Different type of facial paralysis requires different treatment tactics.

Treatment - Neuromuscular Retraining

    Nowadays, physical therapy is still the recommended treatment for facial paralysis. “Gross facial exercises, massage, electrical stimulation, and prosthetic devices or taping to lift a drooping, flaccid face”2 are the traditional therapy techniques in decades ago. However, most therapists found that they are ineffective. Thirty years ago, neuromuscular retraining (NMR) technique appeared in the literature, now known as an effective way for optimal recovery from facial nerve paresis.2
This technique is “a marriage of neurophysiology, psychology, therapeutic science, learning theory, and art.”2 It is using surface electromyography (EMG) feedback and specific mirror exercises to modify the manner in which patients contracted their muscles. However, there are no generic lists of exercises for the patients because clinicians found that each patient has his own functional profile.2

Facial Movements

    Clinically, facial movement falls into two types: volitional and spontaneous. In some cases, a patient lost the ability of expressing emotions but still have the ability of volitional control (Figure 1, right column). Contrarily, there is also a case that a patient is able to produce emotional expression but failed to move their facial muscle voluntarily (Figure 1, left column).3

Figure 1. Voluntary and emotional facial paresis 3

In this case, my game is more suitable for the patient who has the ability of volitional control.

Volitional Movement vs. Facial Muscle Groups

Figure 2. Major facial muscle groups4

Table 1. Volitional movement vs. facial muscle groups2, dotted lines indicates the expressions can be recognized easily.

From this table, we have the idea of what expression movements are exercising which facial muscle groups.

Example Clinical Program For Facial Physical Therapy

Figure 3. Personal exercise program from Chatham Sports Medicine & Physical Therapy

Speech Therapy

Introduction

    Speaking is our most significant channel of communication. Orofacial Myofunctional Disorders are a type of speech disorders/communication disorders where ‘normal’ speech is disrupted. “Orofacial Myofunctional Disorders (OMD) involve behaviors and patterns created by inappropriate muscle functional and incorrect habits involving the tongue, lips, jaw and face.”6 The excessive pressure caused by OMD may result in speech problem. Tongue thrust and abnormal lips resting position are the most common orofacial myofunctional variations.6

Treatment - Oral Facial Myofunctional Therapy

    Oral facial myofunctional therapy is the common speech therapy program for ODM. It provides therapeutic oral and facial exercises to develop appropriate tongue and lip resting postures. “Oral Aerobics”7 is one of the existing treatment programs for ODM. Oral Aerobics is a videotape program which provides a series of lip and tongue exercises together with dynamic music. Below table is a list of lip and tongue exercises provided by the videotape:


The preview video clip can be downloaded from http://www.trainyourbrainco.com/paerobics.htm. This video is one of my major references for the facial expression patterns using in my game.

Bibliography

• Eysenck, Michael. “Psychology: An International Perspective.” Chapter 5: Emotion, stress, and coping. April 2004. On-line. Available from . Internet. Accessed 18 July 2006.


• “Bell’s Palsy Information Site.” Facial Retraining. On-line. Available from http://www.bellspalsy.ws/retrain.htm. Internet. Accessed 10 July 2006.


• Purves, Dale. Neuroscience, 2nd ed. Book On-line. U.S.A., Sinauer Associates, Inc., 2001. Available from . Internet. Accessed 18 July 2006.


• Gray’s Anatomy, 38th ed. New York: Churchill Livingstone, 1999.


• Clemente, Carmine. Gray’s Anatomy, 30th American ed. Philadelphia: Lea & Febiger, 1985.


• “International Association of Orofacial Myology.” What’s Orofacial Myology? On-line. Available from http://www.iaom.com/iaom_whatsom.html. Internet. Accessed 15 July 2006.


• “Train Your Brain.” Oral Aerobics Product. On-line. Available from http://www.trainyourbrainco.com/paerobics.htm. Internet. Accessed 15 July 2006.