Researchers at the University of California, Santa Cruz, have received funding from Microsoft Research to develop a virtual speech therapist, accessible on a cell phone, to aid stroke survivors in Malaysia. The self-contained language rehabilitation program will use a computer-generated talking head that provides realistic speech and mimics the natural movements of lips, tongue, and jaw.
About 40,000 people suffer from stroke every year in Malaysia, and communication impairments are common among stroke survivors, said principal investigator Sri Kurniawan, an assistant professor of computer engineering in the Baskin School of Engineering at UCSC. As in many developing countries, however, access to speech therapy is limited. A shortage of speech therapists in Malaysia is one obstacle, and patients often have difficulty traveling to existing speech therapy centers.
"This project aims to create a virtual speech therapist on a cell phone," Kurniawan said. "Initially, the patient will meet with a therapist to work out individualized therapy programs. Then the programs will be loaded onto a cell phone and given to the patient."
Kurniawan has teamed up with Dominic Massaro, professor of psychology at UCSC, who has developed computer-assisted speech and language tutors that have been used to help autistic and hearing-impaired children. Massaro's animated tutor, named Baldi, has already been programmed to speak several languages in addition to English, including Arabic and Mandarin Chinese.
For the speech therapy project, the researchers will program Baldi to speak Malay, adapt the tutor to work on mobile units such as cell phones, and develop the speech therapy programs. The project will also involve developing a standard assessment program for evaluating speech and language impairments in Malay speakers. "Aphasia" is the medical term that encompasses a range of communication impairments commonly affecting stroke survivors.
"Standardized electronic aphasia assessments and rehabilitation programs do not exist in Malay, as far as we know," said Kurniawan, a native Malay speaker from Indonesia.
Massaro said his group has been working to expand the number of languages that Baldi can use (currently 11, including English). The audio component of Baldi's speech is generated by a text-to-speech (TTS) synthesizer, and Massaro is currently evaluating existing Malay TTS engines. The visual component involves accurately matching Baldi's lip, tongue, and jaw movements to the Malay phonemes, the individual vocal units from which words are constructed.
"After we get Baldi to speak Malay, the next step is to present it on a mobile unit like a cell phone," Massaro said. "Cell phones are certainly a pervasive technology, so it's exciting to think about using this new platform."
The cell phones can be programmed to self-launch at certain times of the day to deliver the speech therapy lessons, Kurniawan said. Researchers will call the patients periodically to assess their progress. An initial study to evaluate the program will involve 10 stroke survivors with aphasia.
The National Stroke Association of Malaysia and researchers at the University of Malaysia will also be involved in the project, Kurniawan said. She currently has a pilot project in Malaysia using cell phones to conduct speech therapy through telephone consultations. But this approach is limited by the shortage of speech therapists, she said.
"The virtual speech therapist has the potential to really help stroke survivors, not only in Malaysia, but in developing countries in general," Kurniawan said.
The project is supported by a $100,000 gift from the Microsoft Research "Cell Phone as a Platform for Healthcare" program.
Note to reporters: You may contact Kurniawan at (831) 459-1037 or srikur@soe.ucsc.edu.
