Computer-Assisted Speech Training for Cochlear Implant
Transcript Of Computer-Assisted Speech Training for Cochlear Implant
Computer-Assisted Speech Training for Cochlear Implant Patients: Feasibility, Outcomes, and Future Directions
Qian-Jie Fu, Ph.D.,1and John J. GalvinIII, Ph.D.1
ABSTRACT
Learning electrically stimulated speech patterns can be a new and difficult experience for cochlear implant patients. Cochlear implantation alone may not fully meet the needs of many patients, and additional auditory rehabilitation may be necessary to maximize the benefits of the implant device. A recently developed computer-assisted speech-training program provides cochlear implant patients with the means to conduct auditory rehabilitation at home. The training software targets important acoustic contrasts between speech stimuli and provides auditory and visual feedback as well as progressive training, thereby maintaining patients’ interest in the auditory training exercises. Recent scientific studies have demonstrated the effectiveness of such specialized auditory training programs in improving cochlear implant patients’ speech recognition performance. Provided with an inexpensive and accessible auditory training program, cochlear implant patients may find the motivation and momentum to get the most from the implant device.
KEYWORDS: Auditory training, cochlear implants, computer-assisted speech training
Learning Outcomes: As a result of this activity, the participant will be able to (1) discuss the importance of auditory training for cochlear implant patients, and (2) summarize the recent development of computer-assisted speech training specially designed for cochlear implant patients.
The cochlear implant (CI) is an electronic
device that provides hearing sensation to pa-
tients with profound hearing loss. As the sci-
ence and technology of the cochlear implant has developed over the past 50 years, the overall speech recognition of CI patients has steadily
1Department of Auditory Implants and Perception, House Ear Institute, Los Angeles, California.
Address for correspondence and reprint requests: QianJie Fu, Ph.D., Department of Auditory Implants and Perception, House Ear Institute, 2100 West Third St., Los Angeles, CA 90057. E-mail: [email protected]
Auditory Training; Guest Editor, Robert W. Sweetow, Ph.D.
Semin Hear 2007;28:141–149. Copyright # 2007 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584–4662. DOI 10.1055/s-2007-973440. ISSN 0734-0451.
141
142 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
improved. With the most advanced implant technology and speech processing strategies, many patients receive great benefit, and are capable of conversing with friends and family over the telephone. However, considerable variability remains in individual patient outcomes. Some patients receive little benefit from the latest implant technology, even after many years of daily use of the device. This variability in patient outcomes is reflected not only in differences in speech recognition performance, but also in the time course of adaptation to novel speech patterns via electric hearing. Although some patients may easily and quickly adapt to their implant, others may require an extensive learning period.
Much research has been devoted to exploring the sources of variability in CI patient outcomes. Some studies have shown that patient-related factors, such as duration of deafness, are correlated with speech performance.1,2 Several psychophysical measures, including electrode discrimination,3 temporal modulation detection,4,5 and gap detection,6– 8 also have been correlated with speech performance. Furthermore, Kelly et al2 found that earlier P2 latencies of auditory evoked potentials (AEPs) were associated with shorter durations of deafness and higher speech scores. In addition, mismatch negativity was absent or degraded in CI patients with poor speech scores. The correlation between duration of deafness and speech performance suggests that early implantation may benefit deaf patients profoundly. The correlation between psychophysical measures and speech performance suggests that both spectral resolution and temporal resolution are important for speech recognition in electric hearing. The correlation between AEPs and speech performance provides objective evidence of central auditory processing differences among experienced CI users.
Besides the high variability in CI patient outcomes, individual patients also differ in terms of the time course of adaptation to electric hearing. During the initial period of use, postlingually deafened CI patients must adapt to differences between their previous experience with normal acoustic hearing and the pattern of activation produced by electrical
stimulation. Many studies have tracked changes in performance over time in na¨ıve or newly implanted CI users. These longitudinal studies showed that most gains in performance occur in the first 3 months of use.9–13 However, continued improvement has been observed over longer periods for some CI patients.14 Experienced CI users also must adapt to new electrical stimulation patterns provided by updated speech processors, speech processing strategies and/or changes to speech processor parameters. For these patients, the greatest gains in performance also occurred during the first 3 to 6 months, with little or no improvement beyond 6 months.15,16
Overall, these results suggest that considerable auditory plasticity exists in CI patients, even after years of experience with their device. Because of the spectrally degraded speech patterns provided by the implant, passive learning via long-term use of the device may not fully engage patients’ capacity to learn novel stimulation patterns. Active auditory training may better exploit CI patients’ auditory plasticity and facilitate learning of electrically stimulated speech patterns. Auditory training, an important facet of aural rehabilitation, has been shown to improve hearingimpaired patients’ speech comprehension and communication.17 Currently, fewer than 10% of practicing audiologists offer hearing-impaired patients comprehensive auditory training, compared with 16% in 1990 and 31% in 1980.18 Hearing healthcare professionals acknowledge that cochlear implantation alone may not fully meet the needs of many patients, and that additional auditory therapy may enhance the benefits of the implant device. However, for a variety of reasons, few hearing healthcare professionals routinely include auditory training in the services they provide to CI patients. Time and cost considerations often preclude providing extensive and intensive auditory therapy. Given the limited number of speech pathologists working with CI patients and the high costs associated with auditory rehabilitation, it has long been urgent to develop an inexpensive and effective auditory training system for CI patients, especially for those patients who have the greatest difficulty with speech understanding.
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 143
METHOD The computer-assisted speech training (CAST) program, developed at the House Ear Institute (Los Angeles, CA), provides CI patients with self-directed auditory rehabilitation via home computer. As an auditory training tool, CAST provides some appealing advantages over conventional, site-specific approaches for auditory rehabilitation. First, the cost of individualized auditory training is significantly lower than that with traditional auditory rehabilitation provided by hearing clinics and hospitals. Second, CAST is easily accessible by CI patients, who can practice at home at any time, provided they have access to a computer. Third, patients’ progress can be easily monitored and shared with their clinical practitioners. Finally, CAST provides auditory training that is specific to the needs of CI users, and more importantly to the needs of individual CI patients. CAST targets acoustic contrasts that are especially problematic for CI users, as opposed to targeting cognitive/developmental learning. Besides being affordable, accessible, and appropriate for CI patients, CAST has many other unique features.
Comprehensive Training Materials CAST provides a comprehensive set of training materials with which to train recognition of many kinds of sounds, not just speech. Training materials include simple pure tones, environmental sounds, monosyllabic words, consonant training stimuli (in v/C, v/C/v, and C/v contexts), familiar words, familiar sentences, simple melodic sequences and familiar melodies. For targeted phonetic contrast training, CAST uses more than 1000 novel monosyllabic words (for initial, medial and final vowel and consonant training) and nonsense words (for initial, medial and final consonant training), spoken by four different talkers. Additional training materials and/or training modules can be easily incorporated for individual patients to maximize training outcomes. For advanced users, CAST also can mimic difficult listening environments by adding background noise or competing speech in real time.
Individualized Training Protocols CAST also provides individualized training protocols for CI patients. The level of difficulty is automatically adjusted according to individual patient performance by increasing the number of response choices and/or reducing the acoustic differences between response choices. During training, both auditory and visual feedback is provided, allowing users to compare incorrect responses to correct responses repeatedly. As performance improves, the level of difficulty is automatically increased. If performance does not improve, the level of difficulty is decreased. At the end of each testing and training session, the program offers training guidance. For example, based on testing or training results, the program suggests the appropriate training level, whether users should go to an easier or more difficult training level, etc. Feedback and encouragement are offered throughout, as the positive effects of good will can help patients build momentum during the rehabilitation process.
Integrated Training Structures CAST provides easy integration for auditory rehabilitation between CI patients and their audiologists or speech pathologists. All training and test results are saved in a standardized database. Users can view results and ongoing progress and data can be exported and shared with the patient’s hearing health practitioner. Results are automatically saved and include user’s name, test and training results, test and training date, training time for each exercise, and total training time. Audiologists can use these results to track a patient’s progress over time, check for potential problems with the CI device, and optimize the speech processor. For example, during a clinical visit, the audiologist can review the test results and adjust processor parameters (e.g., frequency allocation, channel gains, etc.) to improve performance in problem areas (e.g., reception of fricatives, etc.). Over a longer term, different magnetic acoustic pressuresQ1 (MAPs with different parameter set- Q1 tings) can be loaded onto the processor and performance can be tracked over time to determine the optimal MAP; the combination of take-home experience and laboratory testing
144 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
(with easy access to results) will allow for better PHONEME RECOGNITION TRAINING IN
long-term optimization of the processor. CAST provides a visually clean and user-
COCHLEAR IMPLANT PATIENTS
Fu et al25 investigated the effects of computer-
friendly graphic interface, allowing patients to assisted speech training on the recognition
concentrate on listening rather than navigating performance of CI users. Ten adult CI patients
an unfamiliar computer program.
with limited speech recognition capabilities
participated in the study. Baseline (pretraining)
multitalker phoneme recognition performance
TRAINING OUTCOMES
was measured for at least 2 weeks, or until
Although CAST provides inexpensive and ac- performance reached an asymptote. After base-
cessible auditory rehabilitation for CI patients, line measures were complete, participants
the most important question is whether com- trained at home using the CAST program
puter-assisted auditory training can effectively (loaded onto their personal computer or loaner
improve CI patients’ speech recognition per- laptop). Participants were instructed to train at
formance, especially for patients who have great home 1 hour per day, 5 days per week for a
difficulty with speech understanding. Auditory period of 1 month or longer. Participants re-
training has been shown effective in the reha- turned to the laboratory every 2 weeks for
bilitation of children with central auditory processing disorders,19,20 children with language-learning impairment,21,22 and hearing
aid (HA) users.17 Some earlier studies have
retesting (same tests as baseline measures). Results showed that both vowel and consonant recognition significantly improved for all participants after training. Mean vowel recognition
assessed the effects of auditory training on improved from 23.7 to 39.5% correct (paired t
speech recognition by poor-performing CI patients.23,24 Busby et al23 examined the effects of
test: p < 0.0001). Mean consonant recognition improved from 25.1 to 38.6% correct
10 one-hour speech perception training ses- (p < 0.005). For a subset of CI participants,
sions (1 to 2 sessions per week); three prelin- mean sentence recognition also improved
gually deafened CI users (two adolescents and from 27.9 to 55.8% correct (p < 0.01) after
one adult) participated in the experiment. After training. Although performance significantly
the training period, there were only minimal improved for all participants after 4 weeks or
changes in speech performance; the participant more of moderate training, there was signifi-
with the greatest improvement was implanted cant inter participant variability in terms of the
at an earlier age, and therefore had a shorter amount and time course of improvement. For period of deafness. Dawson and Clark24 re- some participants, performance significantly
ported more encouraging results for vowel rec- improved after only a few hours of training,
ognition training in five CI users. Each CI user whereas others required a much longer time
had been deaf for at least 4 years prior to course.
implantation, and none had achieved open-set
speech recognition. Training consisted of one MELODIC SEQUENCE IDENTIFICATION TRAINING
50-minute training session per week for 10 IN COCHLEAR IMPLANT PATIENTS
weeks. Following training, four of the five Music perception and appreciation remains
participants showed improvement in some measures; this improvement was retained on subsequent testing three weeks after training
difficult for most CI patients. Galvin et al Q3(J.J. Galvin 3rd, Fu Q-J, and G. Nogaki, Q3 unpublished data, 2005) developed an objective
was completed. The amount and type of train- test of CI users’ music perception: melodic
ing employed may account for the mixed and sequence identification (MSI). During the
generally poor outcomes from earlier CI speech MSI test, CI patients were asked to identify Q2 training studiesQ2. Several recent studies have simple five-note melodic sequences, in which
evaluated the effects of computer-assisted audi- the pitch contour was systematically varied.
tory training (using the CAST program) on CI The interval between successive notes was sys-
patient performance, and are summarized be- tematically varied between one and five semi-
low.
tones to test the intonation provided by the CI
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 145
device. Results showed large interparticipant variability in MSI performance. The best performers correctly identified more than 90% of the melodic sequences when there were two semitones between notes in a sequence; poor performers correctly identified less than 40% of the sequences with five semitones between notes. Six CI patients with poor MSI performance were trained using the CAST software to see whether MSI performance could be improved with moderate auditory training. Participants were instructed to train for 1 hour per day, 5 days per week, for a period of 1 month or longer. For all participants, performance significantly improved with training (p ¼ 0.004); the amount of improvement ranged from 15.5 to 45.4 percentage points. Pre- and posttraining familiar melody identification (FMI) also was measured in four of the six CI participants; 12 melodies were tested without rhythm cues, similar to Kong et al.26 Results showed that after training, mean FMI performance significantly improved by 20.8 percentage points (p ¼ 0.020). Anecdotal reports suggested that CI patients’ music perception and appreciation generally improved after MSI training. For example, some participants reported that they were better able to separate the singer’s voice from the background music while listening to music in the car.
CHINESE TONE RECOGNITION TRAINING IN
MANDARIN-SPEAKING COCHLEAR IMPLANT
PATIENTS
In tonal languages such as Mandarin Chinese, the tonality of a syllable is lexically important,27,28 and Chinese sentence recognition is highly correlated with tone recognition.29,30 Although the fundamental frequency (F0) contour contributes most strongly to tone recognition,27 other temporal cues that covary with tonal patterns (e.g., amplitude contour and periodicity fluctuations) also contribute to tone recognition, especially when F0 cues are reduced or unavailable.29,31,32 J.-L. Wu, H.-M. Yang, Y.-H. Lin, and Q.-J. Fu (unpublished data, 2006) recently investigated whether moderate auditory training can improve the recognition of Chinese vowels, consonants, and tones by hearing-impaired children. Ten Mandarin-speaking children (seven CI users and
three HA users) participated in the study. Training was conducted at home using the CAST program; participants trained for one half-hour per day, 5 days per week, for a period of 10 weeks. Results showed that mean vowel recognition scores significantly improved from 63.1 to 84.8% correct (paired t test: p ¼ 0.006) after the 10-week training period. Similarly, mean consonant recognition scores significantly improved from 39.7 to 58.2% correct (p < 0.001) and Chinese tone recognition scores significantly improved from 56.0 to 71.1% correct (p ¼ 0.007). Vowel, consonant, and tone recognition performance was re-measured after training was completed; follow-up performance was measured at 1, 2, 4, and 8 weeks after training was completed. Followup measures remained significantly higher than pretraining baseline measured for vowel (p ¼ 0.018), consonant (p ¼ 0.002), and tone recognition (p ¼ 0.009), suggesting that the improved performance with training was retained well after training had stopped.
DISCUSSION The results from these studies demonstrate that moderate amounts of auditory training with the CAST program improved CI and HA users’ recognition of phonemes, melodic sequences, and Chinese tones. Although auditory training generally improved performance in the targeted listening task, the improvement sometimes generalized to auditory tasks that were not explicitly trained (i.e., improved sentence recognition after training with phonetic contrasts, or improved familiar melody identification after training with simple melodic sequences). Importantly, the improved performance was not transitory, as posttraining performance levels remained higher than pretraining baseline levels 1 to 2 months after training was completed. Because most CI participants had a minimum of one year of experience with their device prior to training (i.e., at least one year of passive learning), the results suggest that active learning was necessary to improve speech performance. The improvements in speech recognition with auditory training were comparable to (and often much greater than) gains reported with recent advances in CI technology and
146 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
speech processing. Taken together, the results suggest that computer-assisted auditory training, as implemented in the CAST program, provides an effective, inexpensive, and accessible auditory rehabilitation for CI patients.
The results from these recent training studies with the CAST program have shown successful training outcomes with CI patients; nevertheless, earlier studies have shown somewhat mixed results.23,24 The better training outcomes with the CAST program may have been due to differences in terms of training protocols. For example, one-to-one personal training that targeted a variety of phonetic contrasts was used in Dawson and Clark,24 whereas computer-assisted progressive training that targeted minimal phonetic contrasts was used in the CAST studies. Fu et al33 systematically evaluated the effects of training protocols on NH listeners’ adaptation to eightchannel spectrally shifted speech. Four different training protocols have been evaluated, including test-only (repeated baseline measures), stimulus preview, phonetic contrast training, and sentence training. Results showed a significant effect of the training protocol on the training outcomes. The targeted, progressive phonetic contrast training (which is used in the CAST program) produced the greatest improvement in recognition of spectrally shifted vowels. The improved vowel recognition also generalized to improved consonant and sentence recognition with spectrally shifted speech.
The better training outcomes with the CAST program also may also be due to differences in terms of the frequency of training, i.e., how often the training was conducted during the training period. In previous CI training studies,23,24 participants trained 50 minutes once per week, for 10 weeks; however, in the CAST studies, participants trained 30 to 60 minutes per day, 5 days per week, for a period of one month or longer. G. Nogaki, Q-J Fu, and J.J. Galvin (unpublished data, 2006) systematically evaluated the effect of training frequency on NH listeners’ adaptation to eight-channel spectrally shifted speech. The participants completed five training sessions using the phonetic contrast training protocol.25,33 Eighteen participants were divided into three groups, in which
1-hour training sessions were provided one, three, or five times per week. The results suggest that although more frequent training seemed to provide better adaptation over the five training sessions, the frequency of training did not significantly affect training outcomes (at least for the training frequencies that were studied). Thus, it may be more important to complete an adequate number of training sessions over a reasonable period. These simulation results suggest that CI patients may significantly benefit from auditory training, even when infrequently performed. The results further suggest that the difference in CI training outcomes between these recent studies and other earlier studies may not be simply due to time-intensive training, but rather due to differences in training protocols and (perhaps) training stimuli. CAST makes use of a far greater number of multitalker speech stimuli (more than 4000 stimuli) than used in previous CI speech training studies. The results suggest that it may be more important to develop effective training protocols and stimuli than to simply increase patients’ time commitment.
It is important to note that in the CAST training studies,25,26,36 including unpublished data from G. Nogaki, Q-J Fu, and J.J. Galvin (2006) and J-L Wu, H-M Yang, Y-H Lin, and Q-J Fu (2006), the stimuli used for baseline measures were not used for training. Baseline measures were conducted using standard speech test databases (for example, the vowel stimuli recorded by Hillendbrand et al16 or the consonant stimuli recorded by Shannon et al34), although training was conducted using monosyllable and/or nonsense words produced by completely different talkers. Because improved performance for a trained auditory task often generalized to improved performance in other auditory tasks (for example, phonetic contrast training improved phoneme and sentence recognition, melodic sequence training improved both sequence and familiar melody identification), the large multitalker database used in the CAST may provide some advantage over smaller, more limited databases. The generalized improvements in performance agree with previous neurophysiological studies,35 which showed that both behavioral performance and neurophysiological changes observed after
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 147
auditory training generalized to stimuli not used in the training, thus demonstrating behavioral transfer of learning and plasticity in underlying physiologic processes. Tremblay et al36 reported results further indicating that training-associated changes in neural activity may precede behavioral learning. The results in Fu et al25 also indicated that the time course of improvement varied significantly across CI patients. Depending on patient-related factors (e.g., the number of implanted electrodes, the insertion depth of the electrode array, duration of deafness, etc.), some CI patients may require much more auditory training to noticeably improve their performance. Objective neurophysiological measures may provide useful information about the progression of training, i.e., whether a particular training protocol should be continued or whether the training protocol and/or training materials should be adjusted. These objective measures may allow for the development of more efficient training protocols for CI patients; fortunately, the flexibility in computer-assisted auditory training (as in CAST) allows such changes to be easily implemented.
A future direction that may further improve CI patient training outcomes is to integrate the standardized testing and training features of the CAST program fully into the clinical fitting system. Many previous CI studies15,37,38 have shown that acutely measured performance may significantly underestimate the effects of changes in speech processing. Even a short period of adaptation may result in significantly better speech performance.33,39 Longer adaptation periods may be required to fully evaluate the effects of changes in speech processing. By integrating the test and training platforms into the clinical fitting procedures, CAST may help accelerate the adaptation process over the short and long term. Audiologists have little time available to test and retest performance with different MAPs; CAST allows training and testing with little or no supervision on the part of the audiologist. Thus, during clinical visits, MAPs may be evaluated more accurately after some amount of training and testing. In addition, during a clinical visit, CI patients largely judge the quality of a speech processor according to the audiologist’s voice; by providing multiple
talkers, CAST allows patients to better judge the quality of the MAP. Over the long term, patients can compare multiple MAPs loaded onto their speech processors in everyday listening conditions outside of the clinic (CAST training also can help in this period of adaptation), and then return to the clinic for testing. Testing results can be used to determine optimal MAPs or further adjustments to speech processors.
How a parameter change is introduced also may influence the time course and degree of adaptation. Fu et al37 found that CI patients were able to only partially adapt to an abrupt change in the acoustic frequency allocation, even after 3 months of continuous use. In a follow-up study with one of the CI participants, gradually introducing a severe shift in frequency allocation provided for a more complete adaptation, albeit over a much longer adaptation period (18 months). Thus, although passive learning (incidental learning via everyday experience with the CI device) may allow for some degree of adaptation to changes in speech processing, active auditory training may help to accelerate the adaptation process and provide adaptation that is more complete. Because it is important to evaluate speech processor settings in light of adaptation, and because auditory training may accelerate the adaptation process, test and training results from the CAST program may be used to guide gradual adjustments to speech processor settings, thereby making the adaptation more complete and less stressful in the end.
Advances in computer technology have contributed greatly to the latest-generation CI technology. These advances also have allowed for the development of important rehabilitation tools, which may ultimately provide the greatest benefit for those CI patients who have difficulty with their device. Motivation is an important factor in successful patient outcomes. With the right tools and resources for auditory rehabilitation, CI patients may find the necessary motivation and momentum to get the most from their implant.
SUMMARY AND CONCLUSIONS Computer-assisted auditory training, as implemented in the CAST program, provides an
148 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
economical and effective alternative or comple- 8. Muchnik C, Taitelbaum R, Tene S, Hildesheimer
ment to site-specific auditory rehabilitation. Moderate amounts of auditory training performed at home with the CAST software resulted in significant improvements in CI patients’ speech, music, and Chinese tone recognition. These improvements were retained months after training was completed. Targeted
M. Auditory temporal resolution and open speech recognition in cochlear implant recipients. Scand Audiol 1994;23:105–109 9. George CR, Cafarelli Dees D, Sheridan C, Haacke N. Preliminary findings of the new Spectra 22 speech processor with first-time cochlear implant users. Ann Otol Rhinol Laryngol Suppl 1995; 166:272–275
auditory training tasks with novel training 10. Gray RF, Quinn SJ, Court I, Vanat Z, Baguley
stimuli also generalized to improved performance in other auditory training tasks.
DM. Patient performance over eighteen months with the Ineraid intracochlear implant. Ann Otol Rhinol Laryngol Suppl 1995;166:275–277
11. Loeb GE, Kessler DK. Speech recognition per-
ABBREVIATIONS
formance over time with the Clarion cochlear prosthesis. Ann Otol Rhinol Laryngol Suppl 1995;
166:290–292
CAST computer-assisted speech training
12. Spivak LG, Waltzman SB. Performance of coch-
CI
cochlear implant
lear implant patients as a function of time. J Speech
FMI familiar melody identification
HA hearing aid
Q4
MAP magnetic acoustic pressureQ4
NH normal hearing
Hear Res 1990;33:511–519 13. Waltzman SB, Cohen NL, Shapiro WH. Long-
term effects of multichannel cochlear implant usage. Laryngoscope 1986;96(10):1083–1087 14. Tyler RS, Gantz BJ, Woodworth GG, Fryauf-
Bertschy H, Kelsay DM. Performance of 2- and
3-year-old children and prediction of 4-year from
REFERENCES
1-year performance. Am J Otol 1997;18:S157–S159
15. Dorman MF, Loizou PC. Changes in speech
1. Eggermont JJ, Ponton CW. Auditory-evoked
intelligibility as a function of time and signal
potential studies of cortical maturation in normal
processing strategy for an Inneraid patient fitted
hearing and implanted children: correlations with
with continuous interleaved sampling (CIS) pro-
changes in structure and speech perception. Acta
cessors. Ear Hear 1997;18:147–155
Otolaryngol 2003;123:249–252
16. Hillenbrand J, Getty LA, Clark MJ, Wheeler K.
2. Kelly AS, Purdy SC, Thorne PR. Electrophysio-
Acoustic characteristics of American English
logical and speech perception measures of auditory
vowels. J Acoust Soc Am 1995;97:3099–3111
processing in experienced adult cochlear implant 17. Sweetow R, Palmer CV. Efficacy of individual
users. Clin Neurophysiol 2005;116:1235–1246
auditory training in adults: a systematic review of
3. Donaldson GS, Nelson DA. Place-pitch sensitivity
the evidence. J Am Acad Audiol 2005;16:494–
and its relation to consonant recognition by
504
cochlear implant listeners using the MPEAK and 18. Bloom S. Technologic advances raise prospects for
SPEAK speech processing strategies. J Acoust Soc
a resurgence in use of auditory training. The
Am 1999;107:1645–1658
Hearing Journal 2004;57(8):19–24
4. Cazals Y, Pelizzone M, Saudan O, Boex C. Low- 19. Hesse G, Nelting M, Mohrmann B, Laubert A,
pass filtering in amplitude modulation detection
Ptok M. Intensive inpatient therapy of auditory
associated with vowel and consonant identification
processing and perceptual disorders in childhood.
in subjects with cochlear implants. J Acoust Soc
HNO 2001;49:636–641
Am 1994;96:2048–2054
20. Musiek FE, Baran JA, Schochat E. Selected
5. Fu Q-J. Temporal processing and speech recog-
management approaches to central auditory proc-
nition in cochlear implant users. Neuroreport 2002;
essing disorders. Scand Audiol Suppl 1999;51:63–
13:1635–1640
76
6. Busby PA, Clark GM. Gap detection by early- 21. Merzenich MM, Jenkins WM, Johnston P,
deafened cochlear-implant subjects. J Acoust Soc
Schreiner C, Miller SL, Tallal P. Temporal
Am 1999;105:1841–1852
processing deficits of language-learning impaired
7. Cazals Y, Pelizzone M, Kasper A, Montandon P.
children ameliorated by training. Science 1996;
Indication of a relation between speech perception
271(5245):77–81
and temporal resolution for cochlear implantees. 22. Tallal P, Miller SL, Bedi G, et al. Language
Ann Otol Rhinol Laryngol 1991;100:893–895
comprehension in language-learning impaired
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 149
children improved with acoustically modified 31. Fu Q-J, Zeng F-G. Effects of envelope cues on
speech. Science 1996;271:81–84
Mandarin Chinese tone recognition. Asia-Pacific J
23. Busby PA, Roberts SA, Tong YC, Clark GM.
Speech Lang Hear 2000;5:45–57
Results of speech perception and speech production 32. Whalen DH, Xu Y. Information for Mandarin
training for three prelingually deaf patients using a
tones in the amplitude contour and in brief
multiple-electrode cochlear implant. Br J Audiol
segments. Phonetica 1992;49:25–47
1991;25:291–302
33. Fu Q-J, Nogaki G, Galvin JJ III. Auditory training
24. Dawson PW, Clark GM. Changes in synthetic and
with spectrally shifted speech: an implication for
natural vowel perception after specific training for
cochlear implant users’ auditory rehabilitation.
congenitally deafened patients using a multichan-
J Assoc Res Otolaryngol 2005;6:180–189
nel cochlear implant. Ear Hear 1997;18:488–501 34. Shannon RV, Jensvold A, Padilla M, Robert ME,
25. Fu QJ, Galvin JJ 3rd, Wang X, Nogaki G.
Wang X. Consonant recordings for speech testing.
Moderate auditory training can improve speech
J Acoust Soc Am 1999;106:L71–L74
performance of adult cochlear implant users. 35. Tremblay K, Kraus N, Carrell TD, McGee T.
J Acoust Soc Am 2005;113:1065–1072
Central auditory system plasticity: generalization to
26. Kong YY, Cruz R, Jones JA, Zeng F-G. Music
novel stimuli following listening training. J Acoust
perception with temporal cues in acoustic and
Soc Am 1997;102:3762–3773
electric hearing. Ear Hear 2004;25:173–185
36. Tremblay K, Kraus N, McGee T. The time course
27. Lin M-C. The acoustic characteristics and percep-
of auditory perceptual learning: neurophysiological
tual cues of tones in Standard Chinese. Chinese
changes during speech-sound training. Neuroreport
Q5
Yuwen 1988;204:182–193Q5
1998;9:3557–3560
28. Wang R-H. Chinese phonetics. In: Chen Y-B, 37. Fu QJ, Shannon RV, Galvin JJ. Perceptual learning
Wang R-H, eds. Speech Signal Processing. Hefei,
following changes in the frequency-to-electrode
China: University of Science and Technology of
assignment with the Nucleus-22 cochlear implant.
China Press; 1989:37–64
J Acoust Soc Am 2002;112(4):1664–1674
29. Fu Q-J, Zeng F-G, Shannon RV, Soli SD. 38. Wilson BS, Finley CC, Lawson DT, Wolford RD,
Importance of tonal envelope cues in Chinese
Eddington DK, Rabinowitz WM. Better speech
speech recognition. J Acoust Soc Am 1998;104:
recognition with cochlear implants. Nature 1991;
505–510
352:236–238
30. Fu Q-J, Hsu C-J, Horng M-J. Effects of speech 39. Rosen S, Faulkner A, Wilkinson L. Adaptation by
processing strategy on Chinese tone recognition by
normal listeners to upward spectral shifts of speech:
nucleus-24 cochlear implant patients. Ear Hear
implications for cochlear implants. J Acoust Soc
2004;25:501–508
Am 1999;106(6):3629–3636
Qian-Jie Fu, Ph.D.,1and John J. GalvinIII, Ph.D.1
ABSTRACT
Learning electrically stimulated speech patterns can be a new and difficult experience for cochlear implant patients. Cochlear implantation alone may not fully meet the needs of many patients, and additional auditory rehabilitation may be necessary to maximize the benefits of the implant device. A recently developed computer-assisted speech-training program provides cochlear implant patients with the means to conduct auditory rehabilitation at home. The training software targets important acoustic contrasts between speech stimuli and provides auditory and visual feedback as well as progressive training, thereby maintaining patients’ interest in the auditory training exercises. Recent scientific studies have demonstrated the effectiveness of such specialized auditory training programs in improving cochlear implant patients’ speech recognition performance. Provided with an inexpensive and accessible auditory training program, cochlear implant patients may find the motivation and momentum to get the most from the implant device.
KEYWORDS: Auditory training, cochlear implants, computer-assisted speech training
Learning Outcomes: As a result of this activity, the participant will be able to (1) discuss the importance of auditory training for cochlear implant patients, and (2) summarize the recent development of computer-assisted speech training specially designed for cochlear implant patients.
The cochlear implant (CI) is an electronic
device that provides hearing sensation to pa-
tients with profound hearing loss. As the sci-
ence and technology of the cochlear implant has developed over the past 50 years, the overall speech recognition of CI patients has steadily
1Department of Auditory Implants and Perception, House Ear Institute, Los Angeles, California.
Address for correspondence and reprint requests: QianJie Fu, Ph.D., Department of Auditory Implants and Perception, House Ear Institute, 2100 West Third St., Los Angeles, CA 90057. E-mail: [email protected]
Auditory Training; Guest Editor, Robert W. Sweetow, Ph.D.
Semin Hear 2007;28:141–149. Copyright # 2007 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584–4662. DOI 10.1055/s-2007-973440. ISSN 0734-0451.
141
142 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
improved. With the most advanced implant technology and speech processing strategies, many patients receive great benefit, and are capable of conversing with friends and family over the telephone. However, considerable variability remains in individual patient outcomes. Some patients receive little benefit from the latest implant technology, even after many years of daily use of the device. This variability in patient outcomes is reflected not only in differences in speech recognition performance, but also in the time course of adaptation to novel speech patterns via electric hearing. Although some patients may easily and quickly adapt to their implant, others may require an extensive learning period.
Much research has been devoted to exploring the sources of variability in CI patient outcomes. Some studies have shown that patient-related factors, such as duration of deafness, are correlated with speech performance.1,2 Several psychophysical measures, including electrode discrimination,3 temporal modulation detection,4,5 and gap detection,6– 8 also have been correlated with speech performance. Furthermore, Kelly et al2 found that earlier P2 latencies of auditory evoked potentials (AEPs) were associated with shorter durations of deafness and higher speech scores. In addition, mismatch negativity was absent or degraded in CI patients with poor speech scores. The correlation between duration of deafness and speech performance suggests that early implantation may benefit deaf patients profoundly. The correlation between psychophysical measures and speech performance suggests that both spectral resolution and temporal resolution are important for speech recognition in electric hearing. The correlation between AEPs and speech performance provides objective evidence of central auditory processing differences among experienced CI users.
Besides the high variability in CI patient outcomes, individual patients also differ in terms of the time course of adaptation to electric hearing. During the initial period of use, postlingually deafened CI patients must adapt to differences between their previous experience with normal acoustic hearing and the pattern of activation produced by electrical
stimulation. Many studies have tracked changes in performance over time in na¨ıve or newly implanted CI users. These longitudinal studies showed that most gains in performance occur in the first 3 months of use.9–13 However, continued improvement has been observed over longer periods for some CI patients.14 Experienced CI users also must adapt to new electrical stimulation patterns provided by updated speech processors, speech processing strategies and/or changes to speech processor parameters. For these patients, the greatest gains in performance also occurred during the first 3 to 6 months, with little or no improvement beyond 6 months.15,16
Overall, these results suggest that considerable auditory plasticity exists in CI patients, even after years of experience with their device. Because of the spectrally degraded speech patterns provided by the implant, passive learning via long-term use of the device may not fully engage patients’ capacity to learn novel stimulation patterns. Active auditory training may better exploit CI patients’ auditory plasticity and facilitate learning of electrically stimulated speech patterns. Auditory training, an important facet of aural rehabilitation, has been shown to improve hearingimpaired patients’ speech comprehension and communication.17 Currently, fewer than 10% of practicing audiologists offer hearing-impaired patients comprehensive auditory training, compared with 16% in 1990 and 31% in 1980.18 Hearing healthcare professionals acknowledge that cochlear implantation alone may not fully meet the needs of many patients, and that additional auditory therapy may enhance the benefits of the implant device. However, for a variety of reasons, few hearing healthcare professionals routinely include auditory training in the services they provide to CI patients. Time and cost considerations often preclude providing extensive and intensive auditory therapy. Given the limited number of speech pathologists working with CI patients and the high costs associated with auditory rehabilitation, it has long been urgent to develop an inexpensive and effective auditory training system for CI patients, especially for those patients who have the greatest difficulty with speech understanding.
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 143
METHOD The computer-assisted speech training (CAST) program, developed at the House Ear Institute (Los Angeles, CA), provides CI patients with self-directed auditory rehabilitation via home computer. As an auditory training tool, CAST provides some appealing advantages over conventional, site-specific approaches for auditory rehabilitation. First, the cost of individualized auditory training is significantly lower than that with traditional auditory rehabilitation provided by hearing clinics and hospitals. Second, CAST is easily accessible by CI patients, who can practice at home at any time, provided they have access to a computer. Third, patients’ progress can be easily monitored and shared with their clinical practitioners. Finally, CAST provides auditory training that is specific to the needs of CI users, and more importantly to the needs of individual CI patients. CAST targets acoustic contrasts that are especially problematic for CI users, as opposed to targeting cognitive/developmental learning. Besides being affordable, accessible, and appropriate for CI patients, CAST has many other unique features.
Comprehensive Training Materials CAST provides a comprehensive set of training materials with which to train recognition of many kinds of sounds, not just speech. Training materials include simple pure tones, environmental sounds, monosyllabic words, consonant training stimuli (in v/C, v/C/v, and C/v contexts), familiar words, familiar sentences, simple melodic sequences and familiar melodies. For targeted phonetic contrast training, CAST uses more than 1000 novel monosyllabic words (for initial, medial and final vowel and consonant training) and nonsense words (for initial, medial and final consonant training), spoken by four different talkers. Additional training materials and/or training modules can be easily incorporated for individual patients to maximize training outcomes. For advanced users, CAST also can mimic difficult listening environments by adding background noise or competing speech in real time.
Individualized Training Protocols CAST also provides individualized training protocols for CI patients. The level of difficulty is automatically adjusted according to individual patient performance by increasing the number of response choices and/or reducing the acoustic differences between response choices. During training, both auditory and visual feedback is provided, allowing users to compare incorrect responses to correct responses repeatedly. As performance improves, the level of difficulty is automatically increased. If performance does not improve, the level of difficulty is decreased. At the end of each testing and training session, the program offers training guidance. For example, based on testing or training results, the program suggests the appropriate training level, whether users should go to an easier or more difficult training level, etc. Feedback and encouragement are offered throughout, as the positive effects of good will can help patients build momentum during the rehabilitation process.
Integrated Training Structures CAST provides easy integration for auditory rehabilitation between CI patients and their audiologists or speech pathologists. All training and test results are saved in a standardized database. Users can view results and ongoing progress and data can be exported and shared with the patient’s hearing health practitioner. Results are automatically saved and include user’s name, test and training results, test and training date, training time for each exercise, and total training time. Audiologists can use these results to track a patient’s progress over time, check for potential problems with the CI device, and optimize the speech processor. For example, during a clinical visit, the audiologist can review the test results and adjust processor parameters (e.g., frequency allocation, channel gains, etc.) to improve performance in problem areas (e.g., reception of fricatives, etc.). Over a longer term, different magnetic acoustic pressuresQ1 (MAPs with different parameter set- Q1 tings) can be loaded onto the processor and performance can be tracked over time to determine the optimal MAP; the combination of take-home experience and laboratory testing
144 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
(with easy access to results) will allow for better PHONEME RECOGNITION TRAINING IN
long-term optimization of the processor. CAST provides a visually clean and user-
COCHLEAR IMPLANT PATIENTS
Fu et al25 investigated the effects of computer-
friendly graphic interface, allowing patients to assisted speech training on the recognition
concentrate on listening rather than navigating performance of CI users. Ten adult CI patients
an unfamiliar computer program.
with limited speech recognition capabilities
participated in the study. Baseline (pretraining)
multitalker phoneme recognition performance
TRAINING OUTCOMES
was measured for at least 2 weeks, or until
Although CAST provides inexpensive and ac- performance reached an asymptote. After base-
cessible auditory rehabilitation for CI patients, line measures were complete, participants
the most important question is whether com- trained at home using the CAST program
puter-assisted auditory training can effectively (loaded onto their personal computer or loaner
improve CI patients’ speech recognition per- laptop). Participants were instructed to train at
formance, especially for patients who have great home 1 hour per day, 5 days per week for a
difficulty with speech understanding. Auditory period of 1 month or longer. Participants re-
training has been shown effective in the reha- turned to the laboratory every 2 weeks for
bilitation of children with central auditory processing disorders,19,20 children with language-learning impairment,21,22 and hearing
aid (HA) users.17 Some earlier studies have
retesting (same tests as baseline measures). Results showed that both vowel and consonant recognition significantly improved for all participants after training. Mean vowel recognition
assessed the effects of auditory training on improved from 23.7 to 39.5% correct (paired t
speech recognition by poor-performing CI patients.23,24 Busby et al23 examined the effects of
test: p < 0.0001). Mean consonant recognition improved from 25.1 to 38.6% correct
10 one-hour speech perception training ses- (p < 0.005). For a subset of CI participants,
sions (1 to 2 sessions per week); three prelin- mean sentence recognition also improved
gually deafened CI users (two adolescents and from 27.9 to 55.8% correct (p < 0.01) after
one adult) participated in the experiment. After training. Although performance significantly
the training period, there were only minimal improved for all participants after 4 weeks or
changes in speech performance; the participant more of moderate training, there was signifi-
with the greatest improvement was implanted cant inter participant variability in terms of the
at an earlier age, and therefore had a shorter amount and time course of improvement. For period of deafness. Dawson and Clark24 re- some participants, performance significantly
ported more encouraging results for vowel rec- improved after only a few hours of training,
ognition training in five CI users. Each CI user whereas others required a much longer time
had been deaf for at least 4 years prior to course.
implantation, and none had achieved open-set
speech recognition. Training consisted of one MELODIC SEQUENCE IDENTIFICATION TRAINING
50-minute training session per week for 10 IN COCHLEAR IMPLANT PATIENTS
weeks. Following training, four of the five Music perception and appreciation remains
participants showed improvement in some measures; this improvement was retained on subsequent testing three weeks after training
difficult for most CI patients. Galvin et al Q3(J.J. Galvin 3rd, Fu Q-J, and G. Nogaki, Q3 unpublished data, 2005) developed an objective
was completed. The amount and type of train- test of CI users’ music perception: melodic
ing employed may account for the mixed and sequence identification (MSI). During the
generally poor outcomes from earlier CI speech MSI test, CI patients were asked to identify Q2 training studiesQ2. Several recent studies have simple five-note melodic sequences, in which
evaluated the effects of computer-assisted audi- the pitch contour was systematically varied.
tory training (using the CAST program) on CI The interval between successive notes was sys-
patient performance, and are summarized be- tematically varied between one and five semi-
low.
tones to test the intonation provided by the CI
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 145
device. Results showed large interparticipant variability in MSI performance. The best performers correctly identified more than 90% of the melodic sequences when there were two semitones between notes in a sequence; poor performers correctly identified less than 40% of the sequences with five semitones between notes. Six CI patients with poor MSI performance were trained using the CAST software to see whether MSI performance could be improved with moderate auditory training. Participants were instructed to train for 1 hour per day, 5 days per week, for a period of 1 month or longer. For all participants, performance significantly improved with training (p ¼ 0.004); the amount of improvement ranged from 15.5 to 45.4 percentage points. Pre- and posttraining familiar melody identification (FMI) also was measured in four of the six CI participants; 12 melodies were tested without rhythm cues, similar to Kong et al.26 Results showed that after training, mean FMI performance significantly improved by 20.8 percentage points (p ¼ 0.020). Anecdotal reports suggested that CI patients’ music perception and appreciation generally improved after MSI training. For example, some participants reported that they were better able to separate the singer’s voice from the background music while listening to music in the car.
CHINESE TONE RECOGNITION TRAINING IN
MANDARIN-SPEAKING COCHLEAR IMPLANT
PATIENTS
In tonal languages such as Mandarin Chinese, the tonality of a syllable is lexically important,27,28 and Chinese sentence recognition is highly correlated with tone recognition.29,30 Although the fundamental frequency (F0) contour contributes most strongly to tone recognition,27 other temporal cues that covary with tonal patterns (e.g., amplitude contour and periodicity fluctuations) also contribute to tone recognition, especially when F0 cues are reduced or unavailable.29,31,32 J.-L. Wu, H.-M. Yang, Y.-H. Lin, and Q.-J. Fu (unpublished data, 2006) recently investigated whether moderate auditory training can improve the recognition of Chinese vowels, consonants, and tones by hearing-impaired children. Ten Mandarin-speaking children (seven CI users and
three HA users) participated in the study. Training was conducted at home using the CAST program; participants trained for one half-hour per day, 5 days per week, for a period of 10 weeks. Results showed that mean vowel recognition scores significantly improved from 63.1 to 84.8% correct (paired t test: p ¼ 0.006) after the 10-week training period. Similarly, mean consonant recognition scores significantly improved from 39.7 to 58.2% correct (p < 0.001) and Chinese tone recognition scores significantly improved from 56.0 to 71.1% correct (p ¼ 0.007). Vowel, consonant, and tone recognition performance was re-measured after training was completed; follow-up performance was measured at 1, 2, 4, and 8 weeks after training was completed. Followup measures remained significantly higher than pretraining baseline measured for vowel (p ¼ 0.018), consonant (p ¼ 0.002), and tone recognition (p ¼ 0.009), suggesting that the improved performance with training was retained well after training had stopped.
DISCUSSION The results from these studies demonstrate that moderate amounts of auditory training with the CAST program improved CI and HA users’ recognition of phonemes, melodic sequences, and Chinese tones. Although auditory training generally improved performance in the targeted listening task, the improvement sometimes generalized to auditory tasks that were not explicitly trained (i.e., improved sentence recognition after training with phonetic contrasts, or improved familiar melody identification after training with simple melodic sequences). Importantly, the improved performance was not transitory, as posttraining performance levels remained higher than pretraining baseline levels 1 to 2 months after training was completed. Because most CI participants had a minimum of one year of experience with their device prior to training (i.e., at least one year of passive learning), the results suggest that active learning was necessary to improve speech performance. The improvements in speech recognition with auditory training were comparable to (and often much greater than) gains reported with recent advances in CI technology and
146 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
speech processing. Taken together, the results suggest that computer-assisted auditory training, as implemented in the CAST program, provides an effective, inexpensive, and accessible auditory rehabilitation for CI patients.
The results from these recent training studies with the CAST program have shown successful training outcomes with CI patients; nevertheless, earlier studies have shown somewhat mixed results.23,24 The better training outcomes with the CAST program may have been due to differences in terms of training protocols. For example, one-to-one personal training that targeted a variety of phonetic contrasts was used in Dawson and Clark,24 whereas computer-assisted progressive training that targeted minimal phonetic contrasts was used in the CAST studies. Fu et al33 systematically evaluated the effects of training protocols on NH listeners’ adaptation to eightchannel spectrally shifted speech. Four different training protocols have been evaluated, including test-only (repeated baseline measures), stimulus preview, phonetic contrast training, and sentence training. Results showed a significant effect of the training protocol on the training outcomes. The targeted, progressive phonetic contrast training (which is used in the CAST program) produced the greatest improvement in recognition of spectrally shifted vowels. The improved vowel recognition also generalized to improved consonant and sentence recognition with spectrally shifted speech.
The better training outcomes with the CAST program also may also be due to differences in terms of the frequency of training, i.e., how often the training was conducted during the training period. In previous CI training studies,23,24 participants trained 50 minutes once per week, for 10 weeks; however, in the CAST studies, participants trained 30 to 60 minutes per day, 5 days per week, for a period of one month or longer. G. Nogaki, Q-J Fu, and J.J. Galvin (unpublished data, 2006) systematically evaluated the effect of training frequency on NH listeners’ adaptation to eight-channel spectrally shifted speech. The participants completed five training sessions using the phonetic contrast training protocol.25,33 Eighteen participants were divided into three groups, in which
1-hour training sessions were provided one, three, or five times per week. The results suggest that although more frequent training seemed to provide better adaptation over the five training sessions, the frequency of training did not significantly affect training outcomes (at least for the training frequencies that were studied). Thus, it may be more important to complete an adequate number of training sessions over a reasonable period. These simulation results suggest that CI patients may significantly benefit from auditory training, even when infrequently performed. The results further suggest that the difference in CI training outcomes between these recent studies and other earlier studies may not be simply due to time-intensive training, but rather due to differences in training protocols and (perhaps) training stimuli. CAST makes use of a far greater number of multitalker speech stimuli (more than 4000 stimuli) than used in previous CI speech training studies. The results suggest that it may be more important to develop effective training protocols and stimuli than to simply increase patients’ time commitment.
It is important to note that in the CAST training studies,25,26,36 including unpublished data from G. Nogaki, Q-J Fu, and J.J. Galvin (2006) and J-L Wu, H-M Yang, Y-H Lin, and Q-J Fu (2006), the stimuli used for baseline measures were not used for training. Baseline measures were conducted using standard speech test databases (for example, the vowel stimuli recorded by Hillendbrand et al16 or the consonant stimuli recorded by Shannon et al34), although training was conducted using monosyllable and/or nonsense words produced by completely different talkers. Because improved performance for a trained auditory task often generalized to improved performance in other auditory tasks (for example, phonetic contrast training improved phoneme and sentence recognition, melodic sequence training improved both sequence and familiar melody identification), the large multitalker database used in the CAST may provide some advantage over smaller, more limited databases. The generalized improvements in performance agree with previous neurophysiological studies,35 which showed that both behavioral performance and neurophysiological changes observed after
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 147
auditory training generalized to stimuli not used in the training, thus demonstrating behavioral transfer of learning and plasticity in underlying physiologic processes. Tremblay et al36 reported results further indicating that training-associated changes in neural activity may precede behavioral learning. The results in Fu et al25 also indicated that the time course of improvement varied significantly across CI patients. Depending on patient-related factors (e.g., the number of implanted electrodes, the insertion depth of the electrode array, duration of deafness, etc.), some CI patients may require much more auditory training to noticeably improve their performance. Objective neurophysiological measures may provide useful information about the progression of training, i.e., whether a particular training protocol should be continued or whether the training protocol and/or training materials should be adjusted. These objective measures may allow for the development of more efficient training protocols for CI patients; fortunately, the flexibility in computer-assisted auditory training (as in CAST) allows such changes to be easily implemented.
A future direction that may further improve CI patient training outcomes is to integrate the standardized testing and training features of the CAST program fully into the clinical fitting system. Many previous CI studies15,37,38 have shown that acutely measured performance may significantly underestimate the effects of changes in speech processing. Even a short period of adaptation may result in significantly better speech performance.33,39 Longer adaptation periods may be required to fully evaluate the effects of changes in speech processing. By integrating the test and training platforms into the clinical fitting procedures, CAST may help accelerate the adaptation process over the short and long term. Audiologists have little time available to test and retest performance with different MAPs; CAST allows training and testing with little or no supervision on the part of the audiologist. Thus, during clinical visits, MAPs may be evaluated more accurately after some amount of training and testing. In addition, during a clinical visit, CI patients largely judge the quality of a speech processor according to the audiologist’s voice; by providing multiple
talkers, CAST allows patients to better judge the quality of the MAP. Over the long term, patients can compare multiple MAPs loaded onto their speech processors in everyday listening conditions outside of the clinic (CAST training also can help in this period of adaptation), and then return to the clinic for testing. Testing results can be used to determine optimal MAPs or further adjustments to speech processors.
How a parameter change is introduced also may influence the time course and degree of adaptation. Fu et al37 found that CI patients were able to only partially adapt to an abrupt change in the acoustic frequency allocation, even after 3 months of continuous use. In a follow-up study with one of the CI participants, gradually introducing a severe shift in frequency allocation provided for a more complete adaptation, albeit over a much longer adaptation period (18 months). Thus, although passive learning (incidental learning via everyday experience with the CI device) may allow for some degree of adaptation to changes in speech processing, active auditory training may help to accelerate the adaptation process and provide adaptation that is more complete. Because it is important to evaluate speech processor settings in light of adaptation, and because auditory training may accelerate the adaptation process, test and training results from the CAST program may be used to guide gradual adjustments to speech processor settings, thereby making the adaptation more complete and less stressful in the end.
Advances in computer technology have contributed greatly to the latest-generation CI technology. These advances also have allowed for the development of important rehabilitation tools, which may ultimately provide the greatest benefit for those CI patients who have difficulty with their device. Motivation is an important factor in successful patient outcomes. With the right tools and resources for auditory rehabilitation, CI patients may find the necessary motivation and momentum to get the most from their implant.
SUMMARY AND CONCLUSIONS Computer-assisted auditory training, as implemented in the CAST program, provides an
148 SEMINARS IN HEARING/VOLUME 28, NUMBER 2 2007
economical and effective alternative or comple- 8. Muchnik C, Taitelbaum R, Tene S, Hildesheimer
ment to site-specific auditory rehabilitation. Moderate amounts of auditory training performed at home with the CAST software resulted in significant improvements in CI patients’ speech, music, and Chinese tone recognition. These improvements were retained months after training was completed. Targeted
M. Auditory temporal resolution and open speech recognition in cochlear implant recipients. Scand Audiol 1994;23:105–109 9. George CR, Cafarelli Dees D, Sheridan C, Haacke N. Preliminary findings of the new Spectra 22 speech processor with first-time cochlear implant users. Ann Otol Rhinol Laryngol Suppl 1995; 166:272–275
auditory training tasks with novel training 10. Gray RF, Quinn SJ, Court I, Vanat Z, Baguley
stimuli also generalized to improved performance in other auditory training tasks.
DM. Patient performance over eighteen months with the Ineraid intracochlear implant. Ann Otol Rhinol Laryngol Suppl 1995;166:275–277
11. Loeb GE, Kessler DK. Speech recognition per-
ABBREVIATIONS
formance over time with the Clarion cochlear prosthesis. Ann Otol Rhinol Laryngol Suppl 1995;
166:290–292
CAST computer-assisted speech training
12. Spivak LG, Waltzman SB. Performance of coch-
CI
cochlear implant
lear implant patients as a function of time. J Speech
FMI familiar melody identification
HA hearing aid
Q4
MAP magnetic acoustic pressureQ4
NH normal hearing
Hear Res 1990;33:511–519 13. Waltzman SB, Cohen NL, Shapiro WH. Long-
term effects of multichannel cochlear implant usage. Laryngoscope 1986;96(10):1083–1087 14. Tyler RS, Gantz BJ, Woodworth GG, Fryauf-
Bertschy H, Kelsay DM. Performance of 2- and
3-year-old children and prediction of 4-year from
REFERENCES
1-year performance. Am J Otol 1997;18:S157–S159
15. Dorman MF, Loizou PC. Changes in speech
1. Eggermont JJ, Ponton CW. Auditory-evoked
intelligibility as a function of time and signal
potential studies of cortical maturation in normal
processing strategy for an Inneraid patient fitted
hearing and implanted children: correlations with
with continuous interleaved sampling (CIS) pro-
changes in structure and speech perception. Acta
cessors. Ear Hear 1997;18:147–155
Otolaryngol 2003;123:249–252
16. Hillenbrand J, Getty LA, Clark MJ, Wheeler K.
2. Kelly AS, Purdy SC, Thorne PR. Electrophysio-
Acoustic characteristics of American English
logical and speech perception measures of auditory
vowels. J Acoust Soc Am 1995;97:3099–3111
processing in experienced adult cochlear implant 17. Sweetow R, Palmer CV. Efficacy of individual
users. Clin Neurophysiol 2005;116:1235–1246
auditory training in adults: a systematic review of
3. Donaldson GS, Nelson DA. Place-pitch sensitivity
the evidence. J Am Acad Audiol 2005;16:494–
and its relation to consonant recognition by
504
cochlear implant listeners using the MPEAK and 18. Bloom S. Technologic advances raise prospects for
SPEAK speech processing strategies. J Acoust Soc
a resurgence in use of auditory training. The
Am 1999;107:1645–1658
Hearing Journal 2004;57(8):19–24
4. Cazals Y, Pelizzone M, Saudan O, Boex C. Low- 19. Hesse G, Nelting M, Mohrmann B, Laubert A,
pass filtering in amplitude modulation detection
Ptok M. Intensive inpatient therapy of auditory
associated with vowel and consonant identification
processing and perceptual disorders in childhood.
in subjects with cochlear implants. J Acoust Soc
HNO 2001;49:636–641
Am 1994;96:2048–2054
20. Musiek FE, Baran JA, Schochat E. Selected
5. Fu Q-J. Temporal processing and speech recog-
management approaches to central auditory proc-
nition in cochlear implant users. Neuroreport 2002;
essing disorders. Scand Audiol Suppl 1999;51:63–
13:1635–1640
76
6. Busby PA, Clark GM. Gap detection by early- 21. Merzenich MM, Jenkins WM, Johnston P,
deafened cochlear-implant subjects. J Acoust Soc
Schreiner C, Miller SL, Tallal P. Temporal
Am 1999;105:1841–1852
processing deficits of language-learning impaired
7. Cazals Y, Pelizzone M, Kasper A, Montandon P.
children ameliorated by training. Science 1996;
Indication of a relation between speech perception
271(5245):77–81
and temporal resolution for cochlear implantees. 22. Tallal P, Miller SL, Bedi G, et al. Language
Ann Otol Rhinol Laryngol 1991;100:893–895
comprehension in language-learning impaired
AUDITORY TRAINING FOR COCHLEAR IMPLANT PATIENTS/FU,GALVIN 149
children improved with acoustically modified 31. Fu Q-J, Zeng F-G. Effects of envelope cues on
speech. Science 1996;271:81–84
Mandarin Chinese tone recognition. Asia-Pacific J
23. Busby PA, Roberts SA, Tong YC, Clark GM.
Speech Lang Hear 2000;5:45–57
Results of speech perception and speech production 32. Whalen DH, Xu Y. Information for Mandarin
training for three prelingually deaf patients using a
tones in the amplitude contour and in brief
multiple-electrode cochlear implant. Br J Audiol
segments. Phonetica 1992;49:25–47
1991;25:291–302
33. Fu Q-J, Nogaki G, Galvin JJ III. Auditory training
24. Dawson PW, Clark GM. Changes in synthetic and
with spectrally shifted speech: an implication for
natural vowel perception after specific training for
cochlear implant users’ auditory rehabilitation.
congenitally deafened patients using a multichan-
J Assoc Res Otolaryngol 2005;6:180–189
nel cochlear implant. Ear Hear 1997;18:488–501 34. Shannon RV, Jensvold A, Padilla M, Robert ME,
25. Fu QJ, Galvin JJ 3rd, Wang X, Nogaki G.
Wang X. Consonant recordings for speech testing.
Moderate auditory training can improve speech
J Acoust Soc Am 1999;106:L71–L74
performance of adult cochlear implant users. 35. Tremblay K, Kraus N, Carrell TD, McGee T.
J Acoust Soc Am 2005;113:1065–1072
Central auditory system plasticity: generalization to
26. Kong YY, Cruz R, Jones JA, Zeng F-G. Music
novel stimuli following listening training. J Acoust
perception with temporal cues in acoustic and
Soc Am 1997;102:3762–3773
electric hearing. Ear Hear 2004;25:173–185
36. Tremblay K, Kraus N, McGee T. The time course
27. Lin M-C. The acoustic characteristics and percep-
of auditory perceptual learning: neurophysiological
tual cues of tones in Standard Chinese. Chinese
changes during speech-sound training. Neuroreport
Q5
Yuwen 1988;204:182–193Q5
1998;9:3557–3560
28. Wang R-H. Chinese phonetics. In: Chen Y-B, 37. Fu QJ, Shannon RV, Galvin JJ. Perceptual learning
Wang R-H, eds. Speech Signal Processing. Hefei,
following changes in the frequency-to-electrode
China: University of Science and Technology of
assignment with the Nucleus-22 cochlear implant.
China Press; 1989:37–64
J Acoust Soc Am 2002;112(4):1664–1674
29. Fu Q-J, Zeng F-G, Shannon RV, Soli SD. 38. Wilson BS, Finley CC, Lawson DT, Wolford RD,
Importance of tonal envelope cues in Chinese
Eddington DK, Rabinowitz WM. Better speech
speech recognition. J Acoust Soc Am 1998;104:
recognition with cochlear implants. Nature 1991;
505–510
352:236–238
30. Fu Q-J, Hsu C-J, Horng M-J. Effects of speech 39. Rosen S, Faulkner A, Wilkinson L. Adaptation by
processing strategy on Chinese tone recognition by
normal listeners to upward spectral shifts of speech:
nucleus-24 cochlear implant patients. Ear Hear
implications for cochlear implants. J Acoust Soc
2004;25:501–508
Am 1999;106(6):3629–3636