Cortex
Volume 47, Issue 4 , Pages 416-431 , April 2011

Unification of sentence processing via ear and eye: An fMRI study

  • David Braze

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • Corresponding Author InformationCorresponding author. Haskins Laboratories, 300 George Street, STE 900, New Haven, Connecticut 06511, USA.
    • ,
  • W. Einar Mencl

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • ,
  • Whitney Tabor

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • Department of Psychology, University of Connecticut, Storrs, USA
    • ,
  • Kenneth R. Pugh

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • Department of Pediatrics, Yale University School of Medicine, New Haven, Connecticut, USA
    • ,
  • R. Todd Constable

      Affiliations

    • Department of Diagnostic Radiology, Yale University School of Medicine, New Haven, Connecticut, USA
    • ,
  • Robert K. Fulbright

      Affiliations

    • Department of Diagnostic Radiology, Yale University School of Medicine, New Haven, Connecticut, USA
    • ,
  • James S. Magnuson

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • Department of Psychology, University of Connecticut, Storrs, USA
    • ,
  • Julie A. Van Dyke

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • ,
  • Donald P. Shankweiler

      Affiliations

    • Haskins Laboratories, New Haven, Connecticut, USA
    • Department of Psychology, University of Connecticut, Storrs, USA

Received 16 September 2008 ,Revised 15 June 2009 ,Accepted 10 October 2009.

References 

  1. Ashby J. Prosody in skilled silent reading: Evidence from eye movements. Journal of Research in Reading. 2006;29:318–333
  2. Binder JR, Frost JA, Hammeke TA, Cox RW, Rao SM, Prieto T. Human brain language areas identified by functional magnetic resonance imaging. Journal of Neuroscience. 1997;17:353–362
  3. Bookheimer SY, Zeffiro IA, Blaxton T, Gaillard W, Theodore W. Regional cerebral blood flow during object naming and word reading. Human Brain Mapping. 1995;3:93–106
  4. Bookheimer S. Functional MRI of language: New approaches to understanding the cortical organization of semantic processing. Annual Review of Neuroscience. 2002;25:151–188
  5. Booth JR, Burman DD, Meyer JR, Gitelman DR, Parrish TB, Mesulam MM. Modality independence of word comprehension. Human Brain Mapping. 2002;16:251–261
  6. Braze D, Tabor W, Shankweiler DP, Mencl WE. Speaking up for vocabulary: Reading skill differences in young adults. Journal of Learning Disabilities. 2007;40:226–243
  7. Calvert GA, Campbell R. Reading speech from still and moving faces: The neural substrates of visible speech. Journal of Cognitive Neuroscience. 2003;15:57–70
  8. Caplan D. Functional neuroimaging studies of written sentence comprehension. Scientific Studies of Reading. 2004;8:225–240
  9. Carpentier A, Pugh KR, Westerveld M, Studholme C, Skrinjar O, Thompson JL, et al. Functional MRI of language processing: Dependence on input modality and temporal lobe epilepsy. Epilepsia. 2001;42:1241–1254
  10. Chee MWL, O'craven KM, Bergida R, Rosen BR, Savoy RL. Auditory and visual word processing studied with fMRI. Human Brain Mapping. 1999;7:15–28
  11. Cohen L, Dehaene S. Specialization within the ventral stream: The case for the visual word form area. NeuroImage. 2004;22:466–476
  12. Cohen JD, MacWhinney B, Flatt M, Provost J. PsyScope: An interactive graphic system for designing and controlling experiments in the psychology laboratory using Macintosh computers. Behavior Research Methods, Instruments, & Computers. 1993;25:257–271
  13. Constable RT, Pugh KR, Berroya E, Mencl WE, Westerveld M, Ni W, et al. Sentence complexity and input modality effects in sentence comprehension: An fMRI study. NeuroImage. 2004;22:11–21
  14. Cooke A, Grossman M, Devita C, Gonzalez-Atavales J, Moore P, Chen W, et al. Large-scale neural network for sentence processing. Brain and Language. 2006;96:14–36
  15. Crinion JT, Lambon-Ralph MA, Warburton EA, Howard D, Wise RJS. Temporal lobe regions engaged during normal speech comprehension. Brain. 2003;126:1193–1201
  16. Dapretto M, Bookheimer SY. Form and content: Dissociating syntax and semantics in sentence comprehension. Neuron. 1999;24:427–432[see comments]
  17. Fodor JD, Ni W, Crain S, Shankweiler DP. Tasks and timing in the perception of linguistic anomaly. Journal of Psycholinguistic Research. 1996;25:25–57
  18. Friederici AD. The time course of syntactic activation during language processing: A model based on neuropsychological and neurophysiological data. Brain and Language. 1995;50:259–281
  19. Friederici AD, Alter K. Lateralization of auditory language functions: A dynamic dual pathway model. Brain and Language. 2004;89:267–276
  20. Friederici AD, Pfeifer E, Hahne A. Event related brain potentials during natural speech processing: Effects of semantic, morphological and syntactic violations. Cognitive Brain Research. 1993;1:183–192
  21. Friederici AD, Ruschemeyer SA, Hahne A, Fiebach CJ. The role of left inferior frontal and superior temporal cortex in sentence comprehension: Localizing syntactic and semantic processes. Cerebral Cortex. 2003;13:170–177
  22. Friston K, Holmes A, Worsley K, Proline JB, Frith C, Frackowiak RSJ. Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping. 1995;2:189–210
  23. Friston KJ, Holmes AP, Price CJ, Büchel C, Worsley KJ. Multisubject fMRI studies and conjunction analyses. NeuroImage. 1999;10:385–396
  24. Frost SJ, Mencl EW, Sandak R, Moore DL, Rueckl JG, Katz L, et al. A functional magnetic resonance imaging study of the tradeoff between semantics and phonology in reading aloud. NeuroReport. 2005;16:621–624
  25. Gabrielli JDE. Dyslexia: A new synergy between education and cognitive neuroscience. Science. 2009;325:280–283
  26. Genovese CR, Lazar NA, Nichols TE. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage. 2002;15:870–878
  27. Geschwind N. Disconnection syndromes in animals and man. Brain. 1965;88:237–294
  28. Gibson E. Linguistic complexity: Locality of syntactic dependencies. Cognition. 1998;68:1–76
  29. Hadjikhani N, Roland PE. Cross-modal transfer of information between the tactile and the visual representations in the human brain: A positron emission tomographic study. Journal of Neuroscience. 1998;18:1072–1084
  30. Hagoort P. On Broca, brain, and binding: A new framework. Trends in Cognitive Sciences. 2005;9:416–423
  31. Hinton G, Shallice T. Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review. 1991;98:74–95
  32. Homae F, Hashimoto R, Nakajima K, Miyashita Y, Sakai KL. From perception to sentence comprehension: The convergence of auditory and visual information of language in the left inferior frontal cortex. NeuroImage. 2002;16:883–900
  33. Howard D, Patterson K, Wise RJS, Brown WD, Friston K, Weiller C, et al. The cortical localization of the lexicons – positron emission tomography evidence. Brain. 1992;115:1769–1782
  34. Indefrey P. Neurobiology of syntax. In Hogan PC (Ed), Cambridge Encyclopedia of the Language Sciences. Cambridge, England: Cambridge University Press, 2010.
  35. Jobard G, Vigneau M, Mazoyer B, Tzourio-Mazoyer N. Impact of modality and linguistic complexity during reading and listening tasks. NeuroImage. 2007;34:784–800
  36. Kaan E, Harris A, Gibson E, Holcomb P. The P600 as an index of syntactic integration difficulty. Language and Cognitive Processes. 2000;15:159–201
  37. King JW, Kutas M. Who did what and when? Using word- and clause-level ERPs to monitor working memory usage in reading. Journal of Cognitive Neuroscience. 1995;7:376–395
  38. Kirk RE. Experimental Design: Procedures for the Social Sciences. Belmont, CA: Wadsworth; 1982;
  39. Kluender RE, Kutas M. Bridging the gap: Evidence from ERPs on the processing of unbounded dependencies. Journal of Cognitive Neuroscience. 1993;5:196–214
  40. Ledberg A, O'sullivan B, Kinomura S, Roland P. Somatosensory activations of the parietal operculum of man. A PET study. European Journal of Neuroscience. 1995;7:1934–1941
  41. Lewis R, Vasishth S. An activation-based model of sentence processing as skilled memory. Cognitive Science. 2005;29:375–419
  42. Lindenberg R, Scheef L. Supramodal language comprehension: Role of the left temporal lobe for listening and reading. Neuropsychologia. 2007;45:2407–2415
  43. Love T, Haist F, Nicol J, Swinney D. A functional neuroimaging investigation of the roles of structural complexity and task demand during auditory sentence processing. Cortex. 2006;42:577–590
  44. Markwardt FC. Peabody Individual Achievement Test–Revised. Circle Pines, MN: American Guidance Service, Inc.; 1998;
  45. Matlab. Natick, MA: The MathWorks Inc.; 2001;
  46. Mesulam MM. From sensation to cognition. Brain. 1998;121:1013–1052
  47. Meyer M, Friederici AD, von Cramon DY. Neurocognition of auditory sentence comprehension: Event related fMRI reveals sensitivity to syntactic violations and task demands. Cognitive Brain Research. 2000;9:19–33
  48. Michael EB, Keller TA, Carpenter PA, Just MA. fMRI investigation of sentence comprehension by eye and by ear: Modality fingerprints on cognitive processes. Human Brain Mapping. 2001;13:239–252
  49. Mitchell TM, Shinkareva SV, Carlson A, Chang K, Malave V, Mason R, et al. Predicting human brain activity associated with the meanings of nouns. Science. 2008;320:1191–1195
  50. Murphy K, Corfield DR, Guz A, Fink GR, Wise RJS, Harrison J, et al. Cerebral areas associated with motor control of speech in humans. Journal of Applied Physiology. 1997;83:1438–1447
  51. Ni W, Constable T, Mencl WE, Pugh KR, Fulbright RK, Shaywitz SE, et al. An event-related neuroimaging study: Distinguishing form and content in sentence processing. Journal of Cognitive Neuroscience. 2000;12:120–133
  52. Papademetris X, Jackowski AP, Schultz RT, Staib LH, and Duncan JS. Integrated intensity and point-featured nonrigid registration. Paper presented at the MICCAI, 2004.
  53. Pearlmutter NJ, MacDonald MC. Individual differences and probabilistic constraints in syntactic ambiguity resolution. Journal of Memory and Language. 1995;34:521–542
  54. Poldrack RA, Wagner AD, Prull MW, Desmond JE, Glover GH, Gabrieli JD. Functional specialization for semantic and phonological processing in the left inferior prefrontal cortex. NeuroImage. 1999;10:15–35
  55. Poremba A, Saunders RC, Crane AM, Cook M, Sokoloff K, Miskin M. Functional mapping of the primate auditory system. Science. 2003;299:568–572
  56. Pugh KR, Mencl WE, Jenner AR, Katz L, Frost SJ, Lee JR, et al. Functional neuroimaging studies of reading and reading disability (developmental dyslexia). Mental Retardation & Developmental Disabilities Research Reviews. 2000;6:207–213
  57. Pugh KR, Mencl WE, Shaywitz BA, Shaywitz SE, Fulbright RK, Constable RT, et al. The angular gyrus in developmental dyslexia: Task-specific differences in functional connectivity within posterior cortex. Psychological Science. 2000;11:51–56
  58. Rumsey JM, Horowitz B, Donohue C, Nace K, Maisog JM, Andreason P. Phonologic and orthographic components of word recognition: A PET-rCbf study. Brain. 1997;120:739–759
  59. Seltzer B, Pandya DN. Parietal, temporal and occipital projections to cortex of thesuperior temporal sulcus in the Rhesus monkey. Journal of Comparative Neurology. 1994;343:445–463
  60. Shankweiler DP, Mencl WE, Braze D, Tabor W, Pugh KR, Fulbright RK. Reading differences and brain: Cortical integration of speech and print in sentence processing varies with reader skill. Developmental Neuropsychology. 2008;33:745–776
  61. Shaywitz BA, Shaywitz SE, Pugh KR, Fulbright RK, Skudlarski P, Mencl WE, et al. The functional neural architecture of components of attention in language-processing tasks. NeuroImage. 2001;13:601–612
  62. Spitsyna G, Warren JE, Scott SK, Turkheimer FE, Wise RJS. Converging language streams in the human temporal lobe. Journal of Neuroscience. 2006;26:7328–7336
  63. Stromswold K, Caplan D, Alpert N, Rauch S. Localization of syntactic comprehension by positron emission tomography. Brain and Language. 1996;52:452–473
  64. The Psychological Corporation . Wechsler Abbreviated Scale of Intelligence. San Antonio: Harcourt Brace & Co; 1999;
  65. van der Velde F, de Kamps M. Neural blackboard architectures of combinatorial structures. Behavioral and Brain Sciences. 2006;29:37–108
  66. Van Dyke JA, Lewis R. Distinguishing effects of structure and decay on attachment and repair: A cue-based parsing account of recovery from misanalyzed ambiguities. Journal of Memory and Language. 2003;49:285–316
  67. Vigneau M, Beaucousin V, Herve PY, Duffau H, Crivello F, Houde O, et al. Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing. NeuroImage. 2006;30:1414–1432
  68. Wiederholt JL, Bryant BR. Gray Oral Reading Test (GORT). 4th ed.. Austin, TX: Pre-Ed; 2001;
  69. Zatorre RJ, Meyer E, Gjedde A, Evans AC. PET studies of phonetic processing of speech: Review, replication, and reanalysis. Cerebral Cortex. 1996;6:21–30

PII: S0010-9452(09)00323-2

doi: 10.1016/j.cortex.2009.11.005

Cortex
Volume 47, Issue 4 , Pages 416-431 , April 2011

Article Tools

* Image Usage & Resolution