Paul Smolensky (he/him)
Krieger-Eisenhower Professor
Contact Information
- [email protected]
- On Leave: Summer & Fall 2024
- Krieger 241B | Krieger 234A (lab)
- Personal Website
- Google Scholar Profile
Research Interests: Universal grammar in Optimality Theory, integration of connectionist ('neural') and symbolic computation: computational, linguistic, and philosophical issues
Education: PhD, Indiana University
My research (see Research tab) focuses on integrating symbolic and neural network computation for modeling reasoning and, especially, grammar in the human mind/brain. The work is formal and computational, with emerging applications to neuroscience and applied natural language processing. My research has primarily addressed issues of representation and processing rather than learning. Principal contributions (see Publications tab) are to linguistic theory, the theory of vectorial neural network computation, and the philosophical foundations of cognitive science.
During Fall semesters I am on leave from Johns Hopkins, working at Microsoft Research in Redmond, Washington (for a non-technical synopsis of some of my recent work there, see this link: Mind/Brain Networks). Prior to joining the faculty of the Cognitive Science Department at Johns Hopkins, I was a professor in the Computer Science Department and Institute of Cognitive Science at the University of Colorado Boulder. I had been a postdoc at the Center for Cognitive Science at the University of California at San Diego, where I was a founding member of the Parallel Distributed Processing Research Group and worked with Dave Rumelhart, James McClelland and Geoff Hinton. (I also contributed to the User-Centered System Design group led by Don Norman.) My degrees are an A.B. in Physics from Harvard and, from Indiana University, Bloomington, a M.S. in Physics and a Ph.D. in Mathematical Physics.
Goal
Unification of the sciences of mind & brain through integration of
- compositional, structured, symbolic computation
- at the core of many successful classical theories of the mind
- in particular, the theory of language
- a branch of discrete mathematics
- at the core of many successful classical theories of the mind
- dynamic, distributed, vectorial connectionist computation
- at the core of the theory of neural networks, crucial for
- computational models of the brain
- emergentist models of the mind
- contemporary machine learning and Artificial Intelligence
- a branch of continuous mathematics
- at the core of the theory of neural networks, crucial for
Current
The theory, and application to language, of Gradient Symbolic Computation, a new cognitive architecture in which a single computational system can simultaneously be described formally at two levels:
- a higher ‘abstract mental’ level, where
- data
- consist of symbols that have partial degrees of presence — gradient activity levels
- which blend together to form Gradient Symbol Structures (such as gradient trees)
- processing
- is algebraic operations on vectors and tensors
- data
- a lower ‘abstract neural’ level, where
- data
- consist of distributed activation vectors over many model neurons
- which superimpose to implement Gradient Symbol Structures…
- processing
- is probabilistic spreading of activation (governed by stochastic differential equations)
- through networks with numerically weighted interconnections
- data
- AS.050.326/626 Foundations of Cognitive Science
- AS.050.372/672 Foundations of Neural Network Theory
- AS.050.829 Research Seminar on Formal Theory in Cognitive Science
- AS.050.860 Professional Seminar in Cognitive Science
Proceedings of the Connectionist Models Summer School 1993
co-editor
Lawrence Erlbaum Publishers ,
1993
Below is a list of my primary and secondary PhD student advisees since 1995. To view a complete list of my department’s PhD alumni, please visit our Alumni Placement webpage.
edited 10/2024
Primary Advisor
| Name | Current Position | Dissertation. Graduating Year. |
|---|---|---|
| Tom McCoy co-advisor: T. Linzen | Assistant Professor, Dept of Linguistics, Yale University | Implicit compositional structure in artificial neural networks. 2022. |
| Najoung Kim co-advisors: K. Rawlins, T. Linzen | Assistant Professor, Depts of Linguistics & Computer Science, Boston University | Compositional Linguistic Generalization in Artificial Neural Networks. 2021. |
| Matthias Lalisse | Independent Consultant | Structure assembly in knowledge base representation. 2021. |
| Deepti Ramadoss co-advisor: L. Burzio | Director for Training, Assessment and Career Exploration, University of Pittsburgh | The phonology and phonetics of tone perception. 2011. |
| Jennifer Culbertson co-advisor: G. Legendre | Professor & Director of the Centre for Language Evolution, Dept of Linguistics & English Language, Univ. of Edinburgh | Learning biases, regularization, and the emergency of typological universals in syntax. 2010. |
| Rebecca Morley | Associate Professor, Department of Linguistics, Ohio State University | Generalization, Lexical Statistics and Typologically Rare Systems. 2008. |
| Sara Finley | Associate Professor, Department of Psychology, Pacific Lutheran University | Formal and Cognitive Restrictions on Vowel Harmony. 2008. |
| Gaja Jarosz | Professor, Department of Linguistics, UMass at Amherst | Rich Lexicons and Restrictive Grammars – Maximum Likelihood Learning in Optimality Theory. 2006. |
| Adam Buchwald co-advisor: B. Rapp | Professor, Department of Communicative Sciences & Disorders, NYU | Sound structure representation, repair and well-formedness: Grammar in spoken language production. 2005. |
| Lisa Hsin Davidson | Professor, Department of Linguistics, NYU | The atoms of phonological representation: Gestures, coordination and perceptual features in consonant cluster phonotactics. 2003. |
| John Hale | Professor, Department of Cognitive Science, JHU | Grammar, uncertainty, and sentence processing. 2003. |
| Bruce Tesar | Professor, Department of Linguistics, Rutgers | Computational Optimality Theory. 1995. Computer Science, Univ of Colorado |
Secondary Advisor
| Name | Current Position | Dissertation. Graduation Year. |
|---|---|---|
| Tamara Nicol Medina primary advisor: B. Landau | Associate Teaching Professor, Department of Psychology, University of Delaware | Learning which verbs allow object omission: Verb semantic selectivity and the implicit object construction. 2007. |
| Matthew Goldrick primary advisor: B. Rapp | Professor, Department of Linguistics, Northwestern University | Patterns in sound, patterns in mind: Phonological regularities in speech production. 2002. |
| Colin Wilson primary advisor: L. Burzio | Professor, Department of Cognitive Science, JHU | Patterns in sound, patterns in mind: Phonological regularities in speech production. 2002. |
| Adamantios Gafos primary advisor: L. Burzio | Professor, Department of Linguistics, Postdam University | The Articulatory Basis of Locality in Phonology. 1996. |
What kind of computation is human cognition? A brief history of thought
July 28, 2020, Microsoft Research Talks
Panel Discussion on Context and Compositionality in Biological and Artificial Neural Systems
December 14, 2019, Vancouver; NeurIPS 2019 Workshop
Vertical integration of neural and symbolic computation: Theory and application
January 5, 2018, Salt Lake City; Plenary lecture, inaugural meeting of the Society for Computation in Linguistics and symposium on Perceptrons & Syntactic Structures at 60
Neural networks and linguistic cognition
January 5, 2018, Salt Lake City; Discussion session, Society for Computation in Linguistics and symposium
Discussion session featuring an all-JHU-Cogsci cast (left-to-right): Paul Smolensky (faculty), Matt Goldrick (PhD 2002), Tom McCoy (PhD student), Tal Linzen (faculty), Pyeong Whan Cho (postdoc)
Grammatical theory with Gradient Symbol Structures
January 12, 2016, Budapest; Research Institute for Linguistics, Hungarian Academy of Sciences
Four facts about Tensor Product Representations
December 12, 2015, Montreal; NIPS workshop Cognitive Computation: Integrating Neural and Symbolic Approaches
Gradient Symbols in Grammar
October 26, 2015; Mind, Technology and Society Talk Series, Cognitive and Information Sciences Department, University of California − Merced
Towards Understandable Neural Networks for High Level AI Tasks (short course on Tensor Product Representations)
Fall, 2015; Microsoft Research Talks
Does the success of deep neural network language processing mean — finally — the end of theoretical linguistics?
July 31, 2015, Beijing; Invited talk, with Jennifer Culbertson. CoNLL (Conference on Computational Natural Language Learning; SIGNLL of ACL)
Symbolic roles in vectorial computation
July 14, 2014, Redmond WA; Microsoft Research Faculty Summit panel, Deep Learning for Text Processing