FUSA

This proposal focuess on the modelling of the acquisition of language, ontology, culture and knowledge by babies/children. The project will collect audio-visual and biometric (EEG etc.) data from mother and baby during their interactions. This data will be used to develop self-organizing models that are neuropsychologically realistic and will be used to explore the ability of a child to integrate multimodal sensory-motor information to identify and bind percepts relating to objects and events. Conversely the data will be used to validate the predictions of current and proposed models of the child’s learning processes.

The context for the project is ongoing work at the Flinders University of South Australia that has been undertaken from a Computational Linguistics and Cognitive Psycholinguistics perspective in which unsupervised and multimodally supervised learning is being undertaken using real and simulated robots, and in which preconscious correlates of language, learning, skill acquisition, stress and fatigue have been studied using EEG and fMRI. Previous work has predicted the importance of closed-class or functional words in the language learning processes, and some experimental results tend to confirm this. Unimodal unsupervised learning processes can produce grammar-like structures representing objects (simple noun phrases) or events (simple verb phrases/clauses) which in some cases confirm and in other cases contrast with conventional linguistic theory. Bimodal supervised and unsupervised learning processes have learned linguistically reasonable semantic structures for nouns, ve! rbs, prepositions, etc. However, there is a considerable gap between this work and current neurological proposals for recurrent spatio-temporal neural representations and the empirical association of synchrony with binding.

Data stream

The primary aim is to explore what information is available to a child from at least three months prior to birth (sub-400Hz audio + biometrics) to at least 24 months (audiovisual + biometrics), how the child responds to/learns from that information, and how that real data can be combined in biologically plausible self-organizing neural networks, and whether the biometrics and audio-visual information is consistent with the models and predictions of the various existing and proposed theories and models. Ideally studies will be carried out on a broad enough population to allow comparison of normal and abnormal language and learning processes, as well as early detection and increased potential for rectification of such deficiencies.

Modelling streams

A low level modelling stream aims to realize a low level neural/spreading activation model in which percepts are able to interact, synchrony is predicted to emerge as a consequence of binding rather than a cause of binding, and objects and events are predicted to bind neurons in diverse parts of the simulated cortex and hippocampus. A high level modelling stream aims to demonstrate that the availability of multimodal input allows higher level phonological, semantic and syntactic structures to be learned without overt supervision compared with what is learned with self-organizing networks and unsupervised algorithms within a single modality, and seeks to verify or refute the predictions of the current range of linguistic theory.

Type of Collaboration

The data and models are in both cases intended to be relevant to paediatrics, midwifery, education, speech pathology, audiology etc. and to lead to the development of technologies that will assist in diagnosis and treatment of a variety of language, learning and attention deficits and disorders. Projects will need to have CIs covering engineering, neuroscience, paediatric, learning, language, speech and vision aspects of the project. We have largely computer science, cognitive science and neuroscience expertise in the project at this time and are particularly interested in collaboration from paediatricians, audiologists, cognitive/perceptual psychologists and psycholinguists.

If you are interested in being part of this please contact:

David Powers
powers@ieee.org
0414 824307

A paper, the expression of interest submitted to the ARC re Thinking Systems, and further are available.