Birdsong and Natural Language Group
uzh-wcms-publications.subpageListDialog.labelUnterseiten
We are interested in the neurobiological algorithms associated with vocal learning, especially of birdsong. To research birdsong learning experimentally, we longitudinally observe groups of zebra finches using custom multimodal recording arenas encompassing microphones, video cameras and wireless animal-borne sensor devices. To process the enormous datasets from months-long recordings of large animal groups, we design deep-network based approach in collaboration with the Swiss Data Science Center. We examine the inferred song learning strategies using reinforcement learning theory, which is a modeling framework that is well matched to the organization of the avian brain. Overall, the premise for our work is that animal behavior provides clues about natural intelligence, i.e., the algorithms for solving a problem.
The analysis of animal vocalizations currently relies, directly or indirectly, on human annotation of vocal segments. This processing step is the cornerstone of vocal communication research, to tease apart the vocal signal from noise is a prerequisite of any scientific insight on the structure and meaning of vocal signals. Human judgment still forms the gold standard for distinguishing vocal activity of an individual from the many other sounds in animals’ habitats, since it is virtually impossible to solve this important task without invasive measurements.
To promote vocal annotation efforts, we aggregate datasets of animal vocalizations into a massive database of vocal signals. This effort is directed by the NCCR Evolving Language. Our VocallBase https://vocallbase.evolvinglanguage.ch/ extends across 10,000 species and contains vocalizations that are carefully annotated with the help of domain experts. By adhering to the most stringent annotation standards, we ensure that comparative research is not biased by task variability and cultural differences across research fields.
We plan to integrate VocallBase with our custom web application for human-in-the-loop training of voice activity networks like WhisperSeg that we optimize to detect voice activity across thousands of species. These efforts will enhance species monitoring, biodiversity assessment, ecological research, and will enable proactive strategies to preserve endangered species and habitats.
We also explore the relevance of our biological insights for natural language processing (NLP). We want to explore possible parallels between human language and birdsong and test the relevance of evolutionary vocal learning mechanisms for the processing of human language. We direct our NLP outreach efforts to helping researchers in assimilating the scientific literature and in generating scientific arguments. We deploy our NLP tools in user-friendly web application for assisted scientific writing, see https://endoc.ethz.ch.
We use our mixed educational backgrounds to advance knowledge at the boundary between biology and engineering, encompassing natural behavior, the organization of the songbird brain, computational theories of vocal learning, the evolution of language, and new methods for language processing.
