Deep Phenotypic Profiling of Body Movements and Social Interactions

Abstract

Behavior is primarily conveyed by movement, and thus a basic understanding of the biology of behavior hinges on the ability to measure how animals move. Movement is also intimately linked to brain health and provides a window into disorders of the nervous system. This talk will cover our work to build machine learning approaches for 3D movement quantification in individuals and social groups and apply these approaches to improve the sensitivity of drug and disease phenotyping. We recently developed a technique for high-resolution 3D tracking of postural dynamics and social touch in videos of freely interacting animals, with which we identified a rich landscape of stereotyped actions, interactions, synchrony, and body contacts in mouse and rat dyads. This high-resolution phenotyping revealed a spectrum of changes across seven rat monogenic autism models and in response to amphetamine not resolved by conventional measurements (Klibaite, Li, et al. Cell 2025). We have also applied our approach in the clinic to automate Parkinson's disease (PD) diagnosis (Kim et al. CVPR 2024) and establish patient-specific symptom fingerprints that are predictive of rates of decline in mobility. Finally, we will present how Dannce.ai, inc is working to deploy a commercial system supporting precise video-based clinical PD assessment and analytics at scale.