SaLSa: a combinatory approach of semi-automatic labeling and long short-term memory to classify behavioral syllables

Research output: Contribution to journalArticlepeer-review

11 Downloads (Pure)


Accurately and quantitatively describing mouse behavior is an important area. Although advances in machine learning have made it possible to track their behaviors accurately, reliable classification of behavioral sequences or syllables remains a challenge. In this study, we present a novel machine learning approach, called SaLSa (a combination of semi-automatic labeling and long short-term memory-based classification), to classify behavioral syllables of mice exploring an open field. This approach consists of two major steps. First, after tracking multiple body parts, spatial and temporal features of their egocentric coordinates are extracted. A fully automated unsupervised process identifies candidates for behavioral syllables, followed by manual labeling of behavioral syllables using a graphical user interface (GUI). Second, a long short-term memory (LSTM) classifier is trained with the labeled data. We found that the classification performance was marked over 97%. It provides a performance equivalent to a state-of-the-art model while classifying some of the syllables. We applied this approach to examine how hyperactivity in a mouse model of Alzheimer’s disease develops with age. When the proportion of each behavioral syllable was compared between genotypes and sexes, we found that the characteristic hyperlocomotion of female Alzheimer’s disease mice emerges between four and eight months. In contrast, age-related reduction in rearing is common regardless of genotype and sex. Overall, SaLSa enables detailed characterization of mouse behavior.

Original languageEnglish
Pages (from-to)1-10
Number of pages10
Issue number12
Early online date21 Nov 2023
Publication statusPublished - 1 Dec 2023


  • Alzheimer’s disease
  • classification
  • computational ethology
  • deep learning
  • long short-term memory


Dive into the research topics of 'SaLSa: a combinatory approach of semi-automatic labeling and long short-term memory to classify behavioral syllables'. Together they form a unique fingerprint.

Cite this