Characterizing continual learning scenarios and strategies for audio analysis

Abstract

Abstract Audio analysis is used in many real-world applications, but most methods assume a fixed data distribution. In practice, data can shift over time due to changes in distribution or the appearance of new classes, reducing model performance. Traditional deep learning models fail to adapt, making continual learning (CL) crucial. While some studies explore CL for audio, they mainly focus on a class-incremental scenario and largely ignore a domain-incremental scenario. In applications such as anomaly detection and monitoring, domain incremental scenarios are prevalent. To the best of our knowledge, no existing comprehensive benchmark for audio data supports both domain-incremental and class-incremental learning scenarios. To bridge this gap, we have curated a benchmark using the DCASE dataset (2020–2023) that properly considers both scenarios. Previous works have primarily explored anomaly detection approaches where new classes are treated as anomalies. Our work complements these approaches by examining anomaly detection within established classes, which is crucial for audio applications involving rare or unexpected sounds within known classes. We use the proposed dataset to compare regularization-based (EWC, LwF, and SI) and rehearsal-based (GEM, A-GEM, GDumb, Replay, DER++, and Co 2 L) CL approaches, as well as non-CL approaches (Naive, Cumulative, and Joint training). We evaluate multiple metrics such as accuracy, forward transfer, and backward transfer using two popular backbones: ResNet50 and ViT. The key findings show that Replay outperforms other CL methods, achieving up to 76.53% (domain) and 99.51% (class) accuracy with ViT, and 70.12% (domain) and 96.98% (class) accuracy with ResNet50. The study is very beneficial for researchers and practitioners working in the area of audio analysis for developing adaptive models.