layout: page subheadline: “” sidebar: right title: “Interspeech 2024” teaser: “” header: image_fullwidth: “whale.png” categories:

• blog tags:
• signal processing, speech enhancement, bioacoustics

This was my first Interspeech in person, during the post-pandemic one in Seul I decided not to travel and chaired a session remotely. This is a large conference with multiple parallel sessions, lots of attendees.

This is a list of interesting papers that I have seen at Interspeech 2024 in Kos, Greece. Note that this is not a comprehensive list nor is a ranking of the best paper in the programme. The schedule was quite packed so I had to skip many sessions that overlapped to prioritize topics that I currently work on, i.e. bioacoustics.

To sum up there is a lot of interest in self-supervised models with another benchmark to evaluate these models on, SUPERB 2.0. There are countless papers using these models or even powerful supervised models (Whisper) on various downstream tasks.

There is also an increasing interest in using neural codecs/tokens rather than spectrograms, waveforms, embeddings from other models. This is reflected in a new challenge The Interspeech 2024 Challenge on Speech Processing Using Discrete Units and several very good papers on this topic.

Although bioacoustics was a special theme this year I have not seen many papers on this topic. This was surprising but I guess it was good for the VIHAR workshop ESP organized as a satellite event of Interspeech.

Detection and classification of bioacoustics signals

• Vision Transformer Segmentation for Visual Bird Sound Denoising -> laboriously annotated dataset of spectrogram masks that allows to train a vision transformer for denoising
• DB3V: A Dialect Dominated Dataset of Bird Vocalisation for Cross-corpus Bird Species Recognition -> there was a discussion at the end on what are dialects and whether the geographical stratification of this dataset represents dialects accurately
• Investigating self-supervised speech models’ ability to classify animal vocalizations: The case of gibbon’s vocal signatures
• ANIMAL-CLEAN – A Deep Denoising Toolkit for Animal-Independent Signal Enhancement -> train a noisy2noisy model for each dataset, an extension of orca clean on other datasets
• Bird Whisperer: Leveraging Large Pre-trained Acoustic Model for Bird Call Classification

Generative speech enhancement

• Universal Score-based Speech Enhancement with High Content Preservation

Speech benchmarks:

• ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets -> included new metrics such as macro-average and standard deviation; added different fine-tuning options; compared against supervised methods and they seem to work worse than self-supervised on the downstream tasks

Neural codecs:

• The Interspeech 2024 Challenge on Speech Processing Using Discrete Units
• How Should We Extract Discrete Audio Tokens from Self-Supervised Models?
• Towards Audio Codec-based Speech Separation
• On the Effectiveness of Acoustic BPE in Decoder-Only TTS
• Exploring the Benefits of Tokenization of Discrete Acoustic Units
• NAST: Noise Aware Speech Tokenization for Speech Language Models

Foundation models

• COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning
• Exploring In-Context Learning of Textless Speech Language Model for Speech Classification Tasks
• Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models
• PAM: Prompting Audio-Language Models for Audio Quality Assessment
• Self-Supervised Speech Representations are More Phonetic than Semantic