Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
HeadRouter is a training-free audio token pruning method for large audio language models. It uses task-adaptive head-weight routing to preserve important audio tokens under different semantic and acoustic workloads.
- Task-adaptive routing: dynamically adjusts attention-head weights for different audio task types.
- Training-free pruning: applies without additional model training.
- Audio-focused efficiency: targets long-context audio understanding in large audio language models.
- Strong compression performance: retains competitive or improved performance under aggressive token pruning ratios.
This page accompanies the HeadRouter manuscript and project materials.
- Repository: https://github.com/DabDans/Headrouter_arxiv
- Homepage: https://dabdans.github.io/
Citation information will be added after release.