RadSpecFusion: Dynamic attention weighting for multi-radar human activity recognition

Abstract

This paper presents RadSpecFusion, a novel dynamic attention-based fusion architecture for multi-radar human activity recognition (HAR). Our method learns activity-specific importance weights for each radar modality (24 GHz, 77 GHz, and Xethru sensors). Unlike existing concatenation or averaging approaches, our method dynamically adapts radar contributions based on motion characteristics. This addresses cross-frequency generalization challenges, where transfer learning methods achieve only 11%–34% accuracy. Using the CI4R dataset with spectrograms from 11 activities, our approach achieves 99.21% accuracy, representing a 15.8% improvement over existing fusion methods (83.4%). This demonstrates that different radar frequencies capture complementary information about human motion. Ablation studies show that while the three-radar system optimizes performance, dual-radar combinations achieve comparable accuracy (24GHz+77GHz: 96.1%, 24GHz+Xethru: 95.8%, 77GHz+Xethru: 97.2%), enabling flexible deployment for resource-constrained applications. The attention mechanism reveals interpretable patterns: 77 GHz radar receives higher weights for fine movements (superior Doppler resolution), while 24 GHz dominates gross body movements (better range resolution). The system maintains 71.4% accuracy at 10 dB SNR, demonstrating environmental robustness. This research establishes a new paradigm for multimodal radar fusion, moving from cross-frequency transfer learning to adaptive fusion with implications for healthcare monitoring, smart environments, and security applications.

Keywords: Attention mechanisms, Cross-frequency transfer learning, Human activity recognition, Multi-modal fusion