Real-time Multiple Dyadic Interaction Detection in Surveillance Videos in the Wild

Social distancing measures are proposed as the primary strategy to curb the spread of pandemics caused by respiratory pathogens. Therefore, non-intrusive techniques to identify human-human interactions in public spaces play an important role in curtailing disease spread. This paper proposes a novel computer vision-based system that identifies multiple co-occurring dyadic, or two-person, interactions in crowded scenarios and classifies them into six action classes. Human skeletons are extracted from RGB videos to eliminate background noise, thereby improving overall detection accuracy. Two classifier approaches are used: X3D-M and X3D-M + Attention. The latter achieves higher classification accuracy due to the attention layer's ability to capture long-distance interdependence across video frames. The efficacy of the proposed model was evaluated on two different datasets comprising more than 5,000 frames, enabling robust detection across different environments. To the best of our knowledge, the proposed model is the first dyadic interaction detector designed for in-the-wild scenarios, making it suitable for use in public spaces to identify and mitigate the transmission of respiratory diseases.