Audio-based Toxic Language Detection
88Online gaming has been growing increasingly popular recently. This highly competitive online social platform can sometimes lead to undesired behavior and create an unfriendly community for many players. The detecting of profanity and bullying have been previously explored in text-based platforms (ex. social media, Twitter/Facebook) but when it comes to speech-based applications including online gaming, the field is relatively unexplored. In this project we focus on audio-based toxic language detection, which can be a great asset in scenarios where text transcriptions are not readily available. Additionally, audio-based queues such as speech tone or pitch variation could potentially provide supplementary or orthogonal information to transcribed content and word-based features. We have developed a Self-Attentive Convolutional Neural Network architecture to carry out the detection of toxic segments in naturalist audio recordings, challenged by diverse noise types such as background noise and music, overlapping speech, different microphone types, speech accents, and even languages. In order to tackle these challenges, the self-attention mechanism is used to attend to toxic frames while processing each utterance. We have evaluated our proposed system on a large internal dataset, as well as on publicly available data of a related domain. Our findings and results suggest promising directions toward automated toxic language detection for online gaming scenarios.