Exploring multimodal voice sentiment through SentHBC: A comparative study
Knowledge-Based Systems, cilt.351, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 351
- Basım Tarihi: 2026
- Doi Numarası: 10.1016/j.knosys.2026.116706
- Dergi Adı: Knowledge-Based Systems
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Applied Science & Technology Source, Compendex, INSPEC, Library, Information Science & Technology Abstracts (LISTA), Information Science & Technology Abstracts (LISTA), Academic Search Ultimate (EBSCO), Engineering Source (EBSCO), Technology Collection (ProQuest)
- Anahtar Kelimeler: BERT embeddings, Climate change discourse, Deep learning, MFCC, Multimodal sentiment analysis, SentHBC framework
- Bilecik Şeyh Edebali Üniversitesi Adresli: Evet
Özet
Sentiment analysis has emerged as a pivotal technique for deciphering human emotions and behaviors across diverse domains. While traditional Natural Language Processing (NLP) and Machine Learning (ML) methods have paved the way for speech pattern analysis, challenges regarding noisy real-world data, model robustness, and generalization persist. In this study, SentHBC (Sentimental-Hybrid-BERT–CNN), a novel multimodal deep learning architecture designed to integrate linguistic and acoustic features for enhanced sentiment detection, is proposed. Unlike unimodal approaches, SentHBC leverages contextual BERT embeddings and Mel-Frequency Cepstral Coefficients (MFCC) to capture the nuances of both text and tone. The proposed framework was developed using a specialized dataset focusing on ‘global warming’ and ‘climate change’, and was benchmarked against five leading machine learning algorithms through a rigorous 10-fold cross-validation methodology. Experimental results demonstrate that SentHBC significantly outperforms traditional ML baselines, achieving superior Accuracy, F1-score, and lower RMSE values. The findings validate the efficacy of multimodal fusion in mitigating error propagation in noisy environments and provide a robust solution for large-scale emotional analysis of environmental discourse. Additionally, this demonstrates that the proposed framework possesses high architectural adaptability, remaining open to further development and deployment across different domains.