From Statistical to Deep Learning Models: A Comparative Sentiment Analysis Over Commodity News
Published in Intelligent and Fuzzy Techniques for Emerging Conditions and Digital Transformation: Proceedings of the INFUS 2021 Conference, 2022
Recommended citation: M. S. Sivri, B. S. Korkmaz, A. Ustundag, (2022). "From Statistical to Deep Learning Models: A Comparative Sentiment Analysis Over Commodity News. " Intelligent and Fuzzy Techniques for Emerging Conditions and Digital Transformation: Proceedings of the INFUS 2021 Conference, held August 24-26, 2021. Volume 2. https://link.springer.com/chapter/10.1007/978-3-030-85577-2_18
In this paper, several natural language processing techniques with a varying range from statistical methods to deep learning-based methods were applied on the commodity news. Firstly, the dictionary-based methods were investigated with the most common dictionaries in financial sentiment analysis such as Loughran & McDonald and Harvard dictionaries. Then, statistical models have been applied to the commodity news with count vectorizer and TF-IDF. The compression-based NCD has been also included to test on the labeled data. To improve the results of the sentiment extraction, the news data was processed by deep learning-based state-of-art models such as ULMFit, Flair, Word2Vec, XLNet, and BERT. A comprehensive analysis of all tested models was held. The final analysis indicated the performance difference between the deep learning-based and statistical models for the sentiment analysis task on the commodity news. BERT has achieved superior performance among the deep learning models for the given data.