A Spam Transformer Model for SMS Spam Detection

Abstract

With the prosperity of the Short Message Service (SMS), the increasing number of spam messages has become a serious problem. The need to block spam messages requires us to develop new SMS spam detection technologies. The Transformer, an attention- based sequence to sequence model, has achieved excellent results in multiple different tasks recently. In this thesis, we propose a modified Transformer model for SMS spam messages detection. The evaluation of our proposed modified spam Transformer is performed on SMS Spam Collection v.1 dataset and UtkMl’s Twitter Spam Detection Competition dataset, with the benchmark of multiple established classifiers such as Logistic Regression, Na ̈ıve Bayes, Random Forests, Support Vector Machine, and Long Short-Term Memory. In comparison to all other candidates, our experiments show that the proposed modified spam Transformer achieves the best results in terms of almost all selected performance criteria.