FPGA unsolicited commercial email inline filter design using Levenshtein distance algorithm and longest common subsequence algorithm

Abstract

Today, spammers are known to slightly change their email keyword appearance to evade spam detection filter. The main purpose of this dissertation is to design and develop a high speed coprocessor implementation of spam inline filter engine that is able to filter out obfuscated spam. The advantage of this spam filtering engine is the engine has the ability to detect the spam which had performed the minor string change strategy by spammers. This can be achieved by implement Levenshtein Distance Algorithm and Longest Common Subsequence (LCS) in the detection engine which is a part of the spam filtering engine to detect spam by perform string matching operation between email received and pattern in databases. This spam filtering engine was implemented in Field Programmable Gate Array (FPGA) to increase the performance of this spam filtering engine in term of speed. A dataset from TREC 2007 Spam Track Public Corpus was used to carry out the experiment for this spam filtering engine. The performance of this spam filtering engine was measured by spam precision, accuracy, spam recall and false positive. As an experiment result, this spam filtering engine is able to provide an acceptable result by achieved 93.91% of spam precision at default threshold 70%.