Needleman-Wunsch and smith-waterman implementation for Spam/Uce inline filter

Abstract

Spam have been a significant problem as it consumes bandwidth of the internet, waste surfers time, waste computational resources of internet service providers and reduce the efficiency of email as a way of communication. Despite various anti spam solutions introduced, spam mails tend to be able to avoid detection by slightly modifying their spam signature. This helps to avoid anti-spam solutions from successfully detecting the keywords in emails that are closely associated with spam. Two algorithms named Needleman-Wunsch and Smith-Waterman will be implemented on FPGA as spam detection engine. Both algorithms share its origin from the theory of dynamic programming and are normally implemented in bioinformatics for sequence alignment. As both are well-known for their ability to detect sequences with slight changes caused by mutation, these two algorithms will be used to detect spam messages that slightly change its spam keyword. FPGA have been selected as the device for implementation. As hardware are faster than software, using FPGA helps to reduce the scanning time and reduce the CPU load of the computer. The advancement of FPGA technologies help to make it capable of becoming a standalone scanning unit. The effectiveness of both algorithms in spam scanning will be looked into. The corpus from Text Retrieval Conference (TREC 2007) will be used to test the effectiveness of the anti-spam engines.