Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

  • Create BookmarkCreate Bookmark
  • Create Note or TagCreate Note or Tag
  • PrintPrint
Share this Page URL
Help

6. Document Filtering > Improving Feature Detection

Improving Feature Detection

In all the examples so far, the function for creating the list of features uses just a simple nonalphanumeric split to break up the words. The function also converts all words to lowercase, so there’s no way to detect the overuse of uppercase words. There are several different ways this can be improved:

  • Without actually making uppercase and lowercase tokens completely distinct, use the fact that there are many uppercase words as a feature.

  • Use sets of words in addition to individual words.

  • Capture more metainformation, such as who sent an email message or what category a blog entry was posted under, and annotate it as metainformation.

  • Keep URLs and numbers intact.


  

You are currently reading a PREVIEW of this book.

                                                                                                                    

Get instant access to over $1 million worth of books and videos.

  

Start a Free Trial


  
  • Safari Books Online
  • Create BookmarkCreate Bookmark
  • Create Note or TagCreate Note or Tag
  • PrintPrint