The SVM approach is interesting, though why the paper is called that is confusing because they conclude the neural network approach has the same performance and is more robust. What I was planning was similar to Schölkopf’s one-class SVM. I’m surprised their method of using outliers in the positive data (what I’m evaluating right now) is more effective. It reminds me of using entries on the n-best list as negative examples for sequence tagging. Unfortunately, overall results are rather disappointing, being in the 50% range. I should emphasize we’d really like an approach that could balance precision and recall for different applications (for search we want high recall in the tail; for some other apps, we need high precision).

The problem posed for the 2008 UC San Diego Data Mining Contest: Positive-Only Semi-Supervised Task is exactly what we’re trying to do, though stated in the form of 20 real-valued features instead of raw texts. What’s amazing to me is that there are about 50 distinct entries on the Leaderboard. There’s even prize money for win, place and show associated with this contest.

The problem solved is described here: http://mill.ucsd.edu/index.php?page=Datasets&subpage=Task2, and I’d be happy to get more information on the method if you’re interested.