Comments on: Trieschnigg, Pezik, Lee, de Jong, Kraaij, and Rebhoz-Schumann (2009) MeSH Up: Effective MeSH Text Classification for Improved Document Retrieval

By: lingpipe

lingpipe — Tue, 01 Sep 2009 21:50:39 +0000

The U.S. National Institutes of Health require pre-prints to be submitted to them by any grantholders, which they then put up on PubMed. We’ll see how long that lasts.

We are huge fans of fine-grained tokenizations. Our basic MEDLINE index has an all-info tokenization (no case norm, no punctuation removal, etc.), but a pretty fine-grained tokenization. We have a second standard reduced index. And a third that uses character n-grams.

We’ve found the n-grams to be really robust across languages and to work well for MEDLINE. For instance, we used them in the one TREC genomics evaluation, and we use them for classifiers. We use a range of token lengths, with 3-5 grams as well as 2-grams and longer than 5-grams if we can afford it. And we use the cross-word character n-grams, which can really help with precision (boosting phrases).

The best Chinese indexers I know use unigrams, bigrams, and any substrings matching dictionary entries.

I’ve been recommending that same strategy with stemmers, which is to include “unfortunately”, “unfortunate”, “fortunate” and “fortune” when you see the first. It maintains some precision while increasing recall. The only problem is space and time for the searches.

You can do the same thing with thesauri.

One way of looking at all these maneuvers is as finding a good feature-based representation for classification. There have been lots of interesting papers recently on doing this with everything from multi-view learning to clustering with SVD and other techniques to deep belief nets.

By: Dolf Trieschnigg

Dolf Trieschnigg — Tue, 01 Sep 2009 07:55:39 +0000

Well, it’s the same for our published response to that comment: you have to pay the open-access publication charges to make it available to everyone. Fortunately, you are allowed to make the pre-print version available on your own website, in this case no editors or reviewers made or proposed any changes, so the pre-print on the author’s website is the same as the actual publication except from the final layout.

We also investigated the tokenization for the biomedical domain, have a look at our SIGIR’07 poster (“The Inﬂuence of Basic Tokenization on Biomedical
Document Retrieval”, SIGIR, 2007) and the article by Jiang and Zhai (“An Empirical Study of Tokenization Strategies for Biomedical Information Retrieval”, IR, 2007). In our experience, generating surplus tokens really worked well, so e.g. tokenizing “P53-activator” as [p], [53], [p53], [activator] and [p53activator] increased both precision and recall.
Interestingly, many machine learning methods often use character n-gramming techniques, where in my own experience this doesn’t really work well for biomedical IR.

By: lingpipe

lingpipe — Mon, 31 Aug 2009 19:32:30 +0000

Thanks -- that description of the IR component really helped. Just because it's easy and relatively efficient, we often work with character n-gram language model rescoring of Lucene's n-best TF/IDF results with a fairly simple tokenizer. Looking at all the TF/IDF variants and parameterizations crossed with stemming, stoplisting and other normalization and crossed with realistic queries makes my head hurt. The Névéol, Mork, and Aronson comment on your paper isn't available without a subscription. Ironic given that the authors work for NLM.

By: Dolf Trieschnigg

Dolf Trieschnigg — Sun, 30 Aug 2009 12:51:03 +0000

Thanks for the review of our paper.
Just to clarify things a bit further about improving IR with MeSH:
we used two indices, one with MeSH descriptors, one with full-text. Two query representations were used: the original textual query, and the automatically obtained MeSH representation (determined using the KNN approach). The MeSH query is matched aginst the MeSH index, the text query against the text index and the relevance scores are mixed through linear interpolation.

Your point about the retrieval model is probably right: probably a vector space retrieval model will give similar results. In fact, Neveol et al recently wrote a comment on our paper (see the Bioinformatics website) which shows that the Pubmed Related Citations (PRC) algorithm also gives very good results. In its describing paper, PRC outperformed BM25 (if I am not mistaken). Similarly, LM IR also has shown to outperform BM25. In any case, KNN seems a pretty steady baseline for automatic indexing.