Comments on: The Curse of “Intelligent” Tokenization http://lingpipe-blog.com/2008/06/26/the-curse-of-intelligent-tokenization/ Natural Language Processing and Text Analytics Sat, 04 Feb 2012 20:56:48 +0000 hourly 1 http://wordpress.com/ By: lingpipe http://lingpipe-blog.com/2008/06/26/the-curse-of-intelligent-tokenization/#comment-2582 Mon, 14 Jul 2008 15:11:17 +0000 http://lingpipe.wordpress.com/?p=106#comment-2582 Habash and Rambow perform joint tokenization and morphological tagging in their 2005 ACL paper Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop.

Adler and Elhadad propagate token uncertainty in a lattice through morphological processing in their 2006 ACL paper An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation.

]]> By: Lucene’s Missing Token Stream Factory « LingPipe Blog http://lingpipe-blog.com/2008/06/26/the-curse-of-intelligent-tokenization/#comment-2545 Mon, 07 Jul 2008 04:48:19 +0000 http://lingpipe.wordpress.com/?p=106#comment-2545 [...] LingPipe Blog Alias-i’s Natural Language Processing and Text Analytics API « The Curse of “Intelligent” Tokenization [...]

]]>