Arabic Named Entity Recognition with the ANER Corpus

One thing I forgot to mention on the release notes for LingPipe 3.8.2 is that I’ve added a section on building an Arabic named entity recognizer to the

LingPipe Named Entity Tutorial

Benajiba’s ANER Corpus

It’s based on Yassine Benajiba‘s freely distributed (thanks!) corpus:

ANER Corpus (Arabic Named Entity Recognition)

It’s 150K tokens in CoNLL format (easy to parse, but lossy for whitespace) using person, location, organization and miscellaneous types (like CoNLL’s English corpus). Here’s a sample (rendered with style value direction:rtl to get the ordering right; the actual file’s in the usual character order):

فرانكفورت B-LOC (د O ب O أ) O أعلن O اتحاد B-ORG صناعة I-ORG السيارات I-ORG في O ألمانيا B-LOC ...

Benajiba also supplies small dictionaries for locations, organizations and persons, which we explain how to use in the demo.

LingPipe Model and Evaluation

I applied a simple sentence-chunking heuristic and then built a character 8-gram-based rescoring chunker using otherwise default parameters and training on all the data (but not the dictionaries).
There’s absolutely nothing Arabic-specific about the model.

Overall performance is in line with Benajiba’s, while differing substantially on the four entity types. Here are the LingPipe 6-fold (125K token train, 25K token test) cross-validated results:

Type	Precision	Recall	F1
LOC	0.782	0.788	0.785
PERS	0.634	0.657	0.645
ORG	0.609	0.527	0.565
MISC	0.553	0.421	0.478

COMBINED	0.685	0.661	0.673

Dictionaries improved recall a bit, but hurt precision even more. Bigger dictionaries and more training data would certainly help here.

References

For a description of the corpus, and a description and evaluation of Benajiba’s own Arabic NER system, see:

Benajiba, Y. and P. Rosso 2007. ANERsys 2.0 : Conquering the NER task for the Arabic language by combining the Maximum Entropy with POS-tag information. In IICAI-2007.
Benajiba, Y., P. Rosso, and Benedí J. M. 2007. ANERsys: An Arabic Named Entity Recognition system based on Maximum Entropy. In CICLing-2007.

And of course, there’s more information in LingPipe’s Named Entity Tutorial.

This entry was posted on July 28, 2009 at 2:03 pm and is filed under Carp's Blog, LingPipe in Use, LingPipe News. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

12 Responses to “Arabic Named Entity Recognition with the ANER Corpus”

Laxmikant Agrawal Says:
April 27, 2010 at 7:12 pm | Reply
I am using Lingpipe for named entity recognition in my research. I want to create my own model. For that I am using same code as shown in named entity tutorial and used for CONLL2002 dataset. But I am getting erroors saying illegal line =Codexis B_ORG. First line in my training file is Codexis B_ORG. Same code is working for training files of CONLL2002. I created training file manually. Is there anything special I need to consider. This error is coming from LineTaggingParser.java class and parseString() in that file. Here it checks whether line matches the pattern or not. I dont see any mismatch with the pattern. Please help me out.
- Bob Carpenter Says:
  April 28, 2010 at 12:53 am | Reply
  You may have just gotten bitten by my updating the tutorials to reflect the actual tags used in CoNLL, B-ORG, not B_ORG. The last release of LingPipe (3.9.1), wouldn’t parse CoNLL, but would’ve parsed data with tags like B_ORG>
  
  All you need to do is convert your tags to CoNLL format, B-ORG, not B_ORG, and you should be good to go.
Laxmikant Agrawal Says:
April 28, 2010 at 11:42 am | Reply
I tried it already but I am getting following stacktrace.

F:\TextMining_Dr.Singh\LingPipeDir\lingpipe-3.9.1\demos\tutorial\ne>ant -Ddata.contracts=contracts/ train-contract
Buildfile: F:\TextMining_Dr.Singh\LingPipeDir\lingpipe-3.9.1\demos\tutorial\ne\b
uild.xml

compile:
[mkdir] Created dir: F:\TextMining_Dr.Singh\LingPipeDir\lingpipe-3.9.1\demos
\tutorial\ne\build\classes
[javac] Compiling 14 source files to F:\TextMining_Dr.Singh\LingPipeDir\ling
pipe-3.9.1\demos\tutorial\ne\build\classes

jar:
[jar] Building jar: F:\TextMining_Dr.Singh\LingPipeDir\lingpipe-3.9.1\demo
s\tutorial\ne\neDemo.jar

train-contract:
[java] Setting up Chunker Estimator
[java] Setting up Data Parser
[java] Training with Data from File=contracts\contract1.train
[java] java.util.regex.Matcher[pattern=(\S+)\s(\S+\s)?(O|[B|I]-\S+) region=
0,14 lastmatch=]
[java] Exception in thread “main” java.lang.IllegalArgumentException: Illeg
al =Codexis B-ORG
[java] at com.aliasi.tag.LineTaggingParser.parseString(LineTaggingParse
r.java:158)
[java] at Conll2002ChunkTagParser.parseString(Conll2002ChunkTagParser.j
ava:60)
[java] at com.aliasi.corpus.StringParser.parse(StringParser.java:68)
[java] at com.aliasi.corpus.Parser.parse(Parser.java:99)
[java] at com.aliasi.corpus.Parser.parse(Parser.java:115)
[java] at TrainConll2002.main(TrainConll2002.java:43)
[java] Java Result: 1
- lingpipe Says:
  April 29, 2010 at 11:25 am | Reply
  This really isn’t the best place for a discussion of bugs in LingPipe because I don’t check it as often as mail and it’s not where others go to find bug reports and/or patches.
  
  We have a mailing list linked from the web page and a direct mail address, bugs@alias-i.com.
  
  To answer the question, you need to upgrade to LingPipe 3.9.2. I fixed the bug I introduced in 3.9.1 with respect to NE parsing in CoNLL format. I tested the new parser configurations with all the data we talk about, and it now works with B- and I- tags.
Laxmikant Agrawal Says:
April 29, 2010 at 11:52 am | Reply
Sorry for posting bug report here. Thanks for the answer to my query.
noha Says:
March 21, 2011 at 7:37 am | Reply
hi, i have a question i can’t imagine how can we compute the effeciency of the NER system and what about
thanks in advance
- Bob Carpenter Says:
  March 21, 2011 at 12:42 pm | Reply
  I’m not sure what you mean — there are a bunch of things people often call “efficiency”. The big ones are memory and time. Our three built-in statistical NER systems vary on both of these. If you go to the last section of our NER tutorial, you’ll see a small comparison. The absolute amounts will depend on the number of categories, amount of pruning in the parameters, and the amount of training data and number of features.
  
  The best thing to do is find the NER you want and then time it on your hardware. For absolute throughput, keep in mind all our NER systems can be multithreaded with a single model in memory.
ziadov Says:
May 14, 2012 at 5:21 pm | Reply
Please could any one send me this class
src/ANERXVal.java.
cause i couldn’t find it on the server :(

Thanks in advance
- Bob Carpenter Says:
  May 15, 2012 at 1:41 pm | Reply
  That’s a lower-case “v” for some reason:
  
  http://alias-i.com/lingpipe/demos/tutorial/ne/src/ANERXval.java
  
  The web site mirrors the source tar ball, so it’s in
  
  /demos/tutorial/ne/src/ANERXval.java
  
  in the release.
  - ziadov Says:
    May 21, 2012 at 12:31 pm
    Thanks a lot for help :)
Cyber Says:
August 24, 2016 at 6:02 am | Reply
Hey LingPipe, you have made my life easier with your NLP tutorials and resources. I have successfully built NER model for Tigrigna Language (one of under resourced language like Arabic) using the resources in the LingPipe(ANERXval, an HMM model). But I need to build a model using CRF(Experiment Set-up — Partition: Dividing the entire corpus in to training(i.e. 2/3 or 80%) and testing set(i.e. 1/3 or 20%), and also Cross Validation: experiment with fold cross validation same as ANERXval using CRF to build the NER system (especially for scarce corpora) reading UNICODE format in text file.
What do you recommend me to do so? Please indicate me resources and other links to perform such NLP tasks? Thank you a lot for your excellent tutorials and supports.
- breckbaldwin Says:
  August 24, 2016 at 9:38 am | Reply
  Best source for learning how to use LingPipe at this point is our book. See
  LingPipe Cookbook. Available on Amazon too.