Comments on: Lucene 2.4 in 60 seconds

By: Bob Carpenter

Bob Carpenter — Wed, 19 Oct 2011 05:24:51 +0000

I’m afraid not — I’ve never gotten through the formulas. I believe there’s been discussion in the past on the Lucene mailing lists about BM25.

By: TrulyYours

TrulyYours — Mon, 17 Oct 2011 03:26:11 +0000

Very good tutorial. Simple and easy to understand. Do you have tutorials about BM25 algorithm?

By: Bob Carpenter

Bob Carpenter — Mon, 27 Jun 2011 17:47:59 +0000

That’s a bunch of different questions. It may be characters instead of tokens, because a field is constructed with a string or stream of characters.

The doc for Lucene’s perhaps not the best place to look. Go and look at the source code, and you should be able to track down what it’s doing. Or write a test case.

Lucene also has a very responsive user’s mailing list.

As for what gets skipped, that depends on the Analyzer implementation. Lucene has analyzers for many languages in its extended distribution (beyond the core jar). You can also define your own stop words. Typically, punctuation also gets removed, too.

By: Paul

Paul — Mon, 27 Jun 2011 14:55:56 +0000

Just a quick question since I am just checking on lucene configuration I found differing explainations of the “MaxFieldLength”-parameter.

“The constant IndexWriter.MaxFieldLength.LIMITED defaults to 10,000 characters.”
- is it realy characters or maybe words? on the lucene page it says “The maximum number of terms that will be indexed for a single field in a …”. Also I wonder if it only skips english fill words (and,but, if etc. or also other languages).

By: Sundus Hassan

Sundus Hassan — Wed, 02 Mar 2011 07:29:36 +0000

Is Lucene is a Knowledge base like Wikitology?
That is Wikitology has own knowledge base and index developed on Wikipedia.
But uptill now the examples I have gone through in that we are building index on text given by the user, not on any knowledge base.

Please help me in this regards, to clear this concept.

Will be looking forward for reply.

Thanks in advance.

By: lingpipe

lingpipe — Mon, 10 May 2010 21:56:12 +0000

While it’s nice that the new book is out, I don’t think the authors or Manning Press would appreciate the pirated PDF! So I’m erasing the link.

By: Pankil Patel

Pankil Patel — Mon, 10 May 2010 15:12:59 +0000

[link to copy of new Lucene in Action Book removed].

By: lingpipe

lingpipe — Tue, 10 Mar 2009 16:44:50 +0000

Everyone:

This post and comment thread wasn’t intended to be a replacement for the:

Lucene Mailing Lists

They get a lot of traffic, but are good at answering questions.

Having said that, these questions are easy:

@Antonio: yes. try it.

@Hari: yes, set the default field in the analyzer.

By: hari

hari — Tue, 10 Mar 2009 13:37:56 +0000

Hi,

I am working on the Lucene 2.4 for indexing the documents and searching. I have a requirement where the documents should be searched against a keyword among multiple fields (for each document). Can i search for the documents without mentioning the field names of the document?

Thanks

By: Antonio

Antonio — Tue, 10 Mar 2009 09:41:51 +0000

Hi.

I’m using Lucene 2.4 to index a document with the following text:

HELLO JAMES WELCOME

[stored/uncompressed,indexed, stored/uncompressed,indexed, stored/uncompressed,indexed]

HELLO FATHER GOODBYE JAMES

[stored/uncompressed,indexed, stored/uncompressed,indexed]

HELLO FATHER WELCOME FATHER

[stored/uncompressed,indexed]

GOODBYE JAMES GOODBYE FATHER

[stored/uncompressed,indexed, stored/uncompressed,indexed, stored/uncompressed,indexed]

For each line I created a new document with the instruction doc = new Document(); I saved the text in a Lucene index doc.add(new Field(“p”, line, Field.Store.YES, Field.Index.NOT_ANALYZED)); the number of the line of each phrase doc.add(new Field(“numLine”, numLine, Field.Store.YES, Field.Index.NOT_ANALYZED)); and finally for each people’s name I created a new Lucene index doc.add(new Field(“name”, name_person, Field.Store.YES, Field.Index.NOT_ANALYZED));

I want to make a searcher that gives every files and fields with the searching name, for instance, searching for “JAMES” will create a result in documents 1,2 and 4; searching for “HELLO FATHER” will give documents 2 and 3 and searching “FATHER” will result in documents 2, 4 and 3 (twice)….but…

If I search “hello” the result is the number 1.
If I search “hello f*” the result is the number 1.
If I search “father” the result is nothing.
If I search “james” the result is the number 1,2 and 4. –> Is the only correct.

Right now I just know how to look for one word, for example with “JAMES” I use query: name: james.

How can I search for a phrase or more than one word? Will it be something similar to query: p:hello p:father ?

Bye