This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
In 6.9.1 if the user enters too general query request (e.g. in Add Dependency dialog), our code at NexusRepositoryIndexerImpl.repeatedFlatSearch() tries calling into Nexus indexer search engine (which uses Lucene) several times with a certain limit set for Lucene with BooleanQuery.setMaxClauseCount(i); which correctly gets back to us from Lucene with TooManyClauses exception, and we are able to tell the user that the query is too general. In current trunk this stopped working and regardless of what limit we set by setMaxClauseCount(i) Lucene runs into OOME instead of stopping at the specified clause number limit. I am not sure if this regression is caused by the recent update of Lucene version (to 2.9.3) or by the update of Maven jars. Anyway, Tomas (on CC) offered me a guidance how we should do it better and fix this problem.
This is not a Lucene problem, it's problem of Lucene usage. The org.sonatype.nexus.index.SearchEngine.searchFlatPaged has no limit on number of documents loaded, so it can cause OOM. The NRIImpl.repeatedFlatSearch used setMaxClausesCount to increase the default from default 1K to 8K of clauses. The 8K clauses is also quite big number but better then unlimited. Lucene 2.9 does not use BooleanQuery rewrite by default, it does FQ rewrite as it's faster and takes less memory. This change causes that there is no limit on the number of documents at all (even the maxClauses) and it causes OOM. I will at least switch the queries used by NRIImpl to use bool rewrite, but the searchIteratorPaged should be better. I will try also to use it but it may cause problems with locking.
For now I will add commit the changed rewrite. The usage of Iterator will take more time.
Fixed jet-main 7d3394a9d5a9 For now Queries switched to bool rewrite. I will evaluate the possibility to use iterator.
Integrated into 'main-golden', will be available in build *201009160000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/7d3394a9d5a9 User: Tomas Zezula <tzezula@netbeans.org> Log: #190265:Lucene queries running into OOME instead of throwing TooManyClauses exception
Tomasi, thanks a lot for the fix! It works.
Surprisingly, still encountering some cases where I ran into OOME, e.g. searching for "Class". Looking into it.
The repeat* has to be used to prevent the OOM. There are some searches which does not use it (and I didn't touch them).
Actually finetuning of the max number of clauses was needed. And the following change also improves the way of communicating the problem of the too general query to the user. core-main #ee5cd5612a9e
Integrated into 'main-golden', will be available in build *201009180000* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/ee5cd5612a9e User: Antonin Nebuzelsky <anebuzelsky@netbeans.org> Log: #190265 - (cont.) Lucene queries running into OOME instead of throwing TooManyClauses exception.