This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 194351 - Update lucene to 3.0.3
Summary: Update lucene to 3.0.3
Status: RESOLVED FIXED
Alias: None
Product: editor
Classification: Unclassified
Component: Parsing & Indexing (show other bugs)
Version: 7.0
Hardware: PC Mac OS X
: P3 normal (vote)
Assignee: apireviews
URL:
Keywords: API, API_REVIEW_FAST
Depends on: 194597
Blocks:
  Show dependency tree
 
Reported: 2011-01-14 14:33 UTC by Tomas Zezula
Modified: 2011-01-27 20:46 UTC (History)
4 users (show)

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
patch of main (22.79 KB, patch)
2011-01-14 14:35 UTC, Tomas Zezula
Details | Diff
patch of main (unit test fixed) (27.86 KB, application/octet-stream)
2011-01-15 09:41 UTC, Tomas Zezula
Details
Patch of contrib (40.16 KB, application/octet-stream)
2011-01-15 14:04 UTC, Tomas Zezula
Details
Patch of main (unit test fixed, indexer-core updated to 4.0.0) (1.88 KB, patch)
2011-01-17 17:07 UTC, Tomas Zezula
Details | Diff
Patch of main (unit test fixed, indexer-core updated to 4.0.0) (81.52 KB, patch)
2011-01-17 18:16 UTC, Tomas Zezula
Details | Diff
Patch of main (89.45 KB, patch)
2011-01-19 18:09 UTC, Tomas Zezula
Details | Diff
Patch of main without removal of deprecated APIs (28.32 KB, patch)
2011-01-24 16:44 UTC, Tomas Zezula
Details | Diff
Final diff of main (22.08 KB, patch)
2011-01-24 17:45 UTC, Tomas Zezula
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tomas Zezula 2011-01-14 14:33:41 UTC
Update lucene ti 3.0.3 and remove all deprecated APIs.
Comment 1 Tomas Zezula 2011-01-14 14:35:47 UTC
Created attachment 104995 [details]
patch of main
Comment 2 Tomas Zezula 2011-01-14 14:37:34 UTC
I've added diff of main. I will test it for while and update contrib (gsf) in separate patch.
Then I will switch this task to regular fast API review.
Comment 3 Jesse Glick 2011-01-14 21:34:05 UTC
Does maven.indexer actually continue to work with just this change? Or is it also necessary to patch NexusRepositoryIndexerImpl, and/or update Maven Indexer to 4.0.0? (Let me know if you're not sure how to test Maven indexing usages and want help with that. Might be useful for the changes to be on a branch.)
Comment 4 Tomas Zezula 2011-01-15 09:41:16 UTC
Created attachment 105022 [details]
patch of main (unit test fixed)
Comment 5 Tomas Zezula 2011-01-15 12:56:31 UTC
I think that the NexusRepositoryIndexerImpl does not need to be changed, but the MavenPluginArtifactInfoIndexCreator is still the Lucene 2.9 based. I need to update the indexer-core to 4.0.0. I've tried to download the jars by maven build of some dummy project referencing it (https://repository.apache.org/content/repositories/maven-016/) but the build failed with:

Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could not find artifact org.apache.maven.indexer:indexer-artifact:jar:4.0.0 in central (http://repo1.maven.org/maven2)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:945)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:940)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.flush(WagonRepositoryConnector.java:696)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.flush(WagonRepositoryConnector.java:690)
        at org.sonatype.aether.connector.wagon.WagonRepositoryConnector.get(WagonRepositoryConnector.java:435)
        at org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:438)

Jesse can you point me how to get this jar?
I've added the https://repository.apache.org/content/repositories/maven-016/ into the maven repositories, added the jars into the dummy project as dependency and run build on the project.
Thanks
Comment 6 Tomas Zezula 2011-01-15 14:04:24 UTC
Created attachment 105032 [details]
Patch of contrib
Comment 7 Tomas Zezula 2011-01-17 13:41:38 UTC
I've found the jars but it seems that indexer-core 4.0.0 is not compatible with 3.1.0. The maven.indexer does not build. I am trying to fix the NexusRepositoryIndexImpl.java
Comment 8 Tomas Zezula 2011-01-17 17:07:16 UTC
Created attachment 105074 [details]
Patch of main (unit test fixed, indexer-core updated to 4.0.0)
Comment 9 Tomas Zezula 2011-01-17 17:09:30 UTC
I've updated the indexer-core and indexer-artifact to 4.0.0. But I am getting the following exception:


java.lang.ClassNotFoundException: org.sonatype.aether.version.InvalidVersionSpecificationException
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at org.netbeans.ProxyClassLoader.loadClass(ProxyClassLoader.java:262)
Caused: java.lang.ClassNotFoundException: org.sonatype.aether.version.InvalidVersionSpecificationException starting from ModuleCL@2df36f51[org.netbeans.modules.maven.indexer] with possible defining loaders [ModuleCL@2a9b5441[org.netbeans.modules.maven.embedder]] and declared parents [ModuleCL@6a0d4461[org.netbeans.core.multiview], ModuleCL@2c618b08[org.netbeans.libs.lucene], ModuleCL@cccfa5e[org.netbeans.api.progress], ModuleCL@5c64a871[org.netbeans.api.annotations.common], ModuleCL@2a9b5441[org.netbeans.modules.maven.embedder], ModuleCL@36592bb1[org.openide.windows], org.netbeans.MainImpl$BootClassLoader@4104c575, ModuleCL@6688cf9e[org.netbeans.modules.projectapi], ModuleCL@9e4acce[org.openide.awt]]
	at org.netbeans.ProxyClassLoader.loadClass(ProxyClassLoader.java:264)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Caused: java.lang.NoClassDefFoundError: org/sonatype/aether/version/InvalidVersionSpecificationException
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl$7.run(NexusRepositoryIndexerImpl.java:771)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl$7.run(NexusRepositoryIndexerImpl.java:768)
	at org.openide.util.Mutex.writeAccess(Mutex.java:436)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.filterGroupIds(NexusRepositoryIndexerImpl.java:768)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.getGroups(NexusRepositoryIndexerImpl.java:724)
	at org.netbeans.modules.maven.indexer.api.RepositoryQueries.getGroups(RepositoryQueries.java:85)
	at org.netbeans.modules.maven.repository.GroupListChildren.createKeys(GroupListChildren.java:77)
	at org.openide.nodes.AsynchChildren.run(AsynchChildren.java:189)
Caused: org.openide.util.RequestProcessor$SlowItem: task failed due to
	at org.openide.util.RequestProcessor$Task.schedule(RequestProcessor.java:1465)
	at org.openide.nodes.AsynchChildren.refresh(AsynchChildren.java:134)
	at org.openide.nodes.ChildFactory.refresh(ChildFactory.java:163)
	at org.netbeans.modules.maven.repository.GroupListChildren.stateChanged(GroupListChildren.java:86)
	at org.openide.util.WeakListenerImpl$Change.stateChanged(WeakListenerImpl.java:329)
	at org.netbeans.modules.maven.indexer.api.RepositoryInfo.fireChangeIndex(RepositoryInfo.java:160)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.fireChangeIndex(NexusRepositoryIndexerImpl.java:706)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.access$900(NexusRepositoryIndexerImpl.java:163)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl$5.run(NexusRepositoryIndexerImpl.java:700)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1424)
[catch] at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1968)


Any idea where the InvalidVersionSpecificationException should come from?
Thanks
Comment 10 Tomas Zezula 2011-01-17 18:16:49 UTC
Created attachment 105082 [details]
Patch of main (unit test fixed, indexer-core updated to 4.0.0)
Comment 11 Tomas Zezula 2011-01-17 18:22:57 UTC
Seems that the indexer-core-4.0.0 depends on aether which does not exist. Non existence of aether was already proved by Michelson and Morley, so it's not very surprising (http://en.wikipedia.org/wiki/Michelson-Morley_experiment).
Comment 12 Tomas Zezula 2011-01-17 18:49:46 UTC
If I add aether-api-1.9.jar jar. The CDNF is thrown:


Caused: java.lang.NoClassDefFoundError: org/sonatype/aether/util/version/GenericVersionScheme
	at org.apache.maven.index.ArtifactInfo.<init>(ArtifactInfo.java:213)
	at org.apache.maven.index.context.IndexUtils.constructArtifactInfo(IndexUtils.java:117)
	at org.apache.maven.index.context.IndexUtils.updateDocument(IndexUtils.java:134)
	at org.apache.maven.index.context.IndexUtils.updateDocument(IndexUtils.java:129)
	at org.apache.maven.index.updater.DefaultIndexUpdater.copyUpdatedDocuments(DefaultIndexUpdater.java:343)
	at org.apache.maven.index.updater.DefaultIndexUpdater.unpackIndexArchive(DefaultIndexUpdater.java:272)
	at org.apache.maven.index.updater.DefaultIndexUpdater.loadIndexDirectory(DefaultIndexUpdater.java:202)
	at org.apache.maven.index.updater.DefaultIndexUpdater.access$300(DefaultIndexUpdater.java:75)
	at org.apache.maven.index.updater.DefaultIndexUpdater$LuceneIndexAdaptor.setIndexFile(DefaultIndexUpdater.java:641)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:875)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:156)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexerImpl.indexLoadedRepo(NexusRepositoryIndexerImpl.java:507)


Seems that all aether jars are required. Any idea?
Comment 13 Tomas Zezula 2011-01-18 09:17:34 UTC
Seems that aether-api and aether-util is enough. If I add them to maven.indexer runtime cp it seems to work fine. Should I add them?
Comment 14 Jesse Glick 2011-01-18 15:28:59 UTC
(In reply to comment #13)
> Seems that aether-api and aether-util is enough. If I add them to maven.indexer
> runtime cp it seems to work fine. Should I add them?

No, these are already part of maven.embedder, which maven.indexer depends on. Probably all that is missing is for their packages to be exported.

BTW Maven Indexer 4.0.0 was formally released; check that the JARs you downloaded match the official release.
Comment 15 Tomas Zezula 2011-01-18 17:41:25 UTC
OK, I will export those packages (api, util).
From where the official Maven Indexer 4.0.0 can be downloaded from?

What should be next? Should I create a branch? Or should I put it into API review?
When we want to integrate it?
Comment 16 Jesse Glick 2011-01-19 14:44:24 UTC
(In reply to comment #15)
> From where the official Maven Indexer 4.0.0 can be downloaded from?

http://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-core/4.0.0/indexer-core-4.0.0.jar
http://repo1.maven.org/maven2/org/apache/maven/indexer/indexer-artifact/4.0.0/indexer-artifact-4.0.0.jar

> Should I create a branch? Or should I put it into API review?

I think it needs an API review, since libs.lucene exposes a public API.

> When we want to integrate it?

The usual week review period I guess.
Comment 17 Tomas Zezula 2011-01-19 14:47:21 UTC
OK, thanks Jesse.
I will attach the final diff this evening and send it to API review.
Comment 18 Tomas Zezula 2011-01-19 18:09:31 UTC
Created attachment 105156 [details]
Patch of main
Comment 19 Tomas Zezula 2011-01-19 18:11:46 UTC
Review of non compatible update of Lucene library and migration to it.
Tomas M. will test it tomorrow to decide if we want to put it into NB 7.0
Comment 20 Jesse Glick 2011-01-19 20:30:43 UTC
[JG01] Is there some reason libs.lucene exposes a numeric "OpenIDE-Module-Implementation-Version: 232"? I do not see any mention in http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/artifact/nbbuild/build/generated/impl-deps.txt so I am guessing this is obsolete. If so, remove the explicit impl version, and switch "spec.version.base=3.0.0" in project.properties to "OpenIDE-Module-Specification-Version: 3.0" in manifest.mf.


[JG02] Modules such as maven.hints using new libs.lucene must increment their spec versions; otherwise http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/testReport/org.netbeans.nbbuild/VerifyUpdateCenter/diachronicConsistency/ would fail as the old client could not link against the new libs.lucene. maven.indexer also needs to be incremented for the same reason, and probably maven.* modules using it should ask for the new maven.indexer too, just to be safe.


[JG03] Minor, but it is nicer to use 'hg mv' to rename license files which are mostly the same except for a change of version. For example, this would clearly reveal in the diff that maven.indexer/external/maven-indexer-4.0.0-license.txt is using a different Origin header, when it need not. (In fact the new URL is to a staging repository which has now been dropped -> 404.)


[JG04] I am trying to understand the reason behind a diff like:

-                        bq.add(new BooleanClause(new PrefixQuery(new Term(ArtifactInfo.UINFO, prefix)), BooleanClause.Occur.MUST));
+                        String uinfo = ArtifactInfo.UINFO;
+                        final Term term = new Term(uinfo, prefix);
+                        final Query prefQuery = new PrefixQuery(term);
+                        bq.add(new BooleanClause(prefQuery, BooleanClause.Occur.MUST));

Did you actually change anything here? Or here:

-<<EOL>>
+            <<EOL>>

(See <https://bitbucket.org/marcusl/ml-hgext/src/tip/checkfiles.py> about that, and/or make sure the NB editor is set to trim WS from modified lines.)


[JG05] Changes in maven.indexer (and perhaps also parsing.lucene) which merely remove usage of deprecated APIs in Lucene 2.x, but do not use anything new in 3.x (or Maven Indexer 4.0.0), would better be done as a (non-reviewed, immediate) commit and taken as a baseline for this patch.
Comment 21 Tomas Zezula 2011-01-21 11:53:37 UTC
JG01: Right, seems to be unneeded. I will remove it.

JG02: OK, I will rather increment each module depending on Lucene.

JG03: OK

JG04: I've thought it's enabled by default. I've lost it while migrating to a new computer. Now I reenabled it.

JG05: No problem I will integrate the parts related to removing of deprecated api immediately.
Comment 22 Tomas Zezula 2011-01-24 16:44:01 UTC
Created attachment 105308 [details]
Patch of main without removal of deprecated APIs
Comment 23 Tomas Zezula 2011-01-24 16:45:29 UTC
I've added the new patch of main. The removal of deprecated API is removed from it (it was already integrated).
Jesse can you look at it? Especially to problems with diachronicConsistency test. Thanks
Comment 24 Jesse Glick 2011-01-24 17:12:35 UTC
maven.buildplan is not currently buildable; does not need to be patched now.


maven.indexer/external/maven-indexer-4.0.0-license.txt should not change the Origin header.


I don't think most maven.* modules need to be updated, since maven.indexer hides the Indexer lib behind an API facade. So only if the module's source code and/or deps change does it need to be updated.
Comment 25 Tomas Zezula 2011-01-24 17:45:02 UTC
Created attachment 105309 [details]
Final diff of main
Comment 26 Tomas Zezula 2011-01-24 17:45:35 UTC
If QA agrees I will integrate it tomorrow.
Comment 27 Jesse Glick 2011-01-24 18:10:00 UTC
maven.indexer should request maven.embedder 2.5. And it is odd that maven.repository is being bumped up to 1.11 rather than 1.10. Otherwise looks fine to me, assuming you evaluate bug #194597.
Comment 28 Tomas Zezula 2011-01-25 08:50:33 UTC
Fixed jet-main fbffc579e53e
Fixed contrib 3c793a3084b3
Comment 29 Marian Mirilovic 2011-01-25 08:57:38 UTC
(In reply to comment #26)
> If QA agrees I will integrate it tomorrow.

QA agrees and I approve integration into NB 7.0 code base.
Comment 30 Quality Engineering 2011-01-27 08:44:18 UTC
Integrated into 'main-golden', will be available in build *201101270001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/fbffc579e53e
User: Tomas Zezula <tzezula@netbeans.org>
Log: #194351:Update lucene to 3.0.3
Comment 31 Quality Engineering 2011-01-27 20:46:50 UTC
Integrated into 'main-golden', will be available in build *201101271430* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/fbffc579e53e
User: Tomas Zezula <tzezula@netbeans.org>
Log: #194351:Update lucene to 3.0.3