Bug 195300

Must not publish RELEASE70-BETA2 repository without fixing this.

Any steps how to fix it?

This issue appears for RELEASE69 as well so what does it happened more serious for RELEASE70-BETA2?

(In reply to comment #2)
> Any steps how to fix it?

See URL which documents what you need to do.

(In reply to comment #3)
> This issue appears for RELEASE69 as well

And it was a major problem. Unfortunately it was only caught recently, and it is not good practice to retroactively change Maven release artifacts.

> [is it] more serious for RELEASE70-BETA2?

Yes, because now we may not legally distribute JUnit.

I have checked both tasks trunk and 70-beta2 maven repository generation. Both tasks use the optional parameter nexusIndexDirectory with the correctly setup path to the unzipped Nexus Lucene index. The only suspicious think was exception "org.apache.lucene.index.CorruptIndexException: Unknown format version: -7" in the log for the 70-beta2 maven repository. The exception has disappeared after the Nexus Lucene index replacement by the newest version. In spite of that copies of already existing artifacts are still created. It seems to me like a problem in the nbm:populate-repository plugin itself.

Is a fix in nbm:populate-repository plugin needed from Jesse?

Or does it mean that you are ready to populate repository now?

Yes, IHMO a fix is needed in the plugin. I think, BE doesn't own the plugin source code.

Jesse, please address the problem ASAP. Thanks.

Working for me. I ran the 3.5 repo generator with the nexusIndexDirectory param set to a fresh download of the documented ZIP, on some clusters including java and did not see org/netbeans/external:plexus-interpolation-1.14; ~/.m2/repository/org/netbeans/modules/org-netbeans-modules-maven-embedder/nbdev/org-netbeans-modules-maven-embedder-nbdev.pom referred to org.codehaus.plexus:plexus-interpolation:1.14 as expected.

(There were more org.netbeans.external entries than there really should be, but these appear to be genuine problems with our modules. For example, ~/.m2/repository/org/netbeans/external/xerces-2.8.0/nbdev/xerces-2.8.0-nbdev.jar is the same length but somehow different from xerces/xercesImpl-2.8.0.jar as found in both Central and the official Apache download site, meaning jlahoda's c7ff2776771d is somehow suspect.)

For debugging purposes, PopulateRepositoryMojo.java:746-787 is the relevant code.

I have tried to run the maven repo generation in my own directory and the result is the same as in the official build. I can still see following files after generation:

~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.jar.md5
~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.pom
~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.jar.sha1
~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.pom.sha1
~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.jar
~/maven-snapshot/org/netbeans/external/plexus-interpolation-1.14/SNAPSHOT/plexus-interpolation-1.14-20110228.125137-1.pom.md5

The official build uses the following command to generate the repo:

+ /space/apache-maven-2.2.1/bin/mvn -DdeployUrl=file://maven-snapshot -DforcedVersion=SNAPSHOT -DnetbeansInstallDirectory=nbrepo/netbeans -DnetbeansJavadocDirectory=nbrepo/javadoc -DnetbeansNbmDirectory=nbrepo/nbms -DnetbeansSourcesDirectory=nbrepo/source-zips -DnexusIndexDirectory=nbrepo/nexus/ -DskipInstall=true org.codehaus.mojo:nbm-maven-plugin:3.5:populate-repository --settings settings.xml

Does anybody view any suspicious setting there?

Use Maven 3.0.x, not 2.2.1 please. Other than that I don't see anything amiss in the command line; see what is happening in PopulateRepositoryMojo.java.

The result with Maven 3.0.2 is the same as before. A debugging will be needed very likely.
Does anybody know where is the source code for PopulateRepositoryMojo?

Just one thing before debugging. Jesse, could you please provide setting.xml files (both from maven dir and user dir) you use during successful maven repo generation? Thank you.

During the repository generation by nbm-maven-plugin, an SHA1 string is generated for the external libraries and the class org.apache.lucene.search.IndexSearcher tries to find it out. No record is found for some libraries like plexus-interpolation etc. However some entries for example ext/jaxws22/gmball-api-only.jar or ext/groovy-all.jar are found. Those libraries are not part of the maven repository directory "external" as it is expected.

Any ideas where I could continue with investigation?

(In reply to comment #14)
> No record is found for some libraries like plexus-interpolation

Do the JARs on local disk precisely match those found in Maven Central? Are you sure you are using a recent index of Central?

I use https://nexus.codehaus.org/content/groups/public/.index/nexus-maven-repository-index.zip
Is it a correct file?

(In reply to comment #16)
> https://nexus.codehaus.org/content/groups/public/.index/nexus-maven-repository-index.zip

As specified in the mojo docs (see BZ URL), use http://repo1.maven.org/maven2/.index/nexus-maven-repository-index.zip for the official Maven Central repository index.

The file http://repo1.maven.org/maven2/.index/nexus-maven-repository-index.zip is not accessible. Does any mirror exist?

(In reply to comment #18)
> http://repo1.maven.org/maven2/.index/nexus-maven-repository-index.zip
> is not accessible.

Depends on the HTTP client. They block access for web crawlers, but last I checked you can download it from Firefox. http://repo1.maven.org/maven2/.meta/repository-metadata.xml also shows some mirrors available; http://uk.maven.org/maven2 is official.

> Depends on the HTTP client. They block access for web crawlers, but last I
> checked you can download it from Firefox.

I have tried both Firefox in Windows XP and Safari in Mac OS X but the URL http://repo1.maven.org/maven2/.index/nexus-maven-repository-index.zip is forbidden.

After playing with mirrors, I have find out the following URL http://mirrors.ibiblio.org/pub/mirrors/maven2/dot-index/ so I have used the file http://mirrors.ibiblio.org/pub/mirrors/maven2/dot-index/nexus-maven-repository-index.zip but I'm still not sure if this is a correct one.

Anyway, the  build result of the last maven NetBeans repository with the mentioned index file is available at http://bits.netbeans.org/download/trunk/maven-snapshot/ . It looks good from my point of view but somebody should check or test it.

Well it's got org.codehaus.plexus:plexus-interpolation:1.14 now so that's good. Referring to org.netbeans.external:maven-repository-metadata-3.0.3:SNAPSHOT but that is likely because the 3.0.3 release was very recent and is not yet mentioned in the downloadable index (updated weekly AFAIK). So I think it's working correctly.

Bug #195301 still open.

The issue is fixed and the maven repository with 70-beta2 artifacts has been released.

Verified fixed. RELEASE70 is correctly available from the maven repo. Building our project with RELEASE70 works correctly.
Thanks for the great work!

I think escay was verifying something else. But anyway I can confirm that this looks fixed; there are inappropriate entries from org.netbeans.external but they are either (1) coming from earlier releases before this fix; (2) actually due to NB modules using nonstandard copies of common artifacts (such as Xerces).