Bug 202041

(In reply to comment #0)
> Could some consideration be given to to syncing bits.netbeans.org/maven2
> to central.

Has been considered. The potential problems I can think of:

1. The NB repo is pretty big. How big, I am not sure, but it might significantly add to the size of Central.

2. A lot of org.netbeans.*:*:* artifacts are not "clean" in terms of their dependencies on third-party libraries, but rather dump the library in the org.netbeans.external group. This happens when the library is not present in Central, for whatever reason; when we use a slightly different JAR than was published to Central, as seems to be the case for Xerces for example; or when the repository update script was broken and did not pass in the Central index correctly, as happened for several versions including RELEASE70 (now corrected again).

3. Even when external libraries are correctly referenced - so that the SHA-1 hash of a JAR in Central is successfully matched against a NB module's Class-Path extension, and the generated *.pom for the module is given a <dependency> on the proper coordinates - the *.nbm artifact in the repo will still include a copy of the external library. The same is true for JARs (or really any files) which are present in the *.nbm even when not Class-Path extensions, e.g. library definitions bundled for use in Ant-based user projects. Besides adding bloat to the repository, this practice may be objectionable in principle to the Central maintainers. This could possibly be solved by making nbm:populate-repository check the hash of each netbeans/** entry in an incoming NBM (possibly excluding the modules/*.jar primary artifact) and replacing with an *.external placeholder using the 'm2' URL protocol (both interpreted by NB 7.0+ but not yet nbm-maven-plugin). One catch is that the NBM would have to be resigned, assuming it was not originally created using *.external entries (currently used just for JUnit and JSR 296).

4. nbm-maven-plugin is potentially buggy in its usage of *.nbm vs. *.jar artifacts. The *.jar (naked module, no Class-Path header) is intended to be the primary artifact, as this is what is compiled against. The *.nbm (complete package used by nbm-application projects) is intended to be a secondary artifact, yet it has no classifier, and the "type" is sometimes mysteriously given as 'nbm-file' (rather than 'nbm'). Whether this situation is truly supported by Maven is unclear, but I know that we have had to specially customize the Maven Indexer [1], as well as IDE code [2], to deal with it. MavenNBM4 [3] discusses this. If artifacts in the current structure are published to Central, which presumably uses a vanilla Indexer config, the index might not work correctly.

Of course the first and third concerns, and some of the second, would not apply if we only published _new_ releases to Central, keeping bits.netbeans.org/maven2 around for historical releases.

> using netbeans artifacts more difficult as central is now frowning on
> projects that include <repository> snippets in their poms.

Good point; open-source Maven projects based on the NB Platform may want to publish APIs or even the entire application to Central.


[1] https://bitbucket.org/jglick/nexus-for-netbeans/raw/default/src/main/java/Main.java
[2] https://hg.netbeans.org/core-main/raw-file/default/maven.indexer/src/org/netbeans/modules/maven/indexer/NexusRepositoryIndexerImpl.java#CustomArtifactContextProducer
[3] http://wiki.netbeans.org/MavenNBM4#Dependency_management_and_project_structure

point number 2 is the main reason why I never considered moving the repo to central. I think if we were to merge it, we would have to replace the automatic process of SHA1 matching with a manual one that matches the artifact to correct coordinates. If these coordinate don't exist, we would have to push the lib to central beforehand.

I think point 3 is ok, osgi does something similar imho or any artifact processsed through maven-shade-plugin or assembly plugin does as well.

point 4 is orthogonal imho. the problem is that packaging is used as primary artifact's extension, unless declared otherwise in extension plugin, we need to override it to jar in order to have compilation dependencies work (cannot depend on nbm file for compilation). the nbm-file extension type is there to declare deps with extension nbm (nbm type already taken for the jar file). the problem there is that the extensions plugin needs to be always declared in the project pom, without it maven will look for nbm-file extension instead of nbm. (All statements are just from what I recall, could be outdated or wrong)

Historical: http://blogs.codehaus.org/people/mkleint/2008/12/nb-65-artifacts-in-a-maven-rep.html

open-source Maven project based on the NB Platform point of view.

We are currently publishing our application modules as maven artifacts on maven central. Some of our modules could easily be used by other netbeans platform based applications (like our map/geoviewer). Maven central provides a convenient and very accessible way to make it available for other users.
Our update center ( generated catalog.xml) refers to the modules in the maven central repository. 
With this approach we face 3 inconveniences:

1 - netbeans artifacts are not present in maven central, so they need to be manually removed from the generated catalog.xml. This manual step in the release procedure is not very time efficient and error prone.
2 - All artifacts in maven central are signed. Without signing they are simply not accepted. Still the update center does not recognize them as being signed.

Moving netbeans artifacts to maven central would solve 1 for us.
I don't know what is causing 2. 

Since the nbm vs jar issue was touched in previous comments, wouldn't it be possible to "assume" a default nbm descriptor for plain jar dependencies? If the artifact maintainer did not publish an additional nbm artifact, it would be safe to assume all packages are public for example. This would make the use for library wrappers obsolete. A lot of new platform application builders run into classloader problems when they use some dependency in a second module. It might also ease the problems with external artifacts.

(In reply to comment #4)
> wouldn't it be
> possible to "assume" a default nbm descriptor for plain jar dependencies?

Not a good idea, I think. Anyway that would have no effect on this issue.

The primary problem remains the creation of automated external library maven coordinates.

before netbeans can be in central, all it's dependencies need to in central first. See http://maven.apache.org/guides/mini/guide-central-repository-upload.html

if we were to release the binaries to central, rethinking the cluster artifacts and their usage of "nbm-file" dependency type would be worthwhile. This way depending on the cluster artifact only makes sense from a project that declares nbm-maven-plugin with the <extensions>true</extensions> declaration.

afaik clusters are the only place where it's effectively used, the "nbm" type is most probably also used just rarely (referring to the dependency jar file without transitive dependencies)

(In reply to comment #1)
> making nbm:populate-repository check the hash...

https://jira.codehaus.org/browse/MNBMODULE-170

Here is another (albeit transient) reason *not* to integrate with Central Repo at least for now.

The stability of Central Repo has increased over the years. Currently the actual artifacts are hosted on servers in US and UK and users are automatically directed to the server closest to them and with least load. This is all very good and stable.

However the repo index (corresponding to the .index/ sub-dir) is another matter. This is hosted at mirrors.ibiblio.org no matter where you come from. Unfortunately ibiblio have great problems keeping their servers floating. The index as such is generated once per week but it is pretty difficult - if not impossible - to get to download the index because of the problems at ibiblio.org.

Sonatype say they are taking steps to do something about the matter but it will take some time. I guess the attitude is that the index is not that important.

The Netbeans IDE depends on the index being available in particular for all the right-click-to-search functionality and what have you. 

I think Sonatype first needs to prove that they can run a stable infrastructure. The existing Netbeans Repo is very stable and has no problems for those wishing to download the index or the artifacts. I believe Sonatype are doing a great job and I truly hope they have the funds to create a great Central Repo infrastructure .... in the future. As it stands currently I would hesitate to rely *only* on Central Repo.

(In reply to comment #9)
 
mostly irrelevant, everyone depends on the index of central for any other dependency. In case you are using a repository manager (as everyone is encouraged to do), the index is proxied as well. If you are using a repository manager, the entire discussion is futile, you are already proxying the nb repo and are merging it with central content anyway..