This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 231443 - Provide support for libraries that do not contain bundled jar files and only specify Maven coordinates
Summary: Provide support for libraries that do not contain bundled jar files and only ...
Status: RESOLVED WONTFIX
Alias: None
Product: projects
Classification: Unclassified
Component: Libraries (show other bugs)
Version: 7.3
Hardware: All All
: P3 normal with 1 vote (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on:
Blocks: 231700
  Show dependency tree
 
Reported: 2013-06-18 13:08 UTC by Petr Jiricka
Modified: 2016-07-07 08:38 UTC (History)
8 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Jiricka 2013-06-18 13:08:10 UTC
It would be useful if it was possible to programmatically use Java libraries that are defined in terms of Maven coordinates, and that do not contain bundled jar files. It should be possible to add these libraries to the classpath of both Ant and Maven projects. Some concrete situations where this could be useful are:

Bug 230867 - outdated Hibernate
Enhancement 231440 - additional JPA providers
Some follow-up issues after implementing enhancement 229898 - Jersey 2.0

For these use cases, it is NOT a requirement that the library is visible in Tools | Libraries, and that the user is able to customize it.

Regarding the actual implementation, I can imagine two basic approaches:
1. Client code creates a library for example using o.n.m.java.api.common.util.CommonProjectUtils.createJavaLibraryImplementation, and passes it to ProjectClassPathModifier.addLibraries. Ant projects would need to internally understand that the local jars do not exist, download them from the Maven repository (maybe using some friend contract with the Maven support), and then add the downloaded jars to the classpath.

2. There is a new utility method for use by the client code that adds an artifact identified by Maven coordinates to the project. The implementation of this utility method determines if the project is Ant-based or Maven based. For Ant-based project it downloads the jars from the Maven repository (maybe using some friend contract with the Maven support), and then adds the downloaded jars to the classpath using ProjectClassPathModifier.addRoots. For Maven projects it creates a library from the Maven coordinates using o.n.m.java.api.common.util.CommonProjectUtils.createJavaLibraryImplementation, and passes it to ProjectClassPathModifier.addLibraries.

And then of course the devil is in the detail (for both approaches above):
- displaying progress UI during the download
- handling download failures
- handling proxy settings
- etc.

What do you think?
Comment 1 Tomas Zezula 2013-06-18 13:10:43 UTC
There will be several UI questions like should this libraries be displayed in the LibraryManager?
Should the content be downloaded every time or just once (for Ant project)? Where the content should be stored for non shareable Ant projects? etc.
Adding Petr Somol to cc.
Comment 2 Sergey Petrov 2013-06-18 13:16:21 UTC
It may be nice to have ability to set some properties for these libraries as it's not possible to scan resources without jars, it may be nice to have some properties like 'jpa=yes' or 'jpa_provider=2.1' and get these properties instead of hardcode libraries names. On other side we can ask user to follow some naming convention in case if it's user created maven only library.
Comment 3 Petr Jiricka 2013-06-18 13:44:44 UTC
> There will be several UI questions like should this libraries be displayed in
> the LibraryManager?

I agree these UI questions are important. For the use cases I am aware of, this kind of library does not need to be (and probably should not) be displayed in LibraryManager.

> Should the content be downloaded every time or just once (for Ant project)?

I am thinking if we could use Maven's local cache? I.e. Maven would download the library to .m2/repository once, and then this cached version could be used repeatedly. Milos?

> Where the content should be stored for non shareable Ant projects?

Yes, this is a big question, ideas welcome. The two options that I can think of are:
- $project_dir/netbeans_libs (or some similar directory inside project with a hardcoded name)
- $user_dir/modules/ext

> properties like 'jpa=yes' or 'jpa_provider=2.1'

I believe this should be  doable using Library.getProperties(), no?
Comment 4 Milan Kuchtiak 2013-06-18 15:07:25 UTC
I see quite important difference between Ant and Maven library.

Ant library, in Netbeans, is the set of jar files, where Maven library is set of artifacts. When the Maven artifact is downloaded to local cash it's downloaded together with all dependencies, so one Maven artifact can result in several jar files.

So,something like ability to specify library as:

1. set of Maven repository jar files,(from local Maven Repository) - following the Ant library concept
2. set of (root) artifacts (Maven concept)

The (2) should be somehow "passed?" for Maven to download the artifacts..
The (1) could be used for library definition (for Ant library)
Comment 5 David Konecny 2013-06-19 02:52:51 UTC
> Where the content should be stored for non shareable Ant projects?

I'd think the files can stay in Maven local cache and build.properties would just point directly to them. That could work no?
Comment 6 Milos Kleint 2013-06-19 06:25:53 UTC
(In reply to comment #5)
> > Where the content should be stored for non shareable Ant projects?
> 
> I'd think the files can stay in Maven local cache and build.properties would
> just point directly to them. That could work no?

Would mean some sort of env var to point to the maven local repo root is necessary?
I believe it would be easier to just copy to IDE designated area.

and what does non-shareable mean in this context? you mean project that only has the global Ant libraries referenced (vs. Ant project with it's own designated libraries location)? Even such projects with global Ant libs will have to be shareable across users. How will user B get the right jars locally when user A uses this "maven" lib?
Comment 7 Petr Jiricka 2013-06-19 07:39:37 UTC
> some sort of env var to point to the maven local repo root is necessary?

Could Tools | Ant Variables be used for this?

> How will user B get the right jars locally
> when user A uses this "maven" lib?

You are right, this needs to be supported by the Broken References infrastructure. The good thing is that this infrastructure already has support for Ant Variables I believe, which gets us half way there.
Comment 8 Milos Kleint 2013-06-19 07:44:13 UTC
(In reply to comment #7)
> > some sort of env var to point to the maven local repo root is necessary?
> 
> Could Tools | Ant Variables be used for this?

I suppose.

> 
> > How will user B get the right jars locally
> > when user A uses this "maven" lib?
> 
> You are right, this needs to be supported by the Broken References
> infrastructure. The good thing is that this infrastructure already has support
> for Ant Variables I believe, which gets us half way there.

Yes, but somehow it leads me to believe that the maven GAV artifact coordinates need to be somehow persisted on the ant side of things so that we could re-trigger the maven binary download on the user B computer..
Comment 9 Petr Jiricka 2013-06-19 08:38:32 UTC
Issue 231410 is another case where this enhancement could be useful, cc'ing Martin Fousek.
Comment 10 Milan Kuchtiak 2013-06-19 09:13:47 UTC
Petr. J. wrote me he didn't understand my comments:

> So,something like ability to specify library as:

> 1. set of Maven repository jar files,(from local Maven Repository) 
>   - following the Ant library concept
> 2. set of (root) artifacts (Maven concept)

What I meant to say, was, for Ant case, the library provider would like to specify just subset of all jars (that are transitively downloaded) from Maven repositories.

Example:

Java EE 7 Ant Web application already has JAX-RS API on classpath.
"org.glassfish.jersey.media:jersey-media-moxy:2.0" depends on 
"javax.ws.rs-api:jax-ws.rs:2.0"(JAX-RS API), but library provider wouldn't like to include this artifact into library definition for Ant.
Comment 11 Petr Jiricka 2013-06-19 13:37:14 UTC
> Yes, but somehow it leads me to believe that the maven GAV artifact coordinates
> need to be somehow persisted on the ant side of things

Yes, you are right. So for this reason, using a Library may not be the best option for Ant, because while Library supports storing Maven coordinates, these are not shared among users (via version control), right? Also Library does not seem to support Ant variables - I can not have ${var.MAVEN_LOCAL_REPO}/path/to/my/file.jar in a library.

Rather, we may want to use just jar files in Ant projects, as these support references with variables. As for persisting the Maven coordinates, can we use AuxiliaryProperties? I.e. we would store mavendeps and mavenrepos as auxiliary properties, and if the broken references support in Ant project encounters these, it would set the MAVEN_LOCAL_REPO variable, and attempt to download of all the mavendeps. Would that work?
Comment 12 Petr Somol 2013-06-19 19:08:37 UTC
CC-ing David and UI team.
Comment 13 Tomas Zezula 2013-06-19 20:49:13 UTC
In reply to comment #2: Yes, it's possible from NB 7.3 to assign properties to the library.
In reply to comment #5: Yes, pointing from build.properties to local Maven repo will work fine.
In reply to comment #6 and #11. The Ant variable can be used, however currently the global library definition does not support them 9can be extended). Or more easily special URL handler (similar to nbinst) can be created for it. So the ${var.MAVEN_LOCAL_REPO}/path/to/my/file.jar would become mvm://path/to/my/file.jar.
Comment 14 David Konecny 2013-06-19 23:52:12 UTC
> and what does non-shareable mean in this context? you mean project that only
> has the global Ant libraries referenced (vs. Ant project with it's own
> designated libraries location)? Even such projects with global Ant libs will
> have to be shareable across users. How will user B get the right jars locally
> when user A uses this "maven" lib?

Global "maven" lib is not different from global "ant" lib, no? During IDE startup userdir/build.properties gets updated with absolute paths. For Ant libraries these paths point somewhere into IDE installation. And for Maven libraries it would be user's local Maven cache. As long as user B has all required jars in their local Maven cache everything should work.
Comment 15 Petr Jiricka 2013-06-20 12:32:42 UTC
> So the ${var.MAVEN_LOCAL_REPO}/path/to/my/file.jar 
> would become mvm://path/to/my/file.jar.

Interesting idea. So how about a contract like this?

/**
 * Creates a {@link LibraryImplementation3} that is created based on Maven 
 * coordinates, but may be later used with both Ant and Maven based Java 
 * projects. May not be called in AWT thread, as it may block for a long 
 * time to download files from remote repositories.
 * @param name library name
 * @param mavendeps list of maven dependencies in the form of 
 *    groupId:artifactId:version:type, for example 
 *    org.eclipse.persistence:eclipselink:2.3.2:jar
 * @param mavenrepos list of maven repositories in the form of layout:url, for
 *    example default:http://download.eclipse.org/rt/eclipselink/maven.repo/
 * @param resolve if true, the implementation attempts to download the 
 * specified dependencies from the Maven repository(-ies), and after 
 * downloading it populates the classpath part of the library with a set of 
 * URLs in format mvn://path/to/my/file.jar, corresponding to files downloaded
 * to the local Maven repository (including any additional dependencies) 
 * relative to the local repository root.
 * @return {@link LibraryImplementation3} representing the information passed 
 *    as parameters
 */
public static LibraryImplementation3 createJavaLibraryImplementationFromMaven(
        @NonNull final String name,
        @NonNull final String[] mavendeps,
        @NonNull final String[] mavenrepos,
        boolean resolve)
        
TODO1: How to handle network failures? Throw an exception? Or display a "Retry"
  dialog and return null if the user presses cancel in this dialog?
  
TODO2: What to do about downloading sources and javadoc?

Various use cases would then be solved as follows:

1. Client wants to create a library and add it on the classpath: Create library
   with resolve=false and add using CP Modifier. Maven (Gradle) project just 
   adds the dependencies. Ant project first downloads to local Maven repo. If 
   it's sharable, copies the deps to libraries dir and stores file: URLs in 
   project properties. If it's not sharable, it stores mvn: URLs in project 
   properties, plus absolute file URLs in private properties (or rather 
   build.properties in userdir).

2. User opens an Ant project created by someone else which contains mvn: URLs:
   Ant project displays a broken reference warning, and as a fix offers to 
   download them to the local Maven repo. Then it creates absolute file URLs
   in private properties (or rather build.properties in userdir).
   
3. Adding a subset of files to Ant (comment #10): Client needs to know if it's
   using Ant project or Maven (Gradle) project. If it's Maven project, creates
   library with resolve=false and passes it to CP Modifier. If it's Ant 
   project, it creates library with resolve=true. In the returned library it
   removes classpath entries as necessary and passes it to CP Modifier.

4. Adding additional properties (comment #2): Create library with resolve=false
   and set the properties. Then pass to CP Modifier.
Comment 16 Milan Kuchtiak 2013-06-24 08:57:13 UTC
I've created a similar enhancement: Ability to create User Maven Library type

See the bug 231700.
Comment 17 Martin Balin 2016-07-07 08:38:43 UTC
This old bug may not be relevant anymore. If you can still reproduce it in 8.2 development builds please reopen this issue.

Thanks for your cooperation,
NetBeans IDE 8.2 Release Boss