Bug 47498 - Provide a way to directly specify source and Javadoc information for CP entries in a freeform project
Provide a way to directly specify source and Javadoc information for CP entri...
Status: RESOLVED FIXED
Product: java
Classification: Unclassified
Component: Freeform
4.x
All All
: P2 with 3 votes (vote)
: 7.1
Assigned To: Tomas Zezula
issues@java
http://projects.netbeans.org/buildsys...
PLAN
:
: 46055 47163 48025 48994 49769 49770 95439 119790 (view as bug list)
Depends on: 44035 51151 200698
Blocks: 42682 89629
  Show dependency treegraph
 
Reported: 2004-08-19 16:53 UTC by Jesse Glick
Modified: 2011-10-19 15:29 UTC (History)
8 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments
Changes to project/libraries to support library creation. (10.73 KB, patch)
2005-01-13 14:00 UTC, Tomas Zezula
Details | Diff
Patch od java/freeform module (32.54 KB, patch)
2005-01-13 21:05 UTC, Tomas Zezula
Details | Diff
Proposed schema (10.46 KB, application/octet-stream)
2011-06-03 08:50 UTC, Peter Nabbefeld
Details
Basic (non-working) changes to JavadocQuery for proposed change (6.79 KB, text/x-java-source)
2011-06-03 08:51 UTC, Peter Nabbefeld
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2004-08-19 16:53:43 UTC
Currently in a freeform project, you can associate
source and Javadoc information with classpath
entries (JARs) only indirectly - by creating a
Java library using the same JAR, and specifying
source and Javadoc information for that.

Would perhaps be nicer to be able to specify such
information in the freeform project itself, and
have the project service requests to SFBQ and
JFBQ. This would work better in the freeform UI,
and would enable the source and Javadoc
information to be shared among all users of the
project automatically.

Means having a UI for adding such info which would
partially duplicate the UI in java/j2seplatform,
which is not so nice, esp. for Javadoc since the
checking of a valid base URL is not trivial.
Comment 1 Jesse Glick 2004-08-19 17:00:25 UTC
*** Issue 47163 has been marked as a duplicate of this issue. ***
Comment 2 Jesse Glick 2004-08-19 17:01:21 UTC
*** Issue 46055 has been marked as a duplicate of this issue. ***
Comment 3 Jesse Glick 2004-08-19 17:05:12 UTC
Configuring sources may also be tricky since you may need to provide a
SourceLevelQuery and ClassPathProvider for those as well, in case the
user steps through them during debugging and expects things like Go To
Declaration to work from those sources. I think j2seplatform handles
SLQ by checking for bytecode version in the binaries - not something I
really want to duplicate.
Comment 4 Jesse Glick 2004-08-27 02:16:58 UTC
*** Issue 48025 has been marked as a duplicate of this issue. ***
Comment 5 Tim Lebedkov 2004-08-27 11:07:10 UTC
Jesse Glick wrote:
> Currently in a freeform project, you can associate
> source and Javadoc information with classpath
> entries (JARs) only indirectly - by creating a
> Java library using the same JAR, and specifying
> source and Javadoc information for that.
Could you please tell me where I can specify the libraries my 
freeform project should use?
Comment 6 Jesse Glick 2004-08-30 23:45:11 UTC
Tools -> Library Manager as usual.
Comment 7 Petr Jiricka 2004-09-13 17:33:29 UTC
One other issue that should be addressed is that if you (even in an
ordinary non-freeform project) add a jar file to the project
classpath, and later realize that you also have sources and javadoc
for this jar file, you have to define a library and specify again the
jar files with binaries, although you've already specified it once. It
should be more straightforward to do that. 
Comment 8 Jesse Glick 2004-09-13 19:02:32 UTC
To Petr's last comment: unrelated to this issue, and probably WONTFIX
anyway, but should be discussed separately.
Comment 9 Jesse Glick 2004-09-14 18:12:17 UTC
*** Issue 48994 has been marked as a duplicate of this issue. ***
Comment 10 Tim Lebedkov 2004-09-27 18:34:39 UTC
I hope that project dependencies will also be handled by the
implementation of this enhancement.
Comment 11 Jesse Glick 2004-09-28 16:49:53 UTC
lebedkov: project dependencies are a separate matter. See issue #49640.
Comment 12 Jesse Glick 2004-10-04 17:15:25 UTC
*** Issue 49770 has been marked as a duplicate of this issue. ***
Comment 13 Jesse Glick 2004-10-04 17:16:36 UTC
*** Issue 49769 has been marked as a duplicate of this issue. ***
Comment 14 robertmcauliffe 2004-10-04 22:12:56 UTC
Uh, I'm just a lowly user... 

"project service requests to SFBQ and JFBQ"?

"UI in java/j2seplatform"?

Could someone please explain in userspeak what this feature request
specifies?

does this correspond to the main thrust of 49770 (as opposed to the
alternatives), or is that what 49677 is about?

(trying to determine if this gets at the intent of the issues marked
as duplicate)
Comment 15 Jesse Glick 2004-10-05 16:33:44 UTC
Basically this is about being able to specify, in the GUI as well as
project.xml, locations of sources and Javadoc corresponding to
classpath entries in a freeform project. Only meaningful for entries
which are binary JARs (libraries) - if they come from other projects,
you don't need to do this, because the other project would already be
advertising the locations of sources and Javadoc. (Which is only
currently broken in the case of Javadoc location from a freeform
project - for which see issue #49677.)

In 4.0, you can simply make libraries to match your project's
classpath entries, and associate sources and Javadoc that way. But the
association is not then sharable - it is not kept as part of the
project as such. That should preferably be fixed somehow, in case the
user is keeping a copy of lib sources and/or Javadoc inside the
project and versioning it - then a different user checking out the
project ought not need to reconfigure libraries. Also it is not
obvious to set up libraries to match the freeform project's classpath
entries. That is the main thrust of this RFE - provide a special kind
of library management for use in freeform projects.

TBD whether the existing library management GUI can be somehow reused
rather than duplicating it. Issue #44035 may be related in this regard.

Re. issue #49770 - this is only related to use #1 and sort of to
alternative #1. Use #2 and alternative #2 already work, modulo #49667,
e.g. they should work if the subproject is a j2seproject. Use #3 is
separate since only a parent project uses this project's Javadoc, and
is thus essentially the same as #49667.
Comment 16 Jesse Glick 2004-10-20 19:10:25 UTC
*** Issue 46482 has been marked as a duplicate of this issue. ***
Comment 17 David Konecny 2004-11-03 15:00:23 UTC
If information is stored in freeform itself then this is potentially
API change of project.xml format.
Comment 18 Jesse Glick 2004-12-02 20:53:49 UTC
Hopefully for 4.1; published UI spec expected soon.
Comment 19 Jesse Glick 2005-01-11 00:49:22 UTC
So for E it was suggested to not touch the project.xml, i.e. not make
the src/javadoc info sharable per classpath entry, as this is simpler
and probably enough for most users. (Would only make a difference to
people who keep src and/or javadoc inside a versioned area of the
project directory structure and wish to specify this info once for all
users. But probably most people do not version sources or javadoc of
external libs, even if they version the binary libs themselves.)

See URL for proposed user interface. Pretty simple; basically does the
same thing as creating a library using the Library Manager, but avoids
"polluting the library namespace", and is simply easier (no need to
reenter the JAR path, fewer things to click).

Tomas do you have any time to work on this? I could work on it as
well, but you are probably most familiar with how much can be reused
from the library manager impl. (It is not strictly necessary to reuse
*anything* from the library manager impl; could also be implemented
entirely in java/freeform.)
Comment 20 Tomas Zezula 2005-01-11 08:34:24 UTC
I am not sure If I will get to this before feature freeze (18/1/2005).
Firstly I need to integrate all the API enhancements.
I will let you know tomorrow.
Comment 21 Tomas Zezula 2005-01-12 12:41:36 UTC
I've looked at UI spec, it seems simple, but I have several
implementation questions. There are several ways how to implement it.
1) FreeForm specific - implement it in FreeForm, not nice - duplicates
code.
2) Using the Libraries module - it will create the j2se library. In
this case should be the library accessible from
LibraryManager.getLibraries() API (probably yes), should be visible in
the Library Manager (probably no)? It will probably need API
enhancements, since there is no public API for creating library. If
the library should not be visible it requires either extension to
library definition or to define new folder where the non visible
libraries are stored.
Comment 22 Jesse Glick 2005-01-13 01:37:51 UTC
I agree #2 is probably the better solution for now; agree on all
points you mention. Some new kind of API is probably needed.
Comment 23 Tomas Zezula 2005-01-13 14:00:20 UTC
Created attachment 19671 [details]
Changes to project/libraries to support library creation.
Comment 24 Tomas Zezula 2005-01-13 14:05:41 UTC
I've attached a patch of project/libraries module to support adding
and removing of libraries. There is no API for library modification at
this time, since I don't want to make the Library API object mutable
or let the client to obtain the SPI LibraryImplementation object.
There should be probably something like LibraryManager.updateLibrary
(????).
But for now the the project customizer can update library by remove + add.
Comment 25 Tomas Zezula 2005-01-13 21:05:55 UTC
Created attachment 19685 [details]
Patch od java/freeform module
Comment 26 Tomas Zezula 2005-01-13 21:17:26 UTC
I've added patch of java/freeform. The UI is not complete - no
accessibility, no enabling/disabling of edit button, JFileChoosers
don't remember last used locations.

But I've found another serious problem, the queries which return
Result. The customizer creates library for an archive file which was
already on the project classpath (it was scanned by javacore and
SFBQ.Result was cached for it, the result is EMPTY_RESULT). Now the
librarie's SFBQI starts to provide new SFBQ.Result for it, but it has
no way how to let the javacore know about it. The IDE needs an restart
to refresh the javacore cache.
There is an solution which may result in big memory consumption, there
should be an SFBQI registered as a last one, which provides for each
unrecogized archive root SFBQ.Result listening on the LibraryManager.
Comment 27 Jesse Glick 2005-01-14 20:35:26 UTC
My other suggestion (offline), which may also result in increased
memory consumption, is similar but slightly different - to make SFBQ
itself produce a proxy over all (non-null) SFBQ.R's returned by
available SFBQI's. Would still require the j2seplatform SFBQI to
provide a non-null result for every JAR, but does not require it to be
in any particular position in lookup.

I presume this problem existed before, in that creating a library for
an existing classpath entry would not refresh javacore's cache. The
difference now is that we are promoting a UI which triggers the bug.

Possible workaround: in the freeform customizer code, whenever a fake
library is registered (or any other similar change is made), remove
and then readd the same classpath item (according to the relevant
ClassPath), in an attempt to force an event to be fired and thus to
force javacore to refresh its cache. Could that work?
Comment 28 Tomas Zezula 2005-01-16 10:54:09 UTC
Yes, the workaround works fine. I am using it for now, but I was not
sure if it is not too ugly.
Thanks
Comment 29 Tomas Zezula 2005-01-17 16:48:39 UTC
This issue will be fixed in promo-D. It requires API changes.
Comment 30 Tomas Zezula 2005-01-17 16:49:30 UTC
Sorry, not promo-D but promo-F
Comment 31 Jesse Glick 2005-01-17 17:31:14 UTC
Which API changes does it still require? Did the suggested workaround
not work after all? Please explain.
Comment 32 Tomas Zezula 2005-01-18 09:03:50 UTC
The addLibrary (), removeLibrary () methods in the LibraryManager. And
LibraryFactory class. See the libraries.diff.
Comment 33 Jesse Glick 2007-02-13 17:14:47 UTC
*** Issue 95439 has been marked as a duplicate of this issue. ***
Comment 34 Jesse Glick 2007-04-11 21:57:32 UTC
Tentative plan.
Comment 35 Jesse Glick 2007-10-23 23:58:53 UTC
*** Issue 119790 has been marked as a duplicate of this issue. ***
Comment 36 Milos Kleint 2008-09-10 07:27:29 UTC
I suppose the freeform project type can do what j2se project type does nowadays with declaring javadoc/source for jar
files (instead of libraries)
Comment 37 Antonin Nebuzelsky 2010-01-11 04:28:32 UTC
Changing the default component owner to tzezula.
Comment 38 Peter Nabbefeld 2011-05-27 06:49:10 UTC
This issue is already about seven years old! As infrastructure was greatly improved in the last years, it should be too difficult to fix.

It should also considered to categorize this issue as a defect, as some basic IDE functionality is missing, IMO.
Comment 39 Tomas Zezula 2011-05-31 16:14:42 UTC
In reply to comment #38.
Unfortunately it's not the case. The infrastructure has improved for projects which are based on J2SEProject but this is not a case of the Freeform.
Also the Freeform project is not a main focus now :-(
Comment 40 Peter Nabbefeld 2011-06-03 08:47:22 UTC
(In reply to comment #39)
> In reply to comment #38.
> Unfortunately it's not the case. The infrastructure has improved for projects
> which are based on J2SEProject but this is not a case of the Freeform.
> Also the Freeform project is not a main focus now :-(

Thank You for interpreting my comment correctly, as I've accidently dropped the important word "not"!

I've tried to implement a fix in Java freeform project, but it doesn't work at all, probably I'm just missing some dependencies. However, I'll attach my changes as a proposal (i.e. schema and basic impl.).
Comment 41 Peter Nabbefeld 2011-06-03 08:50:28 UTC
Created attachment 108687 [details]
Proposed schema
Comment 42 Peter Nabbefeld 2011-06-03 08:51:33 UTC
Created attachment 108688 [details]
Basic (non-working) changes to JavadocQuery for proposed change
Comment 43 Tomas Zezula 2011-06-03 12:00:25 UTC
Unfortunately the fix is not so simple.
1st) The schema cannot be changed as it's non compatible API change. Either the new schema has to be introduced and the upgrade from previous versions has to be added as there is for /2 and /3 versions. Or another persistence like AuxiliaryProperties (AuxiliaryCfx).

2nd) The Freeform JFBQuery cannot be used as it's called only for freeform artifacts.

I will try to save some time and do the infrastructure. Without changing the schema as the update code is really pain. I did it for annotation processing support and it was not very nice.
Comment 44 Peter Nabbefeld 2011-06-03 12:03:52 UTC
(In reply to comment #43)
> Unfortunately the fix is not so simple.
> 1st) The schema cannot be changed as it's non compatible API change. Either the
> new schema has to be introduced and the upgrade from previous versions has to
> be added as there is for /2 and /3 versions. Or another persistence like
> AuxiliaryProperties (AuxiliaryCfx).
> 
> 2nd) The Freeform JFBQuery cannot be used as it's called only for freeform
> artifacts.
> 
> I will try to save some time and do the infrastructure. Without changing the
> schema as the update code is really pain. I did it for annotation processing
> support and it was not very nice.

Thank You! But I don't understand, why You cannot extend the schema, as additional element would be optional, so I'd expect them to work properly with existing projects?
Comment 45 Tomas Zezula 2011-06-03 12:10:01 UTC
Here is an example when the compatibility will be broken:
User_1 has NB 7.0 with original schema
User_2 has NB de with changed schema

User_2 creates a new Freeform with the optional new element. IDE validates the XML according to schema and everything is OK.
User_2 shares the project and User_1 downloads it
User_1 tries to open it. As the schema is /3 the IDE (7.0) i happy because it's supported version and tries to validate the project.xml according the schema but now it finds an XML element which is not in the schema and the validation fails.
Comment 46 Peter Nabbefeld 2011-06-03 12:38:49 UTC
(In reply to comment #45)
> Here is an example when the compatibility will be broken:
> User_1 has NB 7.0 with original schema
> User_2 has NB de with changed schema
> 
> [...]

Well, I agree this might cause problems, if one user's IDE is upgraded but the other isn't. However, try to upgrade a project using Swing forms from NB 5.x to current version directly - AFAIK, it will fail. Had those problems earlier, could only resolve the upgrade problem using intermediate versions of NB. And I guess You won't be able to downgrade most project types.

However, it's a problem for me not being able to add libraries nor javadoc, as I need to use third-party libraries (different versions for different projects, so no chance to add them globally). Currently, I'm unpacking the javadoc and then I'm viewing it in my browser, but that's very inconvinient. :-(
Comment 47 Tomas Zezula 2011-06-03 13:22:57 UTC
In the projects infrastructure we tried to upgrade the project.xml only when it's needed. There is some support for it in J2SE based projects (UpgradeHelper) and something similar is in freeform project. The IDE does not do downgrade but it has to ensure at least that it generates the metadata according to schema. And the older version does not open project of newer version.
Also the change to schema will not pass through NB API review process as it's incompatible API change.

Yes I understand importance of this feature and agree that even when the Freeform is not main focus now it should be resolved.
Comment 48 Peter Nabbefeld 2011-06-03 13:28:26 UTC
(In reply to comment #47)
> Yes I understand importance of this feature and agree that even when the
> Freeform is not main focus now it should be resolved.

Thank You! Just because it is so difficult to do this later: Could You probably also consider to add some libraries locally? Doing this in the same working task would probably reduce organizational overhead ...
Comment 49 Peter Nabbefeld 2011-06-03 13:45:45 UTC
I've just noticed a related addition to http://wiki.netbeans.org/wiki/index.php?title=EditorPlan71:

Issue 199143 Javadoc & Sources for Freeform (Tomas, 2days) 

This entry seems to be related, but the bug id is incorrect.
Comment 50 Tomas Zezula 2011-06-03 13:49:37 UTC
Thank you for pointing me on it.
I've just added it into plan but used a wrong browser tab (the issue I am just fixing instead of this one).
I've fixed it. Thanks
Comment 51 Jesse Glick 2011-06-03 15:16:11 UTC
No need to touch the freeform schema; could be a separate project.xml section with its own schema.

Anyway I do not think anything special in freeform is needed; see comment #19. Would be useful for autoprojects etc. as well. Regardless of project type, you ought to be able to immediately associate local source and/or Javadoc with a given binary classpath entry; this association would not be persisted in project sources, only in your userdir somewhere (NbPreferences?). UI to pick source/Javadoc roots, persist them, and global SFBQ/JFBQ impls interpreting these associations should all be trivial. I am just not sure about the entry points for these UIs. I can think of

1. You invoke code completion on something and it says no sources or Javadoc are available. There should be a button to add sources or Javadoc right then, associated with that binary root.

2. Help > Javadoc References > More Javadoc... should have some button to pick a binary root from among those in GlobalPathRegistry with no existing Javadoc association, then select a Javadoc root.

Customizing or removing the associations afterwards is another issue that would need to be addressed. Also there should be some SPI so that modules which have a preferred place to store this information (e.g. Libraries) can do so.
Comment 52 Tomas Zezula 2011-08-01 09:27:43 UTC
In reply to comment #51.
In this case the binding of source to binary is completely project independent. If you close the project it's still active. Which is a bit problematic, but maybe it does not mind (having 2 projects with same jar with different binary<->source mapping is stange anyway). Also when you copy the project to someone you lose the binding, but this is probably OK (you will need to resolve broken ref anyway).

Regarding the UI. I suggest the same behavior as in case of standard ant project "Edit" button which calls some api method attachSources(....), attachJavadoc(...).

So I propose 2 simple methods
 attachSources
 attachJavadoc

If we want to resolve the problem with independence of binding on project an additional factory method is needed.

createExtraSources which creates on OpenProjectHook which needs to be put into the project's lookup which activates the bindings stored for example in NbPreferences ProjectUtil.getPreferences().
Comment 53 Jesse Glick 2011-08-01 14:04:32 UTC
(In reply to comment #52)
> In this case the binding of source to binary is completely project independent.
> If you close the project it's still active.

Indeed. (True even for project-defined associations so long as the project can be located somehow.)

> Also when you copy the project to
> someone you lose the binding

Right, this would just be for one-off associations that are tricky to add in more permanent ways.

> If we want to resolve the problem with independence of binding on project an
> additional factory method is needed.
> 
> createExtraSources which creates on OpenProjectHook which needs to be put into
> the project's lookup which activates the bindings stored for example in
> NbPreferences ProjectUtil.getPreferences().

Possible, though I am not sure this added complexity is worth it. Anyway you might want to make an association for a binary not owned by a project.
Comment 54 Tomas Zezula 2011-08-01 14:51:05 UTC
I've think about it a bit more and it seems that you are right.
Binding it to project is a complication which makes the thing worse.
It would be good if it will be generic enough to work out of the box for freeform, autoproject and NB library wrapper module (currently for lucene I have to have an additional j2seproject lucene with sources).

I looked at Eclipse and do something similar to what Jesse described. When you navigate to some class (CTRL+Click) and it has no sources a button "Attach sources" is shown. I will do something similar, the decompiler is under java.source control so it should not be a problem. In addition to this I will change the Javadoc Code Completion window to allow to attach Javadocs.

I will put the queries implementation + some small API attachSources(binRoot, sourceRoots*), attachJavadoc(binRoot, docRoots*) into java.project.

Any comments?
Comment 55 Jesse Glick 2011-08-01 15:54:31 UTC
(In reply to comment #54)
> I will put the queries implementation + some small API attachSources(binRoot,
> sourceRoots*), attachJavadoc(binRoot, docRoots*) into java.project.

CC apireviews when there is a patch ready to look at.

Also consider last sentence of comment #51. If the user invokes this UI on a binary root which is already capable of making source and/or Javadoc associations by existing means, e.g. a library, then the NbPreferences-based storage should not be used. This probably means that there needs to be an API+SPI similar in concept to ProjectClassPathModifier, with the preferences SPI impl being in last position, and earlier ones such as for LibraryManager.

It may also be valuable to let an SPI offer to _retrieve_ the source or Javadoc association, rather than having the user _specify_ it. In particular, in the case of JARs in the local Maven repository, associations will be offered from [SJ]FBQ when the secondary artifact has already been downloaded. If the artifact is a dependency of some open project, it is possible to request this download from the Projects tab UI. But sometimes you are coming across a local repo artifact in another context (perhaps as a classpath entry in a j2seproject), or it may simply be cumbersome to find the dependency node in Projects when your attention is focused on the editor window. So in this case "Attach Sources..." (resp. "Javadoc") should simply trigger the secondary artifact download, not bring up a filechooser.

Given these use cases, I might suggest something like:

package api;
class SourceJavadocAttacher {
  static boolean attachSources(URL root);
  static boolean attachJavadoc(URL root);
}
package spi;
interface SourceJavadocAttacherImplementation { // for @ServiceProvider
  enum SupportLevel {NONE, EXPLICIT, IMPLICIT}
  SupportLevel supportsAttachSources(URL root);
  SupportLevel supportsAttachJavadoc(URL root);
  void attachExplicitSources(URL root, URL[] sources) throws IOException;
  void attachExplicitJavadoc(URL root, URL[] javadoc) throws IOException;
  void attachImplicitSources(URL root) throws IOException;
  void attachImplicitJavadoc(URL root) throws IOException;
}

where the API methods return true on success, and display a filechooser only when the support level is EXPLICIT. (The chooser could offer a text field for a remote URL in the case of Javadoc.) Probably the Maven impls of the attachImplicit* methods would block until the download completes and show their own progress bar.

Also I think api.java is the correct place for the new API; it does not refer to projects as such, and is closely related to the existing queries. (An SPI impl using LibraryManager would belong in java.project.)
Comment 56 Tomas Zezula 2011-08-01 17:02:45 UTC
It's much more complex then the "freeform" case but it will hopefully solve the attach problems for all artifacts in all project types.
When I will have a diff I will attach it I just wanted to collect all use cases (the maven use cases were completely new for me).
Comment 57 Tomas Zezula 2011-08-11 13:22:48 UTC
Fixed jet-main http://hg.netbeans.org/jet-main/rev/a01b7480064e
When you click (Ctrl+B) on element which has no sources the disassembled view provides Attach sources button which can be used to attach the sources. The similar link for Javadoc is in Javadoc Not Found window.
Comment 58 Jesse Glick 2011-10-19 15:29:24 UTC
*** Bug 160485 has been marked as a duplicate of this bug. ***


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo