Bug 59792

Summary: Separate API during build process
Product: apisupport Reporter: David Strupl <dstrupl>
Component: HarnessAssignee: Jesse Glick <jglick>
Status: RESOLVED FIXED QA Contact: issues <issues.netbeans.org>
Priority: P1 CC: jglick, jtulach, mkrauskopf, phrebejk, rkubacki, santhosh, wadechandler
Version: 4.x   
Target Milestone: 5.x   
Hardware: All   
OS: All   
Whiteboard:
Issue Type: ENHANCEMENT Exception Report:
Bug Depends on: 49371, 70220    
Bug Blocks: 68631, 68716    
Attachments: First draft

Description David Strupl 2005-06-09 14:14:53 UTC
Currently when you build a module and it depends on another module your module
can "see" non-api classes from the other module. This can lead to clean compile
but failure during runtime (NoClassDefFoundErrors). The build script should put
only API packages on "compile-classpath". This can be (hopefully) easily
achieved by putting the API packages in separate .jar file and putting only this
API jar on classpath when building dependant modules.

This idea comes from project AWA (sorry it is in czech):
http://karel.mine.nu/awa/, especially from the arch description (again in
czech): http://karel.mine.nu/awa/docu/arch.html and here
http://karel.mine.nu/awa/docu/awacoding.html
They were inspired by Eclipse if I am not mistaken ...
Comment 1 Jesse Glick 2005-06-09 21:54:04 UTC
I've thought of it before too. Might be a performance problem to create the
API-only JARs; not sure. I guess the algorithm for each classpath entry would be

1. If it comes from an impl dep, leave it alone.

2. If it comes from a spec dep, check if the dep has any public packages (taking
into account OpenIDE-Module-Friends).

2a. If not, halt the build (since you should not have been using such a dep).

2b. If so, create a separate JAR (where??) containing only the public packages,
and replace the classpath entry with this JAR. Can cache such JARs so long as
the original has not changed.
Comment 2 David Strupl 2005-06-09 22:23:46 UTC
I don't think that performance is of any big concern during build time. I agree
that keeping the API jars somewhere is a good idea since they can be used when
building one module only (e.g. from the IDE). If the API jars are put to some
special folder the behaviour of the resulting program will not be affected (only
the build process).

BTW when looking at the mentioned framework they can live with much simpler
module system even during runtime since they keep the jars separated (and thus
they have no need for any complicated run-time checking of the dependencies).
They have only simple dependencies between their jar files.
Comment 3 Jesse Glick 2005-06-09 22:31:15 UTC
Yeah, the question for me is where to put the temp JARs - it's OK for the
netbeans.org build to use e.g. nbbuild/build/apijars/ but what about builds of
external modules? Need to find another spot. Not a big problem, but will take a
bit of work.

I guess I would be the one to work on it.

Re. splitting API from impl even at runtime: neat idea, though I think for NB it
would not be worthwhile at this point; would require too many things to be modified.
Comment 4 David Strupl 2005-06-09 22:39:08 UTC
Totally agree that it is not relevant for NB as it would be lot of work for no
gain (actually slower startup because there would be more jars) - it was just BTW.
Comment 5 Jaroslav Tulach 2005-06-10 07:48:36 UTC
I like the idea of creating two JARs during build time and caching one in 
nbbuild/build somewhere (if it can work). Other possibility would be to change 
the compiler to honor OpenIDE-Module-Public-Packages, Hrebejk could be of some 
help here, but I doubt this would get enough priority on his list. 
 
Btw. From the proposal it seemed to me that when a JAR is gonna be created, 
also its api JAR will be put somewhere. Maybe, for the benefit of external 
modules, the common.xml could create the cache of api jars somewhere when 
compiling a module that depends on them. This might solve the problem with 
cache. It would always be near the target directory (of a suite). Maybe this 
directory could also be driven by a property to specify the cache dir and 
enable or disable this compile time improvement? 
Comment 6 Jesse Glick 2005-08-10 17:02:05 UTC
*** Issue 62038 has been marked as a duplicate of this issue. ***
Comment 7 santhosh 2005-12-09 00:50:02 UTC
it would be even better if we don't list such private classes in auto-complete
popup during java programming in a module project.
Comment 8 Jesse Glick 2005-12-09 00:56:03 UTC
Re. excluding also from code completion - not directly supported by Java
infrastructure in IDE, but could perhaps be faked by putting the API-only JARs
into the IDE's effective classpath.
Comment 9 _ wadechandler 2005-12-09 07:47:40 UTC
########Adding to aid others trying to work through this################
I found some documentation that explained it better to me.  For anyone else
looking look at this URL:
http://www.netbeans.org/download/dev/javadoc/org-openide-modules/org/openide/modules/doc-files/classpath.html

and search the page (include quotes in your search) "back door"
########################################################################

I still think there needs to be another method however.  Couldn't we just have
an explicit means of saying....I don't care if errors arise...I'll deal with
them when they do... something like:

OpenIDE-Module-Module-Dependencies: org.netbeans.modules.project.libraries/1 =
*,org.netbeans.modules.form/2 = *

And this could allow us to not worry about the specific version number, yet we
are aware our module might have issues that could arise from a shifty API.  The
reason I say that is that sometimes a version number is going to change, but
none of the private code a developer was accessing will change, but a bunch of
other stuff will.  That or a simple bug fix will go in.  The user updates from
auto-update with some hot-fix, and now your module is broken.  This way while
you're getting something to work you can still deploy your module and work with
the developers of another module to get some good use cases and public API's
worked out of some existing code they have.  Does this not sound like a good
open compromise?  I say this because it's hard enough to find extra time to work
on some ideas for NB not including having to worry about how you're not going to
replicate a bunch of existing code sitting there you know you can use if you
were able to just access the API at runtime.  This way we know we're opening up
a can of worms when we use the *, but until we can work around it through
mediation with another project it will help get a lot of other plug-ins going a
little faster.  If nothing else help get some proof of concepts going.  For
instance, External Libraries doesn't even have an implementation version, and I
can't seem to figure out what the default value is if one even exists.  So, now
I'm like...ok...I don't want to rewrite the LibraryCustomizer just to have the
same features because I don't have much time...I could always change the
manifest for External Libraries and rebuild it, but then that only addresses my
machine.  I was hoping to bounce this back and fourth some with some other
people without it being such a long drawn out sucker of my time, but currently I
can't not without copying out much if not all of the External Libraries code
into my own module.
Comment 10 Jesse Glick 2005-12-10 21:22:10 UTC
Wade - the last comment was off topic for this issue report. This issue is not
proposing any changes to the module system. It is solely about enforcing the
existing behavior better at compile time, to preempt runtime errors.
Comment 11 _ wadechandler 2005-12-11 02:27:09 UTC
No problem, I just noticed some issues were still being aired out and thought I
would put this in there with them because it made me think about it.  On to a
comment relevent to the discussion.  If you don't show the API's you still need
to  show them and include them in the classpath when an implementation
dependency is being used in the manifest right?  So, I thought I would mention
that it would also be nice to have a more open way of saying that...show me the
APIs and let me access it all.  Still the issue: Basically if you split this out
into two separate jars you'll have to include the jar on the classpath if an
implentation dependency is definied.
Comment 12 Jesse Glick 2005-12-11 03:08:15 UTC
Yes, with issue #68631 and #68716 the algorithm for constructing a classpath
from each dep marked as a <compile-prerequisite/> would be something like

1. Is target module in platform (minus declared excludes)? If no, error.

2. Is target module a matching version? If no, error.

3. Is it an impl dep? If yes, use original target module JAR in CP.

4. Does target module export packages only to friends, but source module is not
a friend? If so, error.

5. Else, create separate API-only JAR somewhere and add to CP.
Comment 13 Jesse Glick 2005-12-11 18:21:04 UTC
Another note: AFAIK it is not currently possible to make code completion, Fix
Imports, Open Class, etc. take into account public package restrictions while
still providing popup Javadoc and other features, unless issue #70220 is fixed.
However, attempted uses of unavailable classes would still appear as errors in
the editor's error stripe, even before you tried to build the module, so this
wouldn't be so bad I think.
Comment 14 Jesse Glick 2005-12-29 19:26:21 UTC
Will not attempt to do anything special for the in-IDE classpath, used for code
completion, background compilation, etc. If you use a nonpublic class you will
get a compiler error from Ant; probably enough. Would require either issue
#70220 or issue #49371 to be implemented to solve this fully and without
unpleasant side effects; #49371 would be preferable as it would not require a
special SourceForBinaryImpl, and would not require the NBM project type to
physically create the split JARs when working with an unbuilt source tree.
Comment 15 Jesse Glick 2005-12-30 03:57:05 UTC
Created attachment 28103 [details]
First draft
Comment 16 Jesse Glick 2005-12-30 07:51:16 UTC
Done, with issue #68631 and issue #68716. Had to make a number of fixes in
netbeans.org modules first, since there were in fact numerous compile-time
violations of public package lists.

A note: public package restrictions are *not* enforced for unit tests in a
module - only main sources.

committed   * Up-To-Date  1.29        apisupport/harness/release/README
committed   * Up-To-Date  1.10        apisupport/harness/release/build.xml
committed   * Up-To-Date  1.11       
apisupport/project/src/org/netbeans/modules/apisupport/project/ui/customizer/ModuleProperties.java
committed   * Up-To-Date  1.26       
apisupport/project/src/org/netbeans/modules/apisupport/project/ui/customizer/SuiteProperties.java
committed   * Up-To-Date  1.9        
nbbuild/antsrc/org/netbeans/nbbuild/JarWithModuleAttributes.java
committed   * Up-To-Date  1.28       
nbbuild/antsrc/org/netbeans/nbbuild/ModuleListParser.java
committed   * Up-To-Date  1.32       
nbbuild/antsrc/org/netbeans/nbbuild/ParseProjectXml.java
committed   * Up-To-Date  1.31        nbbuild/templates/common.xml
committed   * Up-To-Date  1.8         nbbuild/templates/emma.xml
committed   * Up-To-Date  1.64        nbbuild/templates/projectized.xml
committed   * Up-To-Date  1.14        nbbuild/templates/xtest-unit.xml
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo