This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 19912 - Compiler does not fully honor java dependencies
Summary: Compiler does not fully honor java dependencies
Status: CLOSED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Unsupported (show other bugs)
Version: 3.x
Hardware: PC Windows ME/2000
: P2 blocker (vote)
Assignee: Svata Dedic
URL: http://java.netbeans.org/Proposals/de...
Keywords:
Depends on:
Blocks: 21553
  Show dependency tree
 
Reported: 2002-01-29 17:47 UTC by Michael Ottati
Modified: 2007-09-26 09:14 UTC (History)
1 user (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Ottati 2002-01-29 17:47:13 UTC
The NetBeans build system, and the compiler API's do not fully take java
dependencies into account when performing a "build" or "build all" of java
sources. The following discussion is based upon three java files (A, B and C)
spread over 3 packages (packageA packageB and packageC) in NetBeans.

I am filing this as a bug and not an enhacement request because I believe that
it is a reasonable expectation by users to expect this. If the limitations
described below are documented and a different expectation level given to end
users, then I am comfortable both with lowering the priority of this bug and
chagning it to an enhancement. 

The dependency relationship between the java programs is: A depends on B depends
on C.

When A B and C are compiled and A is executed the results are as expected. If C
is subsiquently modified to be an abstract class, neither the build nor
compilation system will detect this when compiling/building A. The simple test
of this assertion is to add the method "myAbstractMethod" to C and recompile C.
If A is then compiled and executed the execution will fail with an abstract
method error.

As a follow on test I performd a "Build All" from the file menu with A selected.
Even when using build all, the change in the dependent file C is not detected. 

Arguably this is the falut of the Java compiler. There appears to be no way to
force the Sun java compiler to fully honor dependencies, the referenced bug
below is one of several on the subject. I have included this link to give a
flavor of the sorts of bugs filed with JavaSoft on this issue.

http://developer.java.sun.com/developer/bugParade/bugs/4175374.html

It is a development, execution and debugging requirement that there be some
mechanism within the IDE to guarentee that all of the dependent class files of a
.java file can reliably be brought up to date. The current IDE does  not support
this requirement either through the compiler API nor through simple IDE commands
such as build all compile all etc.

Other IDEs can do this (Microsoft Visual studio for C programs), and other Java
compilers can do this, Jikes for java programs. 

I will end the discussion with an example of how Jikes handles the same problem
throught the use of it's +F flag on compilation. Jikes appears to reach an a
reasonable performance/completeness comprimise in that it will not by default do
a complete dependency analysis but allows this analysis to take place when the
appropriate switch is set.

This is a request that NetBeans allow the same execution behavior that Jikes
allows as regards dependencies. This new behavior should be available both to
IDE users as well as programatic users through some API.



------------------ Jikes Example ---------------------

$ jikes  packageA/A.java

Michael@DEN /netbeans/sampledir
$ jikes  packageA/A.java ;java packageA.A
My Class name is:packageA.A

----- Now make C abstract with a reference to the new abstract method ----

Michael@DEN /netbeans/sampledir
$ jikes  packageA/A.java ;java packageA.A

Found 1 semantic error compiling "C:/cygwin/netbeans/sampledir/packageA/A.java":

    10. public class A extends B {
                     ^
*** Error: The abstract method "void myAbstractMethod();", inherited from type
"packageC/C", is not implemented in the n
on-abstract class "packageA/A".
My Class name is:packageA.A
Exception in thread "main" java.lang.AbstractMethodError
        at packageC.C.myName(C.java:16)
        at packageA.A.main(A.java:17)

---- This is the same behavior as Sun Javac
---- Now send the "-F" compiler flag into the jikes compile
---- This time jikes detects that A is no longer valid with
---- regard to it's super class C.

Michael@DEN /netbeans/sampledir
$ jikes -F packageA/A.java ;java packageA.A

Issued 1 system warning:

*** Warning: "-F" is an invalid option; use: jikes [-bootclasspath
path][-classpath path][-d dir][-debug][-depend|-Xdepe
nd][-deprecation][-extdirs path][-g][-nowarn][-nowrite][-O][-sourcepath
path][-verbose][-Xstdout][++][+B][+c][+OLDCSO][+
D][+DR=filename][+E][+F][+Kname=TypeKeyWord][+M][+P][+Td...d][+U][+Z] file.java....

Found 1 semantic error compiling "C:/cygwin/netbeans/sampledir/packageA/A.java":

    10. public class A extends B {
                     ^
*** Error: The abstract method "void myAbstractMethod();", inherited from type
"packageC/C", is not implemented in the non-abstract class "packageA/A". Since
the type "packageC/C" was read from a class file, it is possible that it just
needs to be recompiled because after having inherited method "void
myAbstractMethod();" from an interface, the method was subsequently removed from
that interface.
My Class name is:packageA.A
Exception in thread "main" java.lang.AbstractMethodError
        at packageC.C.myName(C.java:16)
        at packageA.A.main(A.java:17)

Michael@DEN /netbeans/sampledir



---------------- A.java  
/*
 * A.java
 *
 * Created on January 28, 2002, 11:09 PM
 */

package packageA;
import packageB.B;

public class A extends B {


    public A() {
    }

    public static void main(String[] args) {
        new A().myName();
    }
}
----------------- B.java -------------------
package packageB;
import packageC.C;

public class B extends C{

    /** Creates a new instance of A */
    public B() {
    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        new B().myName();
    }

}
------------------- C.java -----------------
package packageC;
import java.lang.reflect.*;
public abstract class  C {

    public C() {
    }

    public void myName() {
        System.out.println("My Class name is:" + getClass().getName());
        // Toggle next line with comments
        myAbstractMethod();
    }
    // Toggle next line with comments
    public abstract void  myAbstractMethod();

}
Comment 1 _ ttran 2002-01-29 21:03:00 UTC
Svata, your opinions?  If you think there is a deficiency in the 
Compiler API return this issue back to openide.
Comment 2 Svata Dedic 2002-01-30 09:47:37 UTC
Let me say that your example is quite confusing: In the case, you've
identified as now working well, jikes (and by your statement javac as
well) produced this statement:
`Found 1 semantic error compiling
"C:/cygwin/netbeans/sampledir/packageA/A.java": ...'
The same error, albeit with longer explanation, was printed in the
second case, which you claim to behave correctly. Both cases (javac,
jikes +F) produced errors when you tried to recompile A, which is
correct as far as I understand. If you desire more descriptive
messages from compiler's output, please file the issue against
Javac/JDK team in Bugparade.



Comment 3 Svata Dedic 2002-01-30 10:12:32 UTC
There's a misunderstanding what "Build all" means. Online
documentation states, that when using "Compile", the IDE tries to
up-to-date check and invokes the compilation tool(s) for those
objects, which fail the check. "Build" means that the IDE does not
perform the check, so it invokes compilation unconditionally.
Note that both Compile and Build operation operate on selected files
only and IDE is considering only those files.
The online docs should also clearly say that the "up-to-date check" is
done exactly by comparing timestamps between the source (.java) and
products (.class) of the compilation.
So - this is the current design and intended behaviour, although
definitely not perfect for the user experience.

I made a request to QA to review this comment and possibly confirm
that it IS an enhancement request.

Comment 4 Svata Dedic 2002-01-30 10:15:48 UTC
Last comment -- sort of disclaimer :-)
The behaviour can be changed so, for a given selection, the IDE will
supplement the compiler's dependency checking so that the initial
selection AND its transitive reference closure will be brought up to date.
Please note that there can be more files in the user project, which
weren't initially selected *and* weren't reached by the closure
algorithm, but still depends on a class, which was recently changed.
Such files will *NOT* be recompiled.

Comment 5 Pavel Buzek 2002-01-30 11:48:43 UTC
to Build and Build All: see the online help. It says clearly what these actions do and they behave in 
accordance with this, AFAIK.

I am changing this to enhancement. It may be reasonable expectation (if it was not I would close it as 
"Invalid") so I am not changing the priority, but it is not a bug in the current implementation or 
design. Compile and Compile All behave consisntetly with what help says: 

> Build   Compile (F9) to compile only those files that 
> are new or have changed since the last compile. This 
> command does not compile the files in subfolders.
>
> Build   Compile All (Shift-F9). to compile only those 
> files that are new or have changed since the last 
> compile, including the files in subfolders.
Comment 6 Jan Chalupa 2002-01-30 15:49:04 UTC
From the QA standpoint, I agree that this is a request for
enhancement, not a bug. The current behavior is by design.
Comment 7 Michael Ottati 2002-01-30 20:07:19 UTC
Let me first attempt to clarify the example.

Step 1:  I compile A. B and C are compiled as a side effect.

Invoking A with java runs correctly.

Step 2:  I modify and recompile C. The modification is to add the
abstract method.

Step 3:  I compile and execute A.

In step 3, A can be recompiled either by javac or jikes and both
report no errors. In step 3 the execution fails due to the abstract
method error.

Conclusion the javac compiler has not traversed the dependency tree
far enough to determine if A is really up to date or not.

Finally the jikes example was ment to show that Jikes could do the
compilation analysis completely and emit the compile error that A
should be declared abstract or implement the method myAbstractMethod.

Having said that I guess the concise statement of the RFE is that I
would like both a programatic and IDE accessable mechanism to bring a
java file "up to date" with all of the class depencencies.

Lastly I have a minor lingustic nit to pick with what the help says 
and what actually happens on a compile/build.

The help for build says (according to the previous comment)

> Build   Compile (F9) to compile only those files that 
> are new or have changed since the last compile. This 
> command does not compile the files in subfolders.

In point of fact compile WILL compile dependent, uncompiled java files
in sub folders as part of a normal Javac compile. In that regard the
behavior and documentation are at odds.

As I have said before, the underlying problem is in the javac compiler
being inconsistent as regards class dependencies, it will compile them
fresh as it encounters them but it will stop earlier in cases where
the first dependency is satisfied by what it considers to be an "up to
date" class file.

The RFE really comes down to can we do this better in NetBeans? Also
is there some workaround that I missed?






Comment 8 Svata Dedic 2002-02-01 14:30:13 UTC
I suggest taking this issue to the mailing lists for discussion about
solutions and tradeoffs.
Comment 9 Pavel Buzek 2002-02-04 15:47:34 UTC
Let me discuss some options for how this could possibly be solved, at least at high-level. I think that 
there are actually three ways how to deal with this issue:

1. Use 'Build All' for the java classes, do not relly on dependency checking.
I think that this works well in the sense that nothing is forgotten, AFAIK there are two issues:
a) User cannot use compile on file, instead they have to do 'Build All'. This cannot be easilly changed 
by this solution (#1), but this can IMHO be understood by users. If they want complete, correct and 
reproducible build they have to call "build". Compile action is available on a source file for users who 
know what it does. (alternatively "compile" action on Java source can be reimlemented to do this :-)

b) Subsequent steps of build (like packaging) will have to be done even if nothing actually changed, 
since timestamps change (see also http://www.netbeans.org/issues/show_bug.cgi?id=19688). There can be 
solution for this:

In this case we would first rebuild all classes and then we would need to recognize which classes have 
really changed. I can imagine using MD-5 or some other check-sum to find out what has changed.  
Alternatively we could pack the classes in JAR and check the check sum for JARs.

This is not optimal solution, since it requires full recompilation of all java classes, but it may be a 
lot better then redoing the entire build and deployment.

It is probably possible to implement this in 3.4.

2. Use a compiler which recognizes dependencies between classes and automatically recompiles the 
dependent classes (jikes).

Disadvantage: This is not a general solution (e.g. does not work for javac), but it could remain as an 
option for smarter compilers. 

It is probably possible to implement this in 3.4.

3. To find the dependencies between classes (e.g. using classclosure module, MDR, etc.) and to pass this 
information to the Compilation Engine via Compiler API.

In the example above, this would means that A would be made dependent on B and B dependent on C. Comiler 
API is capable of expressing this.

This solution has two parts: 
a) passing this information to build system is easy (already possible via Compiler API). 
b) finding this information:
 The source of this information can be the classclosure module (Michael noted that it can be fixed 
relatively easilly). Another alternative is using information stored by Java module in MDR, but this will 
be available in NetBeans 4.0, not in 3.4.

In any of these two cases (especially for classclosure module) it may be benificial to cache/persist the 
derived dependency information (per user), but we need to do preformance measurements first.

Storage for derived dependencies and connection to Compiler API can be implemented in 3.4.

To colclude, these three options are not exclusive, in theory every module and/or every compiler can 
choose different approach.

Michael, do you have any other proposal or comments on these?
Comment 10 Svata Dedic 2002-04-24 18:07:17 UTC
See proposal at
http://java.netbeans.org/Proposals/dependencies/index.html

Implemented in a private branch (extrernal binary required)
Comment 11 Svata Dedic 2002-05-13 12:34:22 UTC
Merged into trunk:

core/compiler/src/org/netbeans/core/compiler/CompilationEngineImpl.java;
/cvs/core/compiler/src/org/netbeans/core/compiler/CompilationEngineImpl.java,v
 

/cvs/java/build.xml,v  <--  build.xml
new revision: 1.33; previous revision: 1.32
/cvs/java/manifest.mf,v  <--  manifest.mf
new revision: 1.56; previous revision: 1.55
/cvs/java/src/org/netbeans/modules/java/Bundle.properties,v  <-- 
/cvs/java/src/org/netbeans/modules/java/JCompilerSupport.java,v  <-- 
JCompilerSupport.java
/cvs/java/src/org/netbeans/modules/java/JavaCompilerType.java,v  <-- 
/cvs/java/src/org/netbeans/modules/java/JavaMakeCG.java,v  <-- 
JavaMakeCG.java
/cvs/java/src/org/netbeans/modules/java/settings/Bundle.properties,v 
<--  Bundle.properties
/cvs/java/src/org/netbeans/modules/java/settings/JavaSettings.java,v 
<--  JavaSettings.java
/cvs/java/src/org/netbeans/modules/java/settings/JavaSettingsBeanInfo.java,v
 <--  JavaSettingsBeanInfo.java

Comment 12 Jan Becicka 2002-06-12 09:13:43 UTC
A new property was added: "Use JavaMake". It 
is in the Expert tab of Tools | Options | 
Editing | Java Sources option. The option 
controls whether JavaMake is used or not. It 
defaults to False to preserve current usage 
patterns.
Comment 13 Quality Engineering 2003-07-01 13:18:51 UTC
Resolved for 3.4.x or earlier, no new info since then -> closing.