This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 137060 - Lack of -sourcepath causes *.java in classpath JARs to be recompiled
Summary: Lack of -sourcepath causes *.java in classpath JARs to be recompiled
Status: VERIFIED FIXED
Alias: None
Product: java
Classification: Unclassified
Component: Project (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on: 157692
Blocks:
  Show dependency tree
 
Reported: 2008-06-11 23:47 UTC by emlope
Modified: 2009-05-05 18:41 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description emlope 2008-06-11 23:47:53 UTC
I noticed when I switched to Netbeans 6.0 and later Netbeans 6.1 that if jars in the project classpath contain 
sourcefiles, they are compiled and placed in the project's build/classes folder and consequently, the dist jar. The 
cause is the compiler sourcepath option.

Netbeans 5.5 always resulted in the javac sourcepath to set to ${src.dir}, but in Netbeans 6.0 it does not get set at 
all (the option is omitted, I used ant -verbose to check this). (in build_impl.xml)

Was this change intentional? Our team keeps sourcefiles in jars frequently, so we have to manually set the sourcepath 
option for almost every project we create.

Here is a link to a question I posted, but I never got a response:
http://www.netbeans.org/servlets/ReadMsg?listName=nbusers&msgNo=115985
Comment 1 Tomas Zezula 2008-06-12 07:55:03 UTC
No this change is not intentional. Should be fixed in NB 6.5.
Comment 2 ulim 2008-09-04 08:26:26 UTC
It is not fixed in 6.5beta. This is a very serious bug that can lead to spurious failures in productive systems. In our
case two components A and B depended on V1 and V2 of the same lib and we naturally packaged lib V2 with our application.
However, the JAR of A contained the classes of lib V1 and every now and then the classloader would load these first and
B would not work.

The workaround is easy, just use a --sourcepath /does/not/exist as a compiler option, but anyone unaware of this bug
might not notice it in testing, as it depends on classloader luck.
Comment 3 ulim 2008-11-13 10:27:13 UTC
This is a serious bug that can lead to spurious failures in productive systems. It needs to be fixed ASAP.
Comment 4 Jan Becicka 2008-11-13 10:30:22 UTC
OK. If it is serious bug, it should be P2. But 6.5 development is over.
Comment 5 Tomas Zezula 2008-11-13 10:34:10 UTC
Unfortunately the sourcepath option was removed as a part of excludes support. Adding it back will break the excludes.

Comment 6 ulim 2008-11-13 10:55:04 UTC
Source files, that are contained in Jars, are at compile-time added to the actual project source files without the user
knowing about it. This is not only a bug, but a serious security leak, as 3rd party libs could inject foreign code into
projects built with Netbeans. If this is acceptable for the 6.5 release, then it must be documented as a known bug and
security problem.

However, removing the sourcepath option entirely might actually have fixed this bug. I would test it, if you tell me
which version.
Comment 7 Tomas Zezula 2008-11-13 14:26:22 UTC
No, this is a misunderstanding. The sourcepath is set to empty path, this was the change required by excludes. And this is the problem. The original intention 
was to ensure that the javac sees only the listed files, not the complete classpath as the part of it may be excluded.
Comment 8 ulim 2008-11-13 16:08:55 UTC
Ok... because above you said this change was not intentional and would be fixed.

If it is intentional, then the sourcepath attribute may have been used for a different purpose that it was originally
intended for. Its purpose is a very special case: if you want to automatically compile sources found on the classpath.
For realizing excludes the usual way would be to omit the sourcepath attribute entirely (it defaults to srcdir then) and
use the excludes attribute.
Comment 9 Jesse Glick 2008-11-13 16:18:28 UTC
Right. If not told otherwise, Ant passes -sourcepath <sourcedir> to javac, which for example lets javac automatically
recompile B.java if it is out of date and referenced from A.java which it was asked to compile. However in this default
mode, configuring the project to exclude B.java from compilation would have no effect, which would be surprising to the
user. Passing sourcepath="" to Ant (hence no -sourcepath to javac) ensures that excluded files are never passed to the
compiler, so the build will correctly fail if you have excluded a source file which an included source file depends on.

This is the first report I have heard that lack of -sourcepath causes problems when classpath entries contain source
files. I can reproduce. Clearly that is a problem and should be evaluated for 6.5 "showstopper" status, or at least as
an update patch to 6.5. Since it was never reported for 6.0 or 6.1 (to my knowledge), it is questionable that this would
be considered P1.

I don't see a security issue; if you include a third-party lib in your app's compile classpath, it is likely you are in
fact running against that lib anyway. Even if not and some class does get compiled into your app's JAR, it will have no
effect unless you explicitly call it. Furthermore, annotation processors in a library can always generate classes (never
overriding what you compile explicitly). In general, it seems implausible that you would compile against a library of
unknown and possibly malicious origin.

Passing -sourcepath /does/not/exist to javac is a promising workaround. I can confirm that changing build-impl.xml to say

<attribute default="/does/not/exist" name="sourcepath"/>

works on Linux on JDK 6 and does not result in any compiler warnings. (Have not tried on other JDK/OS combos.) Of course
an individual user can choose to use this workaround (actually adding an override to build.xml - never edit
build-impl.xml directly) if they are aware of the problem.

Arguably the root issue is a bug in javac. I don't see any reason why javac should ever search for sources in its
-classpath, especially not inside JARs, but perhaps I'm missing something.
Comment 10 Tomas Zezula 2008-11-13 16:52:25 UTC
From javac perspective it's not a bug it's intentional when there is no source path it uses the class path.
The ClassReader does something like:
boolean haveSourcePath = fileManager.hasLocation(SOURCE_PATH);
if (!haveSourcePath) {
            fillIn(p, CLASS_PATH,fileManager.list(CLASS_PATH,packageName,kinds,false));
} else {
            fillIn(p, CLASS_PATH,fileManager.list(CLASS_PATH,packageName,classKinds,false));
            fillIn(p, SOURCE_PATH,fileManager.list(SOURCE_PATH,packageName,sourceKinds,false));
}
Comment 11 Tomas Zezula 2008-12-09 16:42:16 UTC
http://hg.netbeans.org/main/rev/50430e031e21
Comment 12 Quality Engineering 2008-12-10 05:57:00 UTC
Integrated into 'main-golden', will be available in build *200812100201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/50430e031e21
User: Tomas Zezula <tzezula@netbeans.org>
Log: #137060:Lack of -sourcepath causes *.java in classpath JARs to be recompiled
Comment 13 Tomas Danek 2009-01-12 16:44:47 UTC
ulim: Can you please confirm that the fix works for you?
Comment 14 ulim 2009-01-13 09:06:44 UTC
Confirmed, thank you very much.
Comment 15 Jiri Prox 2009-01-13 10:29:43 UTC
thanks for verification
Comment 16 pgebauer 2009-01-13 14:57:59 UTC
The fix has been ported into the release65_fixes repository.

http://hg.netbeans.org/release65_fixes/rev/34deb033f54c
Comment 17 medge42 2009-02-24 04:40:15 UTC
I've just run the patches on windows XP Professional(targeting JDK 1.6), now when I get a compiler warning:

warning: [path] bad path element "C:\does\not\exist": no such file or directory
Comment 18 malfunction84 2009-02-24 19:06:18 UTC
medge42, see issue 157692.
Comment 19 Jesse Glick 2009-05-05 18:41:10 UTC
Tricky when working on issue #157692 to verify that I am not regressing this issue because I cannot reproduce it (with
the fix 50430e031e21 reverted). If I create a lib project with one class, and an app project with one class depending on
the lib project, and clear the Excludes field from Packaging in the lib project, nothing extra is compiled into the app
project. If I add to the lib project's build.xml

    <target name="-pre-jar">
        <sleep seconds="1"/>
        <touch>
            <fileset dir="${build.classes.dir}" includes="**/*.java"/>
        </touch>
    </target>

to make sure the *.java is considered newer, same result. If I add to lib project

    <target name="-pre-jar">
        <delete>
            <fileset dir="${build.classes.dir}" includes="**/*.class"/>
        </delete>
    </target>

so that its output contains _only_ the *.java, the app project will not compile at all.

So if you know of a simple self-contained test case, please add it here. Otherwise make sure to reverify this in NB 6.7 RC.