This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
It was introduced in NB 2011-02-16, probable little earlier: in build-impl.xml, "javac.fork" value="false" was changed to "true". Since then, clean+build of multiple-dependent projects is 3 times slower!!! I tried to manually find+replace that to "false" again in all projects' build-impl.xml, and clean+build became as fast as before. System: Athlon X2 64 3GHz, Opensuse 11.3, JDK7 b130. (Note: I can't test it with JDK6, because the only group of multiply dependent projects I have is targeted to JDK7.)
The fix of bug #188514 was rather drastic. Ironically, the reporter of this bug is using JDK 7, for which no fix was needed to begin with, and using Linux, which doesn't really have a problem with unreleased JarFile's either (so long as ulimit is not exceeded and no actual attempt is made to read a JarEntry after the JAR has been overwritten). At a minimum I think the fix should be restricted to Windows / JDK 6 users. A better fix might anyway be to just call System.gc() after an Ant process finishes (*) - from what I understand, that would suffice to make the now-unreferenced URLCL be collected, and the JAR closed. (*) BridgeImpl.gutProject would be the logical place. Or this could be done in e.g. JavaAntLogger.buildFinished, possibly restricted to JDK 6 Win, possibly iff <javac fork="false"> has been called at least once.
>Ironically, the reporter of this bug is using JDK 7, for which no fix was needed to begin with, Unfortunately there is no way how to find out the JDK version. >and using Linux, which doesn't really have a problem with unreleased JarFile's either (so long >as ulimit is not exceeded and no actual attempt is made to read a JarEntry after the JAR has been >overwritten) The problem exists even on Linux just UNIX allows you to delete (rewrite opened file) but it consumes OS resources. When crossing handles per project limit whole IDE will crash (even worse). The only "safe" OS from this point of view is Mac OS X where process has 32K file handles. >A better fix might anyway be to just call System.gc() after an Ant process >finishes (*) - from what I understand, that would suffice to make the >now-unreferenced URLCL be collected, and the JAR closed. Does not help, you can try. The URLClassLoader never closes the JarEntries. The only thing I can do is not to do it on Linux and hope that IOException (Not enough file handlers will not be thrown), but it's a bit dangerous.
Can the reporter somehow support his claim that the clean&build is really 3 times slower? I am tempted to close this as incomplete or works for me. Anybody else can see so bit of a slowdown?
bit/big
Well, I have multiple projects which all are not small, depending on each other. (Half of the projects use ruby-generated templates, BUT they're are SHA-shecked and only rebuilt when templates change, so shouldn't be the problem). By the way, all projects use "compile on save". I could try to reproduce the bug by writing lots of inter-dependent projects, but intuition tells me that it would only be reproducible when projects would be really "full" of classes. Which additional information might be useful apart of full reproduction attempt? And, if it's not a too big story, could someone explain, why javac.fork is suddenly such important?
(In reply to comment #5) > Well, I have multiple projects which all are not small, depending on each > other. (Half of the projects use ruby-generated templates, BUT they're are > SHA-shecked and only rebuilt when templates change, so shouldn't be the > problem). > By the way, all projects use "compile on save". > I could try to reproduce the bug by writing lots of inter-dependent projects, > but intuition tells me that it would only be reproducible when projects would > be really "full" of classes. > Which additional information might be useful apart of full reproduction > attempt? > And, if it's not a too big story, could someone explain, why javac.fork is > suddenly such important? I can confirm that compile time has increased, but that was expected to happen for the fix of bug #188514. The reason for the fix is a bug in pre 7 JDK that does not properly close library files after compilation, the only way to close them is to terminate the JVM that has opened them, hence the delay in compilation, it takes longer time to restart the compiler between compilations than re-using an already loaded instance... IMHO this is nothing than can be fixed easily... Also if you are using JDK7 you can set javac.fork=false manually if running JDK6 you may have compile failures unless javac.fork=true.
> Also if you are using JDK7 you can set javac.fork=false manually if running > JDK6 you may have compile failures unless javac.fork=true. Do I understand right, that javac.fork=true is needed conditionally? So, shouldn't NB decide automatically, when to use it or not? I benchmarked again (did it multiple times, took average numbers, ignoring first two, because of OS-caching the first one-two times are slower). Comipile-on-save, as I thought before, has nothing to do with it: 12 dependend projects, ~1000 classes, copy-libs: off, clean+build: Compile-on-save ON, javac.fork=true: ~45sec Compile-on-save OFF, javac.fork=true: ~45sec Compile-on-save ON, javac.fork=false: ~14sec Compile-on-save OFF, javac.fork=false: ~11sec (I also benchmarked on my notebook, numbers are in similar proportions, but it's not a surprise, most basic properties of hardware+software are same.)
To converginglight: As you are running on JDK 1.7 (on JDK 1.5 or 1.6 the fork=true is required and nothing can be done about it s explained by Chiana in comment #6) the javac.fork is not required. I've extended the build-impl.xml to set the javac.fork to true only if the JDK is either 1.5 or 1.6 and OS is Windows. On UNIX I hope that the handlers will not be over allocated.
fixed jet-main 15bf39be231f (for j2seproject). David, you will need to port the fix into web projects.
fixed jet-main 15bf39be231f
@Tomas Zezula: Thank you for the fix. Indeed, I'm running NB using JDK7, and using JDK7 for all of those dependent projects, I mentioned.
Integrated into 'main-golden', will be available in build *201102260001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/15bf39be231f User: Tomas Zezula <tzezula@netbeans.org> Log: #195823:Since 2011-02-16: javac.fork causes 3-times slower clean+build of multiply dependent projects
Fixed in EE project types - 17864973467f
(In reply to comment #2) > The only "safe" OS from this point of view > is Mac OS X where process has 32K file handles. Linux is fine if you have an appropriate ulimit; see bug #185135. >> call System.gc() after an Ant process finishes > > Does not help, you can try. The URLClassLoader never closes the JarEntries. Where are the JarFile's being held open in this case? I guess this is URLClassPath.JarLoader.jar? But this should be released once the loader is collected. Or is it sun.net.www.protocol.jar.JarFileFactory.fileCache and .urlCache? Then getProject could clear these fields.
(In reply to comment #10) > fixed jet-main 15bf39be231f BTW I don't see why you need two tasks; should suffice to write: <condition property="javac.fork" value="true" else="false"> <!-- ... --> <and>...</and> </condition>
Integrated into 'main-golden', will be available in build *201103011142* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/17864973467f User: David Konecny <dkonecny@netbeans.org> Log: #195823 - Since 2011-02-16: javac.fork causes 3-times slower clean+build of multiply dependent projects.