This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 188514 - NetBeans retains lock on JEE Endorsed Libraries when project is undeployed
Summary: NetBeans retains lock on JEE Endorsed Libraries when project is undeployed
Status: VERIFIED FIXED
Alias: None
Product: javaee
Classification: Unclassified
Component: Web Project (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P3 normal (vote)
Assignee: David Konecny
URL:
Keywords:
: 194059 (view as bug list)
Depends on: 200790 195823
Blocks:
  Show dependency tree
 
Reported: 2010-07-09 03:46 UTC by Catchwa
Modified: 2011-08-10 21:17 UTC (History)
6 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
jaxb-api-osgi.jar locks (GF & NB running) (44.12 KB, image/png)
2010-07-13 00:59 UTC, Catchwa
Details
jaxb-api-osgi.jar locks (GF stopped) (42.15 KB, image/x-png)
2010-07-13 01:00 UTC, Catchwa
Details
NB project (4.06 MB, application/x-zip-compressed)
2011-02-06 04:04 UTC, Chiana
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Catchwa 2010-07-09 03:46:58 UTC
1. Create a new Web Application Project (GlassFish v3.0.1, JEE6)
2. Create a new Web Service
3. Add an operation to your web service
4. Deploy web service
5. Test web service using GlassFish's Tester
6. Undeploy service
7. Stop GlassFish
8. Delete Project from NetBeans. When prompted, tick the option to "Also delete sources under $PROJECT folder"

Observed:

The project directory has not been removed:

C:\NetBeansProjects>ls -R WebApplication1
WebApplication1:
lib

WebApplication1/lib:
javaee-endorsed-api-6.0

WebApplication1/lib/javaee-endorsed-api-6.0:
javax.annotation.jar  jaxb-api-osgi.jar  webservices-api-osgi.jar

Should be:

The project directory should have been removed. If I try and use Windows to remove the directory I get a message saying that the .jar files in question are "currently in use by another person or program".

As GlassFish is no longer running (see step 7 above), is it NetBeans that has a lock on these files? Is this correct behaviour? If I exit NetBeans I am then able to remove the project directory (which is annoying).
Comment 1 David Konecny 2010-07-11 21:53:44 UTC
Passing to web services for initial evaluation. Considering GF is not running then my guess is that it is web services support in IDE which must hold reference to these jars?
Comment 2 Milan Kuchtiak 2010-07-12 09:34:28 UTC
WS support doesn't do anything specific with GlassFish server.
Test WS action just opens the browser with the specific URL.

Isn't that a specific GlassFish+Windows problem where jar files are permanently held by the server ?

On Linux Ubuntu I noticed, after "Project Delete", the directory nbproject/private isn't empty and contains private.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project-private xmlns="http://www.netbeans.org/ns/project-private/1">
    <editor-bookmarks xmlns="http://www.netbeans.org/ns/editor-bookmarks/1"/>
    <open-files xmlns="http://www.netbeans.org/ns/projectui-open-files/1">
        <file>file:/space/home/mkuchtiak/NetBeansProjects/WebApplication5/web/index.jsp</file>
        <file>file:/space/home/mkuchtiak/NetBeansProjects/WebApplication5/src/java/a/A.java</file>
    </open-files>
</project-private>

Other files are deleted successfully.
Comment 3 David Konecny 2010-07-12 18:56:53 UTC
(In reply to comment #2)
> Isn't that a specific GlassFish+Windows problem where jar files are permanently
> held by the server ?

Yes, but user stopped the server so it must be IDE which is holding a reference to these jars. That's why I thought that any IDE feature which opens and uses these jars can be at fault - if it is not web services then who? java parsing infrastructure? I do not have windows to test this.

PetrH, any opinion?
Comment 4 Petr Hejl 2010-07-12 19:10:04 UTC
I guess some class was loaded by the IDE via URLClassLoader or the file was opened via URLConnection.

You can use Tim's tool to diagnose: http://blogs.sun.com/quinn/entry/tool_for_diagnosing_failed_glassfish
Comment 5 Catchwa 2010-07-13 00:52:45 UTC
I've just run the ZipFileMonitor tool referenced by Petr but it doesn't show the files in question being locked (or even opened and closed correctly) by GlassFish. I was using the name of the NetBeans project as a filter. If that was incorrect, please let me know and I'll have another go. I'm pretty sure that GlassFish isn't to blame (see below).

However, using Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx) I've seen some interesting behaviour:

1) When NetBeans is opened (and GlassFish hasn't been started), the endorsed jars have no locks on them (expected).

2) When GlassFish is started, the endorsed JARs (javax.annotation.jar, jaxb-api-osgi.jar, webservices-api-osgi.jar) are locked:
a) in the glassfish endorsed directory (expected), plus
b) within the lib directory of every JEE6 project in my NetBeans workspace, even if it's not currently deployed. (See attachment: glassfish_running.png. Note that nothing is deployed to the GlassFish server.)

3) When GlassFish is stopped (without any applications deployed), the lock of 2)a) is removed but the lock of 2)b) remains. (See attachment: glassfish_stopped.png)

Consequently, if I run GlassFish during a NetBeans session, I am unable to cleanly remove any JEE6 Web Application project from my NetBeans workspace and the filesystem, regardless of whether it was deployed during the NetBeans session.
Comment 6 Catchwa 2010-07-13 00:59:24 UTC
Created attachment 100789 [details]
jaxb-api-osgi.jar locks (GF & NB running)

jaxb-api-osgi.jar locks when GlassFish is running (no applications deployed)
Comment 7 Catchwa 2010-07-13 01:00:40 UTC
Created attachment 100790 [details]
jaxb-api-osgi.jar locks (GF stopped)

jaxb-api-osgi.jar locks when GF is stopped. (Case is the same for javax.annotation.jar and webservices-api-osgi.jar)
Comment 8 David Konecny 2010-07-13 01:19:02 UTC
Thanks. Would you say that lock is created just by starting and stopping glassfish. The smaller test case to reproduce the problem the better. Could you also try to force garbage collection couple of times please and see whether it released locks or not (enable Memory toolbar and click memory usage bar several times). Thanks.
Comment 9 Catchwa 2010-07-13 01:56:30 UTC
>Would you say that lock is created just by 
>starting and stopping glassfish.

The lock itself is created when GlassFish is started. I would expect this lock might be necessary when the application in question is actually deployed, but at the moment it's doing it for every JEE6 project regardless of its deployment status.

> Could you also try to force garbage collection
> couple of times please and see whether it
> released locks or not

I gave this a try but the locks are still there.


Simplest test case possible:

1. Open a clean NetBeans (i.e. no projects currently in the workspace)
2. Create a new Web Application Project. Server GlassFish 3.0.1, Java EE Version: Java EE 6 Web, No Frameworks.
3. Don't touch your new web application at all.
4. Start GlassFish
5. Locks on the following files observed (via ProcessExplorer):

NetBeans:
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\javax.annotation.jar
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\jaxb-api-osgi.jar
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\webservices-api-osgi.jar

GlassFish:
C:\glassfishv3\glassfish\modules\endorsed\webservices-api-osgi.jar
C:\glassfishv3\glassfish\modules\endorsed\jaxb-api-osgi.jar
C:\glassfishv3\glassfish\modules\endorsed\javax.annotation.jar

6. Stop GlassFish server. Aforementioned locks on the glassfish\modules\endorsed\ JARs are gone, but the NetBeans locks remain.

Note: At no point has the JEE6 Web Application in question been deployed to GlassFish. The act of starting the GlassFish server is enough to create these locks that don't disappear until NetBeans is closed.
Comment 10 Catchwa 2010-07-13 02:12:52 UTC
Sorry, a correction to the above couple of posts of mine. It seems to be not dependent on or related to GlassFish at all.

The locks are created either
a.) When the JEE6 Web Application is created (for new projects), or
b.) When NetBeans is opened (for existing projects)

(I think I was possibly checking for locks too soon after creating the project, hence why they didn't show up in Process Explorer)

Revised Test Case:

1. Open a clean NetBeans (i.e. no projects currently in the workspace)
2. Create a new Web Application Project. Server GlassFish 3.0.1, Java EE
Version: Java EE 6 Web, No Frameworks.
3. Locks on the following files observed (via ProcessExplorer):

NetBeans:
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\javax.annotation.jar
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\jaxb-api-osgi.jar
C:\NetBeansProjects\WebApplication1\lib\javaee-endorsed-api-6.0\webservices-api-osgi.jar

4. Delete project. Above locked JAR files remain in the filesystem
5. Close NetBeans. Locks disapper and project directory can be completely removed from the filesystem.


Sorry for any confusion!
Comment 11 Petr Hejl 2010-07-13 08:25:20 UTC
BTW Perhaps I wasn't clear enough, but it is possible to use the mentioned tool for NetBeans itself (to track down the guilty code). The IDE just has to be started in debug mode.
Comment 12 Petr Hejl 2010-07-13 10:19:15 UTC
I made some investigation. Steps to reproduce are:
1) Create web app with enabled "Use Dedicated Folder for Storing Libraries", configured to .\lib
2) Glassfish 3.0.1, Java EE 6 Web
3) Web app created - since this point files are locked

I tracked it with Tim's tool and there was no match on these files. As the tool use breakpoints in JarFile and ZipFile these are locked by a different way. Perhaps it could be some missing stream.close() somewhere.
Comment 13 Catchwa 2010-07-14 00:41:30 UTC
I don't know if this needs a separate bug (let me know if it does), but projects that have web service clients in them have similar locks on the following files:

C:\NetBeansProjects\<ProjectName>\lib\JAX-WS-ENDORSED\jaxb-api.jar
C:\NetBeansProjects\<ProjectName>\lib\JAX-WS-ENDORSED\jaxws-api.jar

To reproduce:

1. New Java Application project (use dedicated folder for storing libraries)
2. Create a new Web Service Client, point it to a WSDL.
3. NetBeans locks the libraries in the JAX-WS-ENDORSED directory (even without the project being run).
4. Try to delete project from NetBeans. The lib\JAX-WS-ENDORSED directory (and its contents) remain in the filesystem.
Comment 14 Petr Jiricka 2011-01-24 16:25:59 UTC
Passing to web services again.
Comment 15 Denis Anisimov 2011-01-25 11:56:43 UTC
1. Please file different bug based on the last steps in the description.
The original issue and the last one is absolutely different.
2. I can't reproduce the issue related to endorsed jars lock at all.
You should know that mentioned jars are used on the stage of WS client creation.
So if you try to delete project right after client wizard is closed then 
your deletion action happens at the same time as WS client creation.
( WS client creation is not quick task ).
As result there is a thread that creates WS client and uses jars in endorsed folder. At the same you delete these jars .
The jars are locked : this is expected behavior . So this is not a bug.
One need to wait until WS client generation finished and then delete project .

So please recheck this again and file a separate bug if you still have a problem AFTER WS client generation .
Comment 16 Catchwa 2011-01-26 22:29:35 UTC
Thanks, I have filed a separate bug for the web service client JAR-locking issue (194753).
Comment 17 David Konecny 2011-01-27 01:01:18 UTC
*** Bug 194059 has been marked as a duplicate of this bug. ***
Comment 18 Chiana 2011-01-27 01:52:27 UTC
Continuing from bug 194059, I've looked thru the source of FileDescriptor and not so convinced anymore that it is an apropriate attack-point as it is using native functions, perhaps using *stream is a better way to implement my suggestion, but that requires more changes to be done...

But as I said, if you can change it I can test it, I have no problems recreating this.
Comment 19 David Konecny 2011-01-27 02:10:24 UTC
Perhaps try tool mentioned in comment #4. It should do the same trick. I have not used it but I think you will have to start NB in debug mode (edit netbeans/etc/netbeans.conf to start IDE with something like "-J-Xdebug -J-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1234") and then run the tool on that port (1234).
Comment 20 Chiana 2011-01-27 02:18:51 UTC
(In reply to comment #19)
> Perhaps try tool mentioned in comment #4. It should do the same trick. I have
> not used it but I think you will have to start NB in debug mode (edit
> netbeans/etc/netbeans.conf to start IDE with something like "-J-Xdebug
> -J-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1234") and then run
> the tool on that port (1234).

Well, I'll check it out...
Comment 21 Chiana 2011-01-27 02:48:22 UTC
Output:
C:\java\jdk1.6.0_23>java -classpath p:/ZipFileMonitor.jar;C:\java\jdk1.6.0_23/li
b/tools.jar util.ZipFileMonitor -host natoth -port 9009 -filter RecieteLib.jar
ZipFileMonitor(18-Mar-2007)
Can use instance filters: true
java.lang.NullPointerException
        at util.ZipFileMonitor$JarFileClassInfo.getTargetConstructorLocation(Zip
FileMonitor.java:454)
        at util.ZipFileMonitor.addJarFileConstructorBreakpoint(ZipFileMonitor.ja
va:284)
        at util.ZipFileMonitor.setBreakpoints(ZipFileMonitor.java:271)
        at util.ZipFileMonitor.run(ZipFileMonitor.java:100)
        at util.ZipFileMonitor.main(ZipFileMonitor.java:75)

And then NB continued startup.
Comment 22 Chiana 2011-01-27 14:16:09 UTC
I've been thinking (again). NetBeans contains JavAssist, doesn't it?

Then one might make a NBM-hack that intercepts call to specific methods making them call a method of your own choice, then you wont have to hack the runtime/systemlibraries, and also it can be used without making any modifications to the original IDE...

Just a thought I had...
Comment 23 Chiana 2011-02-06 03:47:43 UTC
Now, this probable is going to have some effect on my NetCAT points, I've been buzy trying to find a method to track this down, so I wrote a little debugger-stub, put it in a NB module and started the thing up... The following looks like some type of stacktrace, and that is because it is...
WARNING [global]: Dump of "P:\reciete4\RecieteLib\dist\RecieteLib.jar"
WARNING [global]: VM unique object id:14358
WARNING [global]: STACKDUMP
WARNING [global]: ZipFile.java <init> 114
WARNING [global]: JarFile.java <init> 135
WARNING [global]: JarFile.java <init> 72
WARNING [global]: URLClassPath.java getJarFile 646
WARNING [global]: URLClassPath.java access$600 540
WARNING [global]: URLClassPath.java run 607
WARNING [global]: AccessController.java doPrivileged -1
WARNING [global]: URLClassPath.java ensureOpen 599
WARNING [global]: URLClassPath.java <init> 583
WARNING [global]: URLClassPath.java run 333
WARNING [global]: AccessController.java doPrivileged -1
WARNING [global]: URLClassPath.java getLoader 322
WARNING [global]: URLClassPath.java getLoader 299
WARNING [global]: URLClassPath.java access$000 60
WARNING [global]: URLClassPath.java next 195
WARNING [global]: URLClassPath.java hasMoreElements 206
WARNING [global]: URLClassLoader.java run 416
WARNING [global]: AccessController.java doPrivileged -1
WARNING [global]: URLClassLoader.java next 413
WARNING [global]: URLClassLoader.java hasMoreElements 438
WARNING [global]: CompoundEnumeration.java next 27
WARNING [global]: CompoundEnumeration.java hasMoreElements 36
WARNING [global]: Service.java hasNext 255
WARNING [global]: JavacProcessingEnvironment.java hasNext 458
WARNING [global]: JavacProcessingEnvironment.java atLeastOneProcessor 287
WARNING [global]: JavaCompiler.java initProcessAnnotations 875
WARNING [global]: JavaCompiler.java compile 724
WARNING [global]: Main.java compile 353
WARNING [global]: Main.java compile 279
WARNING [global]: Main.java compile 270
WARNING [global]: Main.java compile 69
WARNING [global]: NativeMethodAccessorImpl.java invoke0 -1
WARNING [global]: NativeMethodAccessorImpl.java invoke 39
WARNING [global]: DelegatingMethodAccessorImpl.java invoke 25
WARNING [global]: Method.java invoke 597
WARNING [global]: Javac13.java execute 56
WARNING [global]: Javac.java compile 1134
WARNING [global]: Javac.java execute 912
WARNING [global]: JavacTask.java execute 136
WARNING [global]: UnknownElement.java execute 291
WARNING [global]: DelegatingMethodAccessorImpl.java invoke 25
WARNING [global]: Method.java invoke 597
WARNING [global]: DispatchUtils.java execute 106
WARNING [global]: Task.java perform 348
WARNING [global]: Sequential.java execute 68
WARNING [global]: UnknownElement.java execute 291
WARNING [global]: DelegatingMethodAccessorImpl.java invoke 25
WARNING [global]: Method.java invoke 597
WARNING [global]: DispatchUtils.java execute 106
WARNING [global]: Task.java perform 348
WARNING [global]: MacroInstance.java execute 398
WARNING [global]: UnknownElement.java execute 291
WARNING [global]: DelegatingMethodAccessorImpl.java invoke 25
WARNING [global]: Method.java invoke 597
WARNING [global]: DispatchUtils.java execute 106
WARNING [global]: Task.java perform 348
WARNING [global]: Target.java execute 390
WARNING [global]: Target.java performTasks 411
WARNING [global]: Project.java executeSortedTargets 1399
WARNING [global]: Project.java executeTarget 1368
WARNING [global]: DefaultExecutor.java executeTargets 41
WARNING [global]: Project.java executeTargets 1251
WARNING [global]: BridgeImpl.java run 284
WARNING [global]: TargetExecutor.java run 539
WARNING [global]: RunClassThread.java run 154
INFO [org.netbeans.core.ui.sampler.SelfSamplerAction]: Slowness detector disabled - running in debug mode

This is the callstack of the VM that opened the file that was never closed (yes, I wrote it to messages.log with getLogger.warning...) and consists of <filename><methodname><linenumber>
I will upload the module too, but that is an UGLY, hard to use, hack.
Comment 24 Chiana 2011-02-06 04:04:45 UTC
Created attachment 105682 [details]
NB project

This is how it works;
Puts a breakpoint in ZipFile(File,int) and one in close() of java.util.zip.ZipFile. For each call to ZipFile(.. it records the filename and the callstack in a HashTable using the object unique id from the VM. For each call to close it removes the entry from the table, also it skips certain path's (see catcher.java) that are of no interrest for this. It also have simple window that allows you to see what zip files are open (.jar files are actually .zip files) and also allows you to select one entry and dump the stack from the open call into messages.log.

This is a MESSY hack that sometimes just hangs NB during startup but for this case it was useful, also you cannot use the normal debugger while this thing is running...
Comment 25 Chiana 2011-02-06 04:10:21 UTC
Whops, almost forgot, the tool was run on Nightly Build 201102050000
Comment 26 David Konecny 2011-02-08 22:56:32 UTC
Thanks very much for your help Chiana! The stacktrace contains Javac and so I'm passing it to Java team.
Comment 27 Chiana 2011-02-08 23:07:37 UTC
(In reply to comment #26)
> Thanks very much for your help Chiana! The stacktrace contains Javac and so I'm
> passing it to Java team.

You are most welcome, can you also pass on that I saw several occations where the same file were opened more than once, that is the one that does not close and another that closes as it should.
Comment 28 Jiri Kovalsky 2011-02-09 13:10:27 UTC
(In reply to comment #23)

> Now, this probable is going to have some effect on my NetCAT points, I've been
> buzy trying to find a method to track this down, so I wrote a little
> debugger-stub

Sure, you earned 5 extra CAT points Kent. Thanks a lot for your extra work!
Comment 29 Dusan Balek 2011-02-09 13:29:59 UTC
Known bug in the javac compiler already fixed in JDK 7 (b71). The problem is in java.net.URLClassLoader. The compiler creates a URLClassLoader so that it can load any necessary annotation processors. It would appear that the URLClassLoader is holding on to the jar files, until it is garbage-collected. For more information see http://bugs.sun.com/view_bug.do?bug_id=6558476.

Could be workarounded by forking javac compilation task to a separate VM.
Comment 30 Tomas Zezula 2011-02-09 13:34:18 UTC
Forking the JDK even for default platform is the only possible solution :-(
Comment 31 Chiana 2011-02-09 13:52:25 UTC
(In reply to comment #30)
> Forking the JDK even for default platform is the only possible solution :-(

Correct me if I'm wrong, but does not the builder just create one instance of the compiler and then reuses that througout the entire build? One way in that case it to throw that compiler into the null-bucket and force a GC between project compilations, it will prolong the build but atleast it will finish...
Comment 32 Tomas Zezula 2011-02-09 14:19:29 UTC
You are right, but it's not cause of the problem. The Javac is gced but the ClassLoader is not closed so the jar files stays opened until the application which opened them ends. When the javac in not forked it runs in the ant VM the and runs in NB VM => The jars are opened until the NB is closed. :-(

The problem is that in JDK 1.6 the URLClassPath.JarLoader has no close method so when created it's never closed. In the open jdk there is already a close method and javac (with fix of issue 6558476) closes it.
Comment 33 Chiana 2011-02-09 14:55:39 UTC
(In reply to comment #32)
> You are right, but it's not cause of the problem. The Javac is gced but the
> ClassLoader is not closed so the jar files stays opened until the application
> which opened them ends. When the javac in not forked it runs in the ant VM the
> and runs in NB VM => The jars are opened until the NB is closed. :-(
> The problem is that in JDK 1.6 the URLClassPath.JarLoader has no close method
> so when created it's never closed. In the open jdk there is already a close
> method and javac (with fix of issue 6558476) closes it.

Dang!

Then I guess forking a separate VM in the case of 1.6 is the only viable option...

Just wondering, why does this only show in this case (javaEE) and not in other?
I have several other projs with externally compiled libraries and this has never happened to them...
Comment 34 Tomas Zezula 2011-02-09 15:21:39 UTC
This happens even in j2se projects but only in case that the projects have annotation processors. J2EE projects have them much often. The URLClassLoader is used by javac to load the processor(s). In case when there is no annotation processor the javac does not use the URLClassLoader at all. Even internally in NB's internal javac used by editor and java infrastructure we use custom ClassLoader not URLClassLoader.
Comment 35 Tomas Zezula 2011-02-09 15:23:21 UTC
Fixed for j2seproject c8d0e69c04dc.
The same fix needs to be applied in j2ee and web.
Reassigning to web.
Comment 36 David Konecny 2011-02-09 20:10:17 UTC
Thanks for help Tomas. Fixed for EE project types: 47d2d5b4d3a0
Comment 37 Quality Engineering 2011-02-10 11:31:12 UTC
Integrated into 'main-golden', will be available in build *201102100500* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/c8d0e69c04dc
User: Tomas Zezula <tzezula@netbeans.org>
Log: #188514:NetBeans retains lock on JEE Endorsed Libraries when project is undeployed
Comment 38 Chiana 2011-02-11 00:03:31 UTC
Seems this made it into ß2C2, verified.
Comment 39 Quality Engineering 2011-02-16 11:34:41 UTC
Integrated into 'main-golden', will be available in build *201102160501* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/47d2d5b4d3a0
User: David Konecny <dkonecny@netbeans.org>
Log: #188514 - NetBeans retains lock on JEE Endorsed Libraries when project is undeployed