This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 99895 - Memory leak in ProjectsRootNode.ProjectChildren
Summary: Memory leak in ProjectsRootNode.ProjectChildren
Status: RESOLVED WORKSFORME
Alias: None
Product: projects
Classification: Unclassified
Component: Generic Infrastructure (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Jesse Glick
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2007-04-03 23:00 UTC by Jesse Glick
Modified: 2007-09-27 18:18 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2007-04-03 23:00:38 UTC
In a dev build, I had some NBM projects open, then I closed them. I went away,
did some stuff, did a GC (still no projects open), and found heap still around
60 Mb. After taking a heap dump, I found that the projects were still loaded.
(13 NBM, 1 freeform.) jhat reports that PRN.PC is holding onto them - why, I'm
not sure.

Sample root ref graph (excluding weak references):

  Static reference from org.netbeans.modules.project.ui.OpenProjectList.INSTANCE
(from class org.netbeans.modules.project.ui.OpenProjectList) :                 
           
                                                                               
                                                                               
            
   --> org.netbeans.modules.project.ui.OpenProjectList@0x9c1d90e8 (40 bytes)
(field pchSupport:)                                                            
               
   --> java.beans.PropertyChangeSupport@0x9c1d92a8 (24 bytes) (field listeners:)
                                                                               
           
   --> sun.awt.EventListenerAggregate@0x9c1d92c0 (12 bytes) (field
listenerList:)                                                                 
                         
   --> [Ljava.beans.PropertyChangeListener;@0x9ec391b0 (152 bytes) (Element 5 of
[Ljava.beans.PropertyChangeListener;@0x9ec391b0:)                              
           
   -->
org.netbeans.modules.project.ui.ProjectsRootNode$ProjectChildren@0x9c254970 (45
bytes) (field array:)                                                          
     
   --> org.openide.nodes.Children$1@0x9e784d98 (32 bytes) (field referent:)    
                                                                               
            
   --> org.openide.nodes.ChildrenArray@0x9cba5208 (24 bytes) (field map:)      
                                                                               
            
   --> java.util.WeakHashMap@0x9cba5220 (44 bytes) (field table:)              
                                                                               
            
   --> [Ljava.util.WeakHashMap$Entry;@0x9d942cb0 (136 bytes) (Element 4 of
[Ljava.util.WeakHashMap$Entry;@0x9d942cb0:)                                    
                 
   --> java.util.WeakHashMap$Entry@0x9d9430c0 (36 bytes) (field value:)        
                                                                               
            
   --> java.util.LinkedList@0x9d9430e8 (20 bytes) (field header:)              
                                                                               
            
   --> java.util.LinkedList$Entry@0x9d943100 (20 bytes) (field previous:)      
                                                                               
            
   --> java.util.LinkedList$Entry@0x9d943118 (20 bytes) (field element:)       
                                                                               
            
   --> org.netbeans.modules.project.ui.ProjectsRootNode$BadgingNode@0x9d921258
(61 bytes) (field original:)                                                   
             
   --> org.netbeans.modules.project.ui.PhysicalView$GroupNode@0x9d920968 (74
bytes) (field project:)                                                        
               
   --> org.netbeans.modules.apisupport.project.NbModuleProject@0x9d570338 (36
bytes)                                                                         
              
 
Note that the root refs seem to be split about 50-50 between two instances of
PRN.PC, which I guess means Projects and Files tabs.

The refs look normal if the projects were still open - which they were not (and
they were not displayed in the Projects tab). From inspecting the heap dump, it
seems that the ChildrenArray.map has a 32-slot array of which 11 slots are
filled. Bug in children perhaps? I would expect that there would only be one
active Children.Info object.
Comment 1 Petr Nejedly 2007-04-04 00:38:32 UTC
> From inspecting the heap dump, it seems that the ChildrenArray.map has
> a 32-slot array of which 11 slots are filled. Bug in children perhaps? 

Not really.
The map is weak and it expects that leftovers will drop off themselves.
But in case there is a loop from value to the key (a common bug when dealing
qwith WHM), it will not happen.
I'll look at this tomorrow, hopefully reproducing it myself, so I could trace
the path between the value (in fact the PRN$WN or something else in the list)
and its corresponding key.
Comment 2 Jesse Glick 2007-04-04 18:14:30 UTC
Only one WHM.E had a non-null referent:
org.openide.nodes.Children$Info@0x9cba51f0. The value was an empty list. The
other entries had as their values LinkedList's containing one ProjectBadgingNode
apiece. I suppose the other entries ought to have been cleared out of the map,
but they weren't, even after clicking on the GC button multiple times (which is
supposed to also run finalizations).
Comment 3 Petr Nejedly 2007-04-04 20:00:25 UTC
Sure. I already realized this, just haven't time to add the note here:
Another problem with WHM is that even if the keys are freed, the values (and
WHM$Es) stick there up to next WHM access (get or mutation) - there is a
ReferenceQueue associated with the entries which is checked on every operation
but never asynchronously (no thread or finalizer).
Opening another project should free the old items, but it is still far from OK.
 
Comment 4 Jesse Glick 2007-04-04 22:25:02 UTC
In this case, maybe Children* is still at fault. I certainly don't see a problem
in the project system; it is called setKeys(Collections.emptySet()); there is
nothing more it can do.

Children.Info.finalize() is supposed to call ChildrenArray.finalizeNodes, which
calls remove(null) in an attempt to clear out the entries with collected keys.
So why isn't this working?
Comment 5 Jesse Glick 2007-04-05 00:48:56 UTC
I think it's worse than that. As far as I can tell, the projects don't get
collected at all. I had four NBM projects and a freeform project open. I closed
them, waited fifteen seconds, did a GC. I opened an unrelated j2seproject, did a
GC, closed it, waited, did a GC, etc. Still, I can see that a number of projects
are loaded, including projects that I have not been working on for hours now:

$ jmap -histo 15752|egrep 'Project$'|fgrep -v action
1379:        16         640  org.netbeans.modules.apisupport.project.NbModuleProject
1386:         5         640  org.apache.maven.project.MavenProject
2880:         1          80  org.netbeans.modules.java.j2seproject.J2SEProject
4330:         1          24  org.netbeans.modules.ant.freeform.FreeformProject

That's 23 nonopen projects loaded! And my standing heap is 123 Mb. Not good.
Raising priority since we seem to have a serious and (by me) reproducible leak.
Comment 6 Milos Kleint 2007-04-05 10:25:03 UTC
this sounds interesting, but is probably unrelated.
1386:         5         640  org.apache.maven.project.MavenProject
this class is not the actual project type in netbeans but the underlying maven
object holding the maven project definition. The maven project type class is
named NbMavenProject an is missing from your list, so it's being sucessfully
GCed. The mistery remains who holds these MavenProject instances and why.
Comment 7 Petr Nejedly 2007-04-05 16:12:47 UTC
Jesse,
I still can't reproduce the problem you describe.
There were few related leaks though.
uilogger used to keep closed projects in memory from the event descriptor (event
was: close those 12 projects, so the references sneaked in). Fixed today.
Navigator seems to reference the very first opened file which in turn probably
references its project (issue 100122).
There were more editor leaks (reported and fixed recently) that might have kept
their projects in memory as well.
So in which build do you see the problem? Can you reproduce it with newest
(tomorrows) build?
 
Comment 8 Petr Nejedly 2007-04-06 21:16:11 UTC
BTW: Your observation may be related to issue 100245 (in case you've got VWP
instaled).
Comment 9 Jesse Glick 2007-04-06 21:54:07 UTC
I do not run VWP.
Comment 10 Jesse Glick 2007-04-06 22:49:24 UTC
Can't reproduce today. I filed #100263 for another leak. :-) Then after working
around that, I closed all my projects and checked again. This time I still have
many projects in memory... but JHAT reports no strong root references. So I am
not sure why they are not being collected.

BTW the OQL query

select heap.livepaths(p) from instanceof org.netbeans.api.project.Project p

doesn't seem to work; it seems that instanceof does not work on interfaces??
Comment 11 Petr Nejedly 2007-04-23 10:57:43 UTC
BTW: you can try Timers/counters, node "Important instances->Project" and "find
refs" to find directly from the running IDE.
Comment 12 Antonin Nebuzelsky 2007-08-27 15:39:28 UTC
Reassigning to Radim.
Comment 13 Petr Nejedly 2007-09-21 16:02:48 UTC
Still reproducible?
BTW: there's common misconception of Children.removeNotify, where many people tend to do cleanup of listeners and other
things which usually prevent the children nodes from being freed - so removeNotify is never called and the structure
leaks wholesale. But that's probably not the case here.
Comment 14 Jesse Glick 2007-09-26 18:17:50 UTC
My understanding was that Children.removeNotify should be called when the node has been collapsed for a while, or is no
longer displayed at all, even if the Children object is still strongly referenced. Not true? If so, there are going to
be a _lot_ of broken Children subclasses out there.
Comment 15 Petr Nejedly 2007-09-26 20:50:28 UTC
Well, that's actually the case.
It seems I'll need to add a big fat warning into the removeNotify() javadoc.
removeNotify() is basically good just for cleaning up the key objects (which may be expensive) from memory once the
nodes themselves are freed.
Comment 16 Petr Nejedly 2007-09-27 15:10:26 UTC
I have updated the javadoc of Children.removeNotify to warn about the pitfalls.
openide/nodes/src/org/openide/nodes/Children.java,v1.24

Reassigning back.
Comment 17 Jesse Glick 2007-09-27 17:00:36 UTC
I don't see how the Javadoc update to Children.removeNotify applies here. ProjectsRootNode.ProjectChildren does not hold
explicit references to its created subnodes that I can see. So I still suspect a bug in Children code. But need to check
again whether the originally reported problem is reproducible.
Comment 18 Jesse Glick 2007-09-27 18:18:30 UTC
Not reproducible for me any more.

1. Start 070927, fresh userdir, JDK 6, Ubuntu. Pass 'openide/util openide/fs openide/nodes openide/loaders
openide/modules openide/text openide/windows openide/actions openide/awt openide/dialogs' on command line so they get
opened. Accept license, close welcome screen.

2. Wait for CP scanning to finish.

3. File > Project Group > (none) to close all.

4. Wait 30+ seconds, do a full GC. Heap goes from 63 -> 26 Mb. Another GC, -> 21 Mb. Another, still 21 Mb.

5. Get a heap dump using jmap and inspect for NbModuleProject's. None found.