This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 162176 - [67cat] Memory leak in 6.7
Summary: [67cat] Memory leak in 6.7
Status: VERIFIED FIXED
Alias: None
Product: projects
Classification: Unclassified
Component: Libraries (show other bugs)
Version: 6.x
Hardware: Macintosh (x86) Mac OS X
: P2 blocker (vote)
Assignee: Tomas Zezula
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2009-04-07 22:47 UTC by theshadow27
Modified: 2009-04-10 14:37 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Histogram from 200904031400? after 8 hours of opperation (692.01 KB, text/plain)
2009-04-08 04:00 UTC, theshadow27
Details
Histogram from 200904070200 after 3 hours of operation (548.07 KB, text/plain)
2009-04-08 04:01 UTC, theshadow27
Details

Note You need to log in before you can comment on or make changes to this bug.
Description theshadow27 2009-04-07 22:47:43 UTC
There is a potential memory leak in the 6.7 dev builds. After running for over 8 hours, system memory usage increases past the Xmx and PermSize settings, 
and continues to grow at a linear rate. I'm assigning this to projects.libraries at the advice of Tomas P in the "[67cat][other]Major speed slowdown after 5+ 
hours" message thread. I am filing it under x86 Mac and Mac OS X because I can't personally confirm it happens in the other platforms, but if someone can 
feel free to change it.
Comment 1 theshadow27 2009-04-07 22:53:54 UTC
History/Background:

Eric S. 26 MAR 09:
I've had 200903250219 running for about 5 hours doing some significant ruby development and editing and within the last 15 minutes the performance 
has gone through the floor.  Currently i have 7 files open (2 controllers and 5 haml files) of which only 1 controller is not saved.  No servers running (strictly 
editing).  130M/158M memory allocation.

Chris M. 26 MAR 09:
At the start of my day the memory usage is around 40MB. By the end of the
day it's not uncommon for the memory usage to have gone up to 150-200MB so
there's definitely a memory leak in one or more places. I sometimes
double-click on the memory usage meter to force garbage collection and that
drops the current usage from 125+ back down to the 60-80MB range, but it
climbs back up again over time.

Ephraim R. 26 MAR 09:
I am seeing the same behavior with Java development. I left my ide
(200903190219) open overnight and when I came in it grew to 705MB. I
have no app servers running and all I had open were 3 java files in the
editor.

Jacob D. 3 APR 09 
I found the same issue with M3: after leaving it running overnight the memory usage had gone from ~100mb to 600mb. 
Comment 2 theshadow27 2009-04-07 23:07:52 UTC
Data from Milestone 3 (20090401):

Sample 1: Running overnight from 2 APR to 3 APR, System reported memory usage starting at ~100mb, ending at 600mb
http://theeshadow.com/files/m3/heapdump0.hproof (173mb)

Sample 2: Running for 12+ hours, took a histogram (jmap -histo:live PID) then a heapdump (VisualVM). Did not perform GC first, so heapdump is big. 
http://theeshadow.com/files/m3/histogram1.txt
http://theeshadow.com/files/m3/heapdump1.hproof (364mb - no GC before dump)
Comment 3 theshadow27 2009-04-07 23:10:46 UTC
Data from 200904051400:

Sample 1: The IDE was running for 10.01 hours with 6 hours of development and 4 hours of downtime. 

Operating system reported 850.52MB physical usage [1], 1.89GB virtual usage, interesting since the JVM args [2] indicate -Xmx=512m and 
PermSize=200m. I personally have no issue with a 800+ MB JVM, but NetBeans "thinks" it's using 404MB, which is not the case (This might be a JRE 
limitation/flaw and not NB).  More interesting is that according to OS X, the NB JVM had over 900 file handles open [3]. 

As per Tomas's last recomendation, I ran a garbage collection before taking the histogram [4] (jmap -histo:live), heap dump [5] (Visual VM) and application 
profile [6] (Visual VM). 

[1] http://theeshadow.com/files/200904051400/memoryusage0.png (77k)
[2] http://theeshadow.com/files/200904051400/jvmargs0.txt (<4k)
[3] http://theeshadow.com/files/200904051400/openfiles0.txt (<4k)
[4] http://theeshadow.com/files/200904051400/histogram0.hist (629k)
[5] http://theeshadow.com/files/200904051400/heapdump0.hprof (188m) 
[6] http://theeshadow.com/files/200904051400/application0.apps (37m)
Comment 4 theshadow27 2009-04-08 03:56:05 UTC
Notes for build 200904070200
messages2.log [4]:
  Heap memory usage: initial 32.0MB maximum 490.7MB
  Non heap memory usage: initial 34.4MB maximum 248.0MB
Indicates a maximum of 738MB of memory. See datapoints 2,3 
usage > 738mb in 3 hours. 

...

Uptime < 30m,  no editing/usage, for memory baseline
histogram1.txt [1] - 07 APR 09 7:25 PM - PID 18822
NB Reports 164MB
Activity Monitor Reports 325MB (Real)
Visual VM 80MB (PermGen), 125MB (Heap), 110MB/341MB (Old Gen)
-- Note, this is what I would expect for NB6.5 on my machine

...

Uptime 3h 12 min, Heavy editing in a single 'medium' project 
histogram2.txt [2]- 07 APR 09 21:51EST - PID 18822
histogram3.txt [3]- 07 APR 09 21:58EST - PID 18822
NB Reports 348MB
Activity Monitor Reports 799MB (Real) 1.48GB (Virtual)
Visual VM 120MB (PermGen), 350MB (Heap), 242MB/341MB (Old Gen)
Some Histo [3] higlights 
20,000 instances of org.netbeans.api.project.libraries.Library
19,892 instances of org.openide.modules.Dependency

 5,061 instances of org.openide.filesystems.MultiFileObject
-- Note for this, there are < 900 open files and ports in openfiles3.txt [6]
See heapdump3.hprof [5] (174mb)

[1] http://theeshadow.com/files/200904070200/histogram1.txt
[2] http://theeshadow.com/files/200904070200/histogram2.txt
[3] http://theeshadow.com/files/200904070200/histogram3.txt
[4] http://theeshadow.com/files/200904070200/messages2.log
[5] http://theeshadow.com/files/200904070200/heapdump3.hprof
[6] http://theeshadow.com/files/200904070200/openfiles3.txt

PS All files are listed publicly http://theeshadow.com/files/200904070200/ 
Comment 5 theshadow27 2009-04-08 04:00:02 UTC
Created attachment 79698 [details]
Histogram from 200904031400? after 8 hours of opperation
Comment 6 theshadow27 2009-04-08 04:01:24 UTC
Created attachment 79699 [details]
Histogram from 200904070200 after 3 hours of operation
Comment 7 Tomas Zezula 2009-04-08 13:17:18 UTC
I've found a gc root through SFBQImpl cache, LibraryImpl listener, Listener.
I've removed the hard reference in LibraryImpl listener.
Hopefully it was the only one.
Thanks for the heapdump file, it was really useful.
Fixed jet-main:daaf82db3bb0
Comment 8 Quality Engineering 2009-04-09 19:36:35 UTC
Integrated into 'main-golden', will be available in build *200904091401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/daaf82db3bb0
User: Tomas Zezula <tzezula@netbeans.org>
Log: #162176:[67cat] Memory leak in 6.7
Comment 9 theshadow27 2009-04-10 14:37:55 UTC
Verified fixed in 2009 04 09 1401 

Ran for 10 hours, with projects open (same as before). At the end, was using exactly Xmx + perm gen (around 630mb, 512+128). Thanks!

BTW here is a heap dump at the end of the above test, if you want to check to make sure what you fixed worked:

http://theeshadow.com/files/200904091401/heapdump-1239368782505.hprof