This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 105234 - Memory leak - regression between 200705300000 & 200705310000
Summary: Memory leak - regression between 200705300000 & 200705310000
Status: CLOSED FIXED
Alias: None
Product: ide
Classification: Unclassified
Component: Performance (show other bugs)
Version: 3.x
Hardware: All All
: P1 blocker (vote)
Assignee: issues@performance
URL:
Keywords: PERFORMANCE
: 105358 105372 105776 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-05-31 10:11 UTC by Marian Mirilovic
Modified: 2011-05-25 11:37 UTC (History)
7 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
JProfiler Memory View - While class path Scanning (336.75 KB, application/octet-stream)
2007-06-05 18:19 UTC, Winston Prakash
Details
JProfiler Memory View 2 - Telemetry view (226.05 KB, application/octet-stream)
2007-06-05 18:19 UTC, Winston Prakash
Details
JProfiler Memory View 3 - After scanning completes (350.29 KB, application/octet-stream)
2007-06-05 18:20 UTC, Winston Prakash
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marian Mirilovic 2007-05-31 10:11:27 UTC
We've found serious memory leak in IDE by performance tests. After following
test cases tests are failing because "Java Heap Space" ...
- Expand nodes (project, source, package, folder)
- Open java, txt, properties file
- open form file -> after this test case OOME rises

All these test cases work well with build 200705300000, so this is regression in
build 200705310000
Comment 1 Petr Nejedly 2007-06-01 08:24:10 UTC
For some reason, the IDE tries to open all the projects it finds in your NBcvs
checkout. This can be something between 500-900 projects, which is well beyound
what our project system can handle, given the default heap size.
In case you can reproduce it that easily, it would be trivial to add some
logging to find out why are the projects opened.

It is possible that the IDE did opened many projects before under the hood and
they are just not getting collected now. The projects are being held in memory
by some listener on a shared Lookup$Result.
Comment 2 _ rkubacki 2007-06-04 15:04:50 UTC
The most interesting commit is adding of dirchooser. OTOH I am not aware of any
usage of this in perf tests.
Comment 3 Petr Hrebejk 2007-06-04 15:33:10 UTC
I think that in my case. I did not have dirchooser in it. But I'm not sure.
Comment 4 Marian Mirilovic 2007-06-04 15:38:35 UTC
Nope.

Radim & Petr, 
I have a heap-dump :

Ordered by number of references : String, HashMap$Entry, char[] ... all >140k

Order by size :
byte[] (54M - 48%)
char[] (19M - 17%)
int[]/String (4MB - 3%)

from nb source base :
org.openide.text.PositionRef$Manager$ChainItem - 18k instances
org.netbeans.editor.MultiMark - 16k instances
Comment 5 Petr Nejedly 2007-06-05 10:52:32 UTC
Marian,
your case seems to be slightly different, as the rest of the heap dumps I have
are dominated by Strings, char[] and HashMap$Entry's, but not byte[].
I'll look at your heap dump.
Comment 6 Petr Nejedly 2007-06-05 18:09:26 UTC
*** Issue 105776 has been marked as a duplicate of this issue. ***
Comment 7 Petr Nejedly 2007-06-05 18:11:26 UTC
Tom, did you added the logging we agreed on into the project manager?
How is it activated?
Comment 8 Winston Prakash 2007-06-05 18:18:32 UTC
Since 105776 is marked as duplicate of this issue, I'm attaching the JProfiler
pictures here.
Comment 9 Winston Prakash 2007-06-05 18:19:11 UTC
Created attachment 43262 [details]
JProfiler Memory View - While class path Scanning
Comment 10 Winston Prakash 2007-06-05 18:19:56 UTC
Created attachment 43263 [details]
JProfiler Memory View 2 - Telemetry view
Comment 11 Winston Prakash 2007-06-05 18:20:41 UTC
Created attachment 43264 [details]
JProfiler Memory View 3 - After scanning completes
Comment 12 Petr Nejedly 2007-06-05 18:43:36 UTC
I have an additional working hypothesis:
Let's assume (a) that the other projects always used to be opened temporarily
(ephemerally) one after one during some operation.
Now, as most of the projects I do find in the heap dumps of affected IDEs are
reachable through a listener on shared Lookup result (b), it is well possible
that they were used to be also collected immediately one after one.

Now, while both of the conditions are pretty suspicious alone, together they
have huge impact.

I'm not sure about (a) though, so we need to find a call that leads to loading
all of the projects anyway.

I can easily verify (b) alone though by opening few projects and closing most of
them. Runtime watches track project instances...
Comment 13 Winston Prakash 2007-06-05 18:56:27 UTC
FYI, I'm not opening several module projects. Just only one project is opned i.e
"visualweb/insync". That is why this huge memory consumption surprised me.
Comment 14 Petr Nejedly 2007-06-05 19:10:44 UTC
But your project have dependencies, which are opened behind-the-scenes as well,
AFAIK. 
Comment 15 Tomas Pavek 2007-06-06 10:33:49 UTC
*** Issue 105372 has been marked as a duplicate of this issue. ***
Comment 16 Tomas Pavek 2007-06-06 10:41:01 UTC
*** Issue 105358 has been marked as a duplicate of this issue. ***
Comment 17 pzajac 2007-06-06 12:47:49 UTC
We found another one  problem during adding dependency in nb.org module project. 
Steps to reproduce:
1) open a module project from NetBeans cvs
2) Invoke add module dependency action, a dialog is shown
3) All netbeans modules are sceanned, it took more then 140MB on heap. 
4) Close the dialog.

The project's were not GC-ed.    
Comment 18 Petr Nejedly 2007-06-06 14:17:32 UTC
The leak seems to be caused by a lookup listener in
projects/projectapi/src/org/netbeans/spi/project/support/LookupProviderSupport.java

The lookup that contains LookupProviders seems to be shared between the projects
and so is the Result. Individual LookupProviderSupport instances listen strongly
on the result and keep themself the project in memory through its baselookup.
I'll try a fix and write more...
Comment 19 Petr Nejedly 2007-06-06 14:23:50 UTC
So my hypothesis was right.
I just fixed the leak:
projects/projectapi/src/org/netbeans/spi/project/support/LookupProviderSupport.java,v1.9

Can somebody write a regression test for this? The setup might be nontrivial.
Test scenario would be:
1) Create one LPS, ask it for something, keep the instance.
2) Create another LPS, ask it for something
3) Check reachability of second LPS.
 
Comment 20 Marian Mirilovic 2007-06-19 13:23:47 UTC
verified
Comment 21 Winston Prakash 2007-06-19 21:03:00 UTC
The issue I reported (105776) has been marked as duplicate of this issue.
But I still see the problem. The memory hikes up to 310 MB when I first 
open a netbeans module project with a clean user dir. It is impossible to
open the module project with default netbeans memory settings. Should I 
reopen 105776.