This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 185350

Summary:	Profiled program with exec() calls returns OutOfMemory in Linux
Product:	profiler	Reporter:	jcchak <jcchak>
Component:	Engine	Assignee:	Tomas Hurka <thurka>
Status:	RESOLVED FIXED
Severity:	normal	Keywords:	VISUALVM
Priority:	P3
Version:	6.x
Hardware:	All
OS:	All
Issue Type:	DEFECT	Exception Reporter:
Attachments:	Simple test program that uses exec() infinite times Screenshot of the test run Screenshot after the crash Heap dump (pid7988)

Description jcchak 2010-05-03 11:21:36 UTC

Created attachment 98358 [details]
Simple test program that uses exec() infinite times

Hi,

We have found that when a program contains an exec() and you use this method several times, the Netbeans Profiler in Linux seems to interfere somehow in the program that in the end, it crashes with a OutOfMemory error.

In the VM Telemetry Overview we can see that the number surviving generations of some objects doesn't stop to increase during all the execution time, so at the end there isn't free heap memory.

We have runned a simple test program with several different versions of JVM (the last one has been Java 6 update 20) and when we use the Netbeans Profiler in Linux we always get this error. But when running in a terminal or inside Netbeans (without the Profiler), the program doesn't have this problem and is able to run "to infinity".

Futhermore, the same test program (but just changing the "ls" command for a "cmd /c dir") in the Netbeans Profiler in Windows doesn't have the same behaviour, because the number of surviving generations is stable during all the execution time.

We attach you the test program we have made, so you can check that there isn't any code that can provocate a memory leak. The settings we use for the Profiler were:

- Record both object creation and garbage collection.
- Track every 100 object allocations.
- Record stack trace for allocations.

But it seems that you can use any kind of settings because this doesn't affect to this bug.

We have write a post in Oracle's forum with our background history: http://forums.sun.com/thread.jspa?messageID=10982870

The Netbeans build we are using is NetBeans IDE 6.8 (Build 200912041610), but we have used also the last beta of Netbeans 6.9 and we still get the OutOfMemory error.


Thanks in advance.

Comment 1 Tomas Hurka 2010-05-03 12:24:37 UTC

I am sorry, but I am unable to reproduce the problem. I was able to profile your test application without any apparent memory leak. See attached screenshot. I tested it on FedoraCore 12, NB 6.8,  Sun JDK 1.6.0_19.

Comment 2 Tomas Hurka 2010-05-03 12:26:18 UTC

Created attachment 98363 [details]
Screenshot of the test run

Comment 3 jcchak 2010-05-03 12:40:27 UTC

(In reply to comment #1)
> I am sorry, but I am unable to reproduce the problem. I was able to profile
> your test application without any apparent memory leak. See attached
> screenshot. I tested it on FedoraCore 12, NB 6.8,  Sun JDK 1.6.0_19.

Hi Thomas,

You have reproduced the problem! You just have to leave it more time to get the error.

The behaviour of the program in Linux is totally different than in Windows. In Windows the number of surviving generations is stable, but at Linux all the time is increasing that number. In your screenshot the number is about more than 600, but it can go up to 10000. And under this number of SG, the heap collapses.

Just try to leave it more time, please. As I said, if you try the same program but without the profiler, you don't get the OutOfMemory error, so that's why I think that there is something in the Profiler.


Thanks in advance.

Comment 4 jcchak 2010-05-03 12:44:08 UTC

Id like to clarify that it's the test program that in the end crashes with an OutOfMemory error, not Neatbeans or the Profiler. But it's something that just happens with the Netbeans Profiler in Linux and I'd like to know why.


Thanks in advance.

Comment 5 Tomas Hurka 2010-05-03 13:00:25 UTC

I do not think I reproduced the problem. If you take a look at memory graph - there is no sign of memory leak. Also if you take a look at live results - (they are sorted by generation count) - there is also no visible memory leak. The global surviving generation count is not very precise, so you can see it going high. I can leave it running for some time if you want. Can you post the exception from the out-of-memory-error?

Comment 6 jcchak 2010-05-03 15:18:51 UTC

(In reply to comment #5)
> I do not think I reproduced the problem. If you take a look at memory graph -
> there is no sign of memory leak. Also if you take a look at live results -
> (they are sorted by generation count) - there is also no visible memory leak.
> The global surviving generation count is not very precise, so you can see it
> going high. I can leave it running for some time if you want. Can you post the
> exception from the out-of-memory-error?

Hi Tomas,

First of all, thanks for your attention!

The exception that we get is as follows:

testFastExec
java.lang.OutOfMemoryError: GC overhead limit exceeded
Dumping heap to /root/Projects/PENGUIN_TEST/nbproject/private/profiler/java_pid7988.hprof ...
Heap dump file created [9085144 bytes in 0.142 secs]
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.lang.Runtime.exec(Runtime.java:427)
        at java.lang.Runtime.exec(Runtime.java:328)
        at processExecutorTest.testFastExec.main(testFastExec.java:24)
Profiler Agent: Connection with agent closed
Profiler Agent: Connection with agent closed

I have configured the VM with this settings "-Xms5m -Xmx2500k" to get the error as soon as possible. Also I have modified a little bit the program to catch and print the Exception.

Comment 7 jcchak 2010-05-03 15:20:51 UTC

Created attachment 98376 [details]
Screenshot after the crash

Comment 8 Tomas Hurka 2010-05-03 15:23:34 UTC

Can you please zip and attach the heap dump  /root/Projects/PENGUIN_TEST/nbproject/private/profiler/java_pid7988.hprof

Comment 9 jcchak 2010-05-03 15:32:26 UTC

Created attachment 98378 [details]
Heap dump (pid7988)

Comment 10 Tomas Hurka 2010-05-06 11:23:49 UTC

I found out that there is about 100 bytes leak for every started thread. Therefor you will see it only if application generates a lot of short-living threads and have very small heap.

Comment 11 Tomas Hurka 2010-05-07 15:45:13 UTC

Fixed in profiler-main
changeset:   170004:bd544011413a
user:        Tomas Hurka <thurka@netbeans.org>
date:        Fri May 07 14:56:11 2010 +0200
summary:     bugfix #185350, extend threadInfos array only if there are no dead threads stored in threadInfos

Comment 12 jcchak 2010-05-07 15:58:59 UTC

(In reply to comment #11)
> Fixed in profiler-main
> changeset:   170004:bd544011413a
> user:        Tomas Hurka <thurka@netbeans.org>
> date:        Fri May 07 14:56:11 2010 +0200
> summary:     bugfix #185350, extend threadInfos array only if there are no dead
> threads stored in threadInfos

Hi Tomas,

Thanks for the fix and for your time.


Cheers.

Comment 13 Quality Engineering 2010-05-10 09:20:59 UTC

Integrated into 'main-golden', will be available in build *201005100200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/
User: 
Log: