Bug 185350 - Profiled program with exec() calls returns OutOfMemory in Linux
Profiled program with exec() calls returns OutOfMemory in Linux
Status: RESOLVED FIXED
Product: profiler
Classification: Unclassified
Component: Engine
6.x
All All
: P3 (vote)
: 6.x
Assigned To: Tomas Hurka
issues@profiler
: VISUALVM
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-05-03 11:21 UTC by jcchak
Modified: 2010-05-26 10:52 UTC (History)
0 users

See Also:
Issue Type: DEFECT
:


Attachments
Simple test program that uses exec() infinite times (730 bytes, text/x-java)
2010-05-03 11:21 UTC, jcchak
Details
Screenshot of the test run (107.44 KB, image/png)
2010-05-03 12:26 UTC, Tomas Hurka
Details
Screenshot after the crash (164.29 KB, image/png)
2010-05-03 15:20 UTC, jcchak
Details
Heap dump (pid7988) (1.26 MB, application/zip)
2010-05-03 15:32 UTC, jcchak
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jcchak 2010-05-03 11:21:36 UTC
Created attachment 98358 [details]
Simple test program that uses exec() infinite times

Hi,

We have found that when a program contains an exec() and you use this method several times, the Netbeans Profiler in Linux seems to interfere somehow in the program that in the end, it crashes with a OutOfMemory error.

In the VM Telemetry Overview we can see that the number surviving generations of some objects doesn't stop to increase during all the execution time, so at the end there isn't free heap memory.

We have runned a simple test program with several different versions of JVM (the last one has been Java 6 update 20) and when we use the Netbeans Profiler in Linux we always get this error. But when running in a terminal or inside Netbeans (without the Profiler), the program doesn't have this problem and is able to run "to infinity".

Futhermore, the same test program (but just changing the "ls" command for a "cmd /c dir") in the Netbeans Profiler in Windows doesn't have the same behaviour, because the number of surviving generations is stable during all the execution time.

We attach you the test program we have made, so you can check that there isn't any code that can provocate a memory leak. The settings we use for the Profiler were:

- Record both object creation and garbage collection.
- Track every 100 object allocations.
- Record stack trace for allocations.

But it seems that you can use any kind of settings because this doesn't affect to this bug.

We have write a post in Oracle's forum with our background history: http://forums.sun.com/thread.jspa?messageID=10982870

The Netbeans build we are using is NetBeans IDE 6.8 (Build 200912041610), but we have used also the last beta of Netbeans 6.9 and we still get the OutOfMemory error.


Thanks in advance.
Comment 1 Tomas Hurka 2010-05-03 12:24:37 UTC
I am sorry, but I am unable to reproduce the problem. I was able to profile your test application without any apparent memory leak. See attached screenshot. I tested it on FedoraCore 12, NB 6.8,  Sun JDK 1.6.0_19.
Comment 2 Tomas Hurka 2010-05-03 12:26:18 UTC
Created attachment 98363 [details]
Screenshot of the test run
Comment 3 jcchak 2010-05-03 12:40:27 UTC
(In reply to comment #1)
> I am sorry, but I am unable to reproduce the problem. I was able to profile
> your test application without any apparent memory leak. See attached
> screenshot. I tested it on FedoraCore 12, NB 6.8,  Sun JDK 1.6.0_19.

Hi Thomas,

You have reproduced the problem! You just have to leave it more time to get the error.

The behaviour of the program in Linux is totally different than in Windows. In Windows the number of surviving generations is stable, but at Linux all the time is increasing that number. In your screenshot the number is about more than 600, but it can go up to 10000. And under this number of SG, the heap collapses.

Just try to leave it more time, please. As I said, if you try the same program but without the profiler, you don't get the OutOfMemory error, so that's why I think that there is something in the Profiler.


Thanks in advance.
Comment 4 jcchak 2010-05-03 12:44:08 UTC
Id like to clarify that it's the test program that in the end crashes with an OutOfMemory error, not Neatbeans or the Profiler. But it's something that just happens with the Netbeans Profiler in Linux and I'd like to know why.


Thanks in advance.
Comment 5 Tomas Hurka 2010-05-03 13:00:25 UTC
I do not think I reproduced the problem. If you take a look at memory graph - there is no sign of memory leak. Also if you take a look at live results - (they are sorted by generation count) - there is also no visible memory leak. The global surviving generation count is not very precise, so you can see it going high. I can leave it running for some time if you want. Can you post the exception from the out-of-memory-error?
Comment 6 jcchak 2010-05-03 15:18:51 UTC
(In reply to comment #5)
> I do not think I reproduced the problem. If you take a look at memory graph -
> there is no sign of memory leak. Also if you take a look at live results -
> (they are sorted by generation count) - there is also no visible memory leak.
> The global surviving generation count is not very precise, so you can see it
> going high. I can leave it running for some time if you want. Can you post the
> exception from the out-of-memory-error?

Hi Tomas,

First of all, thanks for your attention!

The exception that we get is as follows:

testFastExec
java.lang.OutOfMemoryError: GC overhead limit exceeded
Dumping heap to /root/Projects/PENGUIN_TEST/nbproject/private/profiler/java_pid7988.hprof ...
Heap dump file created [9085144 bytes in 0.142 secs]
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.lang.Runtime.exec(Runtime.java:427)
        at java.lang.Runtime.exec(Runtime.java:328)
        at processExecutorTest.testFastExec.main(testFastExec.java:24)
Profiler Agent: Connection with agent closed
Profiler Agent: Connection with agent closed

I have configured the VM with this settings "-Xms5m -Xmx2500k" to get the error as soon as possible. Also I have modified a little bit the program to catch and print the Exception.
Comment 7 jcchak 2010-05-03 15:20:51 UTC
Created attachment 98376 [details]
Screenshot after the crash
Comment 8 Tomas Hurka 2010-05-03 15:23:34 UTC
Can you please zip and attach the heap dump  /root/Projects/PENGUIN_TEST/nbproject/private/profiler/java_pid7988.hprof
Comment 9 jcchak 2010-05-03 15:32:26 UTC
Created attachment 98378 [details]
Heap dump (pid7988)
Comment 10 Tomas Hurka 2010-05-06 11:23:49 UTC
I found out that there is about 100 bytes leak for every started thread. Therefor you will see it only if application generates a lot of short-living threads and have very small heap.
Comment 11 Tomas Hurka 2010-05-07 15:45:13 UTC
Fixed in profiler-main
changeset:   170004:bd544011413a
user:        Tomas Hurka <thurka@netbeans.org>
date:        Fri May 07 14:56:11 2010 +0200
summary:     bugfix #185350, extend threadInfos array only if there are no dead threads stored in threadInfos
Comment 12 jcchak 2010-05-07 15:58:59 UTC
(In reply to comment #11)
> Fixed in profiler-main
> changeset:   170004:bd544011413a
> user:        Tomas Hurka <thurka@netbeans.org>
> date:        Fri May 07 14:56:11 2010 +0200
> summary:     bugfix #185350, extend threadInfos array only if there are no dead
> threads stored in threadInfos

Hi Tomas,

Thanks for the fix and for your time.


Cheers.
Comment 13 Quality Engineering 2010-05-10 09:20:59 UTC
Integrated into 'main-golden', will be available in build *201005100200* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/
User: 
Log:


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo